fix(consul): one invalid node should not discard the remaining nodes of the service#13513
Merged
nic-6443 merged 2 commits intoJun 12, 2026
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR fixes a reliability bug in the Consul service discovery client where a single malformed entry (missing the Service field) could incorrectly skip the rest of the nodes for that service, potentially causing the service to disappear from discovery and break traffic routing.
Changes:
- Make the “skip malformed node” logic local to the per-node loop (so one bad entry doesn’t discard the entire service’s node list) and emit a warning log for the skipped entry.
- Add a regression test that mocks a Consul health response containing one malformed entry followed by valid nodes.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
apisix/discovery/consul/client.lua |
Adjusts node-iteration control flow to skip only malformed nodes and logs a warning for the skipped entry. |
t/discovery/consul-malformed-node.t |
Adds a regression test covering malformed health entries to ensure remaining valid nodes are still used. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
shreemaan-abhishek
approved these changes
Jun 12, 2026
AlinsRan
approved these changes
Jun 12, 2026
membphis
approved these changes
Jun 12, 2026
membphis
left a comment
Member
There was a problem hiding this comment.
Reviewed. No blocking issues found.
wistefan
pushed a commit
to wistefan/apisix
that referenced
this pull request
Jun 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A single malformed entry in the consul health API response (an entry without the
Servicefield, e.g. from an agent on a reclaimed cloud instance) currently wipes out the whole service. The per-node check infetch_services_from_server()doesgoto CONTINUE, but the::CONTINUE::label sits outside the node loop, so one bad entry skips all the remaining nodes, the sort, and theup_services[key] = nodesassignment. If the malformed entry comes first, the service ends up with no nodes,update_all_services()deletes it from the shared dict, and requests start failing with "no valid upstream node".The fix makes the skip local to the node loop (
goto CONTINUE_NODEwith the label at the end of the loop body) and logs a warning for the skipped entry. After the change, a malformed entry only drops that one node and the remaining healthy nodes keep serving traffic.Added a regression test (
t/discovery/consul-malformed-node.t) with a mock consul server returning one malformed entry followed by two valid ones; it fails without the fix and passes with it.Which issue(s) this PR fixes:
Fixes #12937
Checklist