Skip to content

Fix retry bug in tree model query which ignore the time filter#18179

Open
JackieTien97 wants to merge 4 commits into
masterfrom
fix-retry-bug
Open

Fix retry bug in tree model query which ignore the time filter#18179
JackieTien97 wants to merge 4 commits into
masterfrom
fix-retry-bug

Conversation

@JackieTien97

@JackieTien97 JackieTien97 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Description

Problem

The first query dispatch may fail for a retryable reason. In the reported case, the DataNode hosting the region leader had reached the upper limit of 1,000 internal connections.

QueryExecution then analyzes the same parsed statement again. However, tree-model analysis previously modified parser-owned AST state in place, including extracting the global time predicate from WHERE. As a result, the retry could analyze a statement different from the original SQL and lose its time filter. Other analysis-time mutations, such as normalized expressions, ORDER BY, LIMIT/OFFSET pushdown, and template-specific rewrites, had the same retry-safety risk.

Changes

  • Make global-time-predicate extraction and predicate simplification non-mutating. They now return the extracted time predicate and a residual predicate, path-copying only expression branches that need to change.
  • Add a per-query TreeAnalysisMutationJournal for tree-model analysis:
    • lazily record only parser-owned fields that are actually replaced;
    • use copy-on-write for nested mutable state such as ORDER BY;
    • avoid an unconditional deep copy of the complete tree-query AST.
  • Define the analysis-attempt lifecycle in QueryExecution:
    • begin an attempt before analysis;
    • keep its working state while generated plans and the scheduler still use it;
    • after a retryable dispatch failure, stop and clean up the failed attempt before rolling it back;
    • reanalyze from the original parser-owned state;
    • commit only after the query reaches a successful running or finished state, and roll back on analysis or terminal failures.
  • Keep implicit DEVICE and TIME sort keys for DeviceView in planning-owned state instead of appending them to the parsed QueryStatement.
  • Analyze parser-owned table-model SHOW DEVICES and COUNT DEVICES statements on per-analysis working copies. Internal schema-fetch ShowDevice statements retain their pre-populated traversal state because that state is request input rather than derived analysis data.
  • Intentionally share raw WHERE expressions in table-model working copies. This analysis path treats them as read-only, replaces rewritten roots only on the working statement, and avoids a deep-copy cost on normal one-attempt queries.

These changes ensure that every retry observes SQL semantics equivalent to the original statement while limiting copying to state that analysis actually needs to modify.

Tests

Added regression unit tests covering:

  • repeated analysis preserving tree-query time filters;
  • mixed AND, OR, and NOT predicates without mutating input expressions;
  • retry safety for GROUP BY, HAVING, ORDER BY, SELECT INTO, LIMIT/OFFSET, count_time(*), and template analysis;
  • analysis-attempt commit and rollback behavior for successful retries and exhausted retry limits;
  • two-stage DeviceView and SortNode planning without modifying or accumulating sort keys in the parsed statement;
  • repeated analysis of table-model SHOW DEVICES and COUNT DEVICES;
  • preservation of pre-populated internal ShowDevice traversal state.

@codecov

codecov Bot commented Jul 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 80.50139% with 70 lines in your changes missing coverage. Please review.
✅ Project coverage is 42.60%. Comparing base (c716365) to head (e77abee).
⚠️ Report is 14 commits behind head on master.

Files with missing lines Patch % Lines
.../db/queryengine/plan/execution/QueryExecution.java 61.29% 12 Missing ⚠️
...gine/plan/analyze/TreeAnalysisMutationJournal.java 89.90% 11 Missing ⚠️
...gine/plan/analyze/TemplatedAggregationAnalyze.java 64.00% 9 Missing ⚠️
...ression/visitor/predicate/PredicateSimplifier.java 70.96% 9 Missing ⚠️
...db/db/queryengine/plan/analyze/PredicateUtils.java 85.45% 8 Missing ⚠️
.../db/queryengine/plan/planner/TreeModelPlanner.java 0.00% 8 Missing ⚠️
...db/db/queryengine/plan/analyze/AnalyzeVisitor.java 82.85% 6 Missing ⚠️
...he/iotdb/db/queryengine/plan/planner/IPlanner.java 0.00% 3 Missing ⚠️
.../db/queryengine/plan/analyze/TemplatedAnalyze.java 0.00% 2 Missing ⚠️
...ne/plan/relational/analyzer/StatementAnalyzer.java 80.00% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18179      +/-   ##
============================================
+ Coverage     42.04%   42.60%   +0.56%     
  Complexity      318      318              
============================================
  Files          5312     5323      +11     
  Lines        375102   376815    +1713     
  Branches      48422    48742     +320     
============================================
+ Hits         157693   160548    +2855     
+ Misses       217409   216267    -1142     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant