Skip to content

[Load] Preserve pending tablets when parser fails during fallback - #18487

Merged
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/load-preserve-pending-tablets
Aug 24, 2026
Merged

jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/load-preserve-pending-tablets

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Problem

When tree-model Load falls back from scan parsing to query parsing, a parser exception can occur after earlier Tablets have been converted but before the pending batch reaches its normal flush boundary. The exception path released the conversion context and discarded that batch.

Fix

Flush pending Tablets before returning a normal parser error. Temporary-unavailable results retain the pending batch and original parser status for the next managed-task retry; memory-pressure and interruption exceptions keep their existing retry behavior.

Tests

  • mvn -pl iotdb-core/datanode -am test -Dtest=LoadTreeStatementDataTypeConvertExecutionVisitorTest -Dsurefire.failIfNoSpecifiedTests=false -DfailIfNoTests=false -DskipITs
  • Spotless check
  • Checkstyle: 0 violations

The separate exact-query versus wildcard-query observation is intentionally out of scope.

@luoluoyuyu

Copy link
Copy Markdown
Member

The idea is reasonable: flush pending tablets before reporting a parser failure so already-converted data isn’t lost. Two things need confirmation though: 1) In the synchronous pipe path, a non-success result is marked retryable by the caller, but this change clears the conversion context after flushing, so a retry can restart from the beginning and re-insert the same tablets; 2) this also affects ordinary LOAD, where a parser failure would now become partial write + error. Consider limiting the flush to receiver-owned/terminal tasks, or mark the task terminal after flushing, and add coverage for the managed path and for retries not duplicating writes.

@jt2594838
jt2594838 merged commit 429587f into apache:master Aug 24, 2026
41 of 42 checks passed
@jt2594838
jt2594838 deleted the fix/load-preserve-pending-tablets branch August 24, 2026 04:14
jt2594838 pushed a commit that referenced this pull request Aug 25, 2026
#18509)

* Load: preserve pending tablets when parser fails (#18487)

(cherry picked from commit 429587f)

* Fix pending tablet load test row count
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants