Skip to content

Stop workers when job finalization fails - #768

Open
rafael-pissardo wants to merge 7 commits into
rails:mainfrom
rafael-pissardo:fix/748-claimed-execution-stuck-on-finished-failure
Open

Stop workers when job finalization fails#768
rafael-pissardo wants to merge 7 commits into
rails:mainfrom
rafael-pissardo:fix/748-claimed-execution-stuck-on-finished-failure

Conversation

@rafael-pissardo

@rafael-pissardo rafael-pissardo commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • wrap finalization failures while a claimed execution is still present in a dedicated error
  • stop the owning worker after that unrecoverable error so shutdown releases the claim for retry
  • keep workers running after ordinary job failures and avoid joining from a pool thread

Fixes #748.

Test plan

  • TARGET_DB=sqlite bundle exec ruby -Itest test/unit/worker_test.rb
  • TARGET_DB=sqlite bundle exec ruby -Itest test/models/solid_queue/claimed_execution_test.rb
  • TARGET_DB=sqlite bundle exec ruby -Itest test/integration/jobs_lifecycle_test.rb
  • RuboCop on changed Ruby files

@rafael-pissardo
rafael-pissardo force-pushed the fix/748-claimed-execution-stuck-on-finished-failure branch from 249f2ec to db4a139 Compare July 24, 2026 20:03
Prevent claimed executions from remaining stuck after transient database errors by terminating the owning worker so shutdown releases the claim.
@rafael-pissardo
rafael-pissardo force-pushed the fix/748-claimed-execution-stuck-on-finished-failure branch from db4a139 to d903a36 Compare July 24, 2026 20:03
rafael-pissardo and others added 6 commits August 10, 2026 22:15
…failure

Adapt FinalizationError worker shutdown to the new Pool / ThreadPool /
FiberPool hierarchy while preserving claim release on finalization failure.

Co-authored-by: Cursor <cursoragent@cursor.com>
Align instrumentation and FiberPool builder specs with current API after
merging main (on_unrecoverable_error) and FinalizationError wrapping.

Co-authored-by: Cursor <cursoragent@cursor.com>
Execute-phase errors should propagate unchanged to thread_error
instrumentation. Reserve FinalizationError for failures while the claim
is still held after finished/failed_with, and align the worker spec with
a real finalization failure stub.

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Count only FinalizationError entries so incidental duplicate error
reports from the execution wrapper do not fail the spec on CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ClaimedExecution can be stuck in claimed state forever when #finished raises

1 participant