Skip to content

fix(ci): flaky unit tests — prompt-during-run and hook timeout #121

Description

@terisuke

Summary

Two unit tests fail intermittently in CI (GitHub Actions Linux runner), causing unit (linux) job to report failure. These are pre-existing flakes unrelated to any recent PR changes.

Failing Tests

1. prompt submitted during an active run is included in the next LLM input

  • Location: Likely in test/session/ or test/prompt/
  • Failure mode: Timing-dependent — test expects a prompt submitted during an active LLM run to be queued and included in the next input. Race condition between prompt submission and run completion.
  • Frequency: Fails consistently in CI, may pass locally due to faster I/O
  • Root cause hypothesis: Non-deterministic timing in async prompt submission pipeline. Test lacks explicit synchronization barrier.
  • Suggested fix: Add explicit waitFor/flush before assertion, or mock the timing to remove non-determinism.

2. hook.execute > runHook > timeout returns pass

  • Location: Likely in test/hook/
  • Failure mode: Takes 5000.98ms — just barely exceeds the 5000ms timeout threshold
  • Frequency: Fails in CI where runners are slower, passes locally
  • Root cause hypothesis: The timeout boundary is too tight for CI environments. The test expects the hook to resolve within exactly 5000ms but CI jitter pushes it over.
  • Suggested fix: Either increase the timeout threshold by a small margin (e.g., 5100ms) or use a more generous timeout in CI environments via process.env.CI detection.

Impact

  • These failures are non-blocking for PRs that don't touch session/hook code
  • unit results (linux) secondary job also reports failure as a consequence
  • Developers may waste time investigating false positives

Evidence

CI logs from PR #117-#120 (all show identical failures):

(fail) prompt submitted during an active run is included in the next LLM input [283.00ms]
(fail) hook.execute > runHook > timeout returns pass [5000.98ms]

Suggested Actions

  • Investigate and fix the timing issue in prompt-during-run test
  • Adjust hook timeout test to account for CI jitter
  • Consider adding @flaky tag or retry annotation as interim measure
  • Check if these are known upstream issues (anomalyco/opencode)

Ref

Observed across PRs #117, #118, #119, #120 (2026-04-06)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions