Skip to content

adapter: fix the XREAD wrong-type test's shared timing budget - #1249

Open
bootjp wants to merge 1 commit into
mainfrom
fix/redis-xread-block-deadline
Open

bootjp wants to merge 1 commit into
mainfrom
fix/redis-xread-block-deadline

Conversation

@bootjp

@bootjp bootjp commented Sep 14, 2026

Copy link
Copy Markdown
Owner

TestRedis_StreamXReadBlockChecksWrongTypeAtDeadline is timing-dependent and took down test (ubuntu-latest) on #1222, which touches only internal/snapshotoffload.

Mechanism

The test blocked for 2 s and waited up to 2 s for the stream waiter to register before overwriting the key:

Block: 2 * time.Second,
...
requireStreamWaiterRegistered(...)   // require.Eventually, 2s timeout
require.NoError(t, rdbWriter.Set(ctx, key, "now-a-string", 0).Err())

The wrong-type check runs when the block deadline fires, so a SET that lands after it gets the ordinary block-timeout nil instead:

Error: "redis: nil" does not contain "WRONGTYPE"

Registration and the remaining block window drew on the same 2 s, so a loaded machine could consume the whole window before the SET was sent.

Reproduced, not guessed

Delaying the SET past the deadline produces the identical failure:

--- FAIL: TestRedis_StreamXReadBlockChecksWrongTypeAtDeadline (3.89s)
    Error: "redis: nil" does not contain "WRONGTYPE"

Fix

Block window 6 s, registration wait 1 s — the two no longer share a budget. The registration timeout becomes a parameter, because any caller whose assertion depends on acting before the reader's deadline has to bound it well under that deadline, and it now carries its own failure message, so a slow registration fails as "the waiter never appeared" rather than surfacing later as a confusing nil read.

Risk

None to production: test-only, no production file touched. The test's subject — that the wrong-type check happens at the deadline — is unchanged; only the margin around it is.

Test evidence

  • go test ./adapter/ -run TestRedis_StreamXReadBlockChecksWrongTypeAtDeadline -count=5 — ok
  • With a 2.5 s delay injected before the SET (the condition that broke it): passes at the new 6 s window, and fails with the original 2 s window — so the widened window is what does the work, not the injected delay.
  • golangci-lint --config=.golangci.yaml run ./adapter/... — 0 issues

Self-review

  1. Data loss — none; test-only.
  2. Concurrency / distributed failures — removes a timing dependency. The registration wait is still bounded, so a genuinely broken waiter registration still fails, now with a clearer message.
  3. Performance — the test takes ~6 s instead of ~2 s in the deadline path. It is t.Parallel(), so the cost to the suite is minimal.
  4. Data consistency — untouched; the wrong-type detection path is unchanged.
  5. Test coverage — same assertions, now deterministic under load.

Fourth timing-dependent test flake in this batch (#1236 carries three others). All four share a shape: an assertion pinning one of several equally-correct outcomes, or a budget shared between setup and the thing being measured.

@codex review
@claude review

https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE

…tration

TestRedis_StreamXReadBlockChecksWrongTypeAtDeadline blocked for 2s and
waited up to 2s for the stream waiter to register before overwriting the
key. The wrong-type check runs when the block deadline fires, so a SET
that lands after it gets the ordinary block-timeout nil instead -- which
the test reports as

  "redis: nil" does not contain "WRONGTYPE"

On a loaded machine the registration wait can consume the whole window
before the SET is sent. Reproduced exactly by delaying the SET past the
deadline.

The block window is now 6s and the registration wait 1s, so the two no
longer share a budget: a slow registration fails as "the waiter never
appeared", with its own message, rather than surfacing later as a
confusing nil read. The registration timeout is a parameter now, since
any caller whose assertion depends on acting before the reader's deadline
has to bound it well under that deadline.

Claude-Session: https://claude.ai/code/session_013rNHooj7NF3giihWVba8QE
@coderabbitai

coderabbitai Bot commented Sep 14, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 53 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 0bc9de5b-4fb5-423b-af37-8e5a2250acd4

📥 Commits

Reviewing files that changed from the base of the PR and between 4ca7e90 and f1e1027.

📒 Files selected for processing (1)
  • adapter/redis_compat_commands_stream_test.go

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 14, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-14T08:44:02.801209Z f1e1027 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f1e10274af

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

// machine could consume the whole window before the SET was sent.
const (
blockWindow = 6 * time.Second
waiterRegistering = time.Second

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Allow more time for waiter registration

When a loaded runner needs more than one second to perform the XREAD's initial work and register the otherwise healthy waiter, this newly shortened timeout fails the test before the widened six-second block window can provide any benefit. The original helper allowed two seconds, and the reported flake demonstrates that this setup can be delayed substantially; keep at least that tolerance (while still remaining safely below blockWindow) so the timing fix does not replace the confusing nil failure with a more frequent registration-timeout flake.

Useful? React with 👍 / 👎.

@bootjp

bootjp commented Sep 15, 2026

Copy link
Copy Markdown
Owner Author

CI の test (ubuntu-latest) 失敗はこの PR の変更とは無関係でした。TestGRPCS3BlobClusterAuthenticatedCapabilityPushAndFetchcodes.Unauthenticated を期待して codes.Unknown を受け取っていたもので、原因は main に元からあるクライアントストリームのバグです。

gRPC はサーバ側が既に終了させたストリームへの Send を素の io.EOF として返し、実際の status は受信側 (CloseAndRecv) からしか読めません。PushChunkBlob を含む 4 箇所がその io.EOF をそのまま呼び出し元へ返していたため、reset が Send に競り勝った実行でだけ status が失われます。テストの flake ではなく本番のバグで、s3_blob_replicator.go:136 のリトライ判断がこのエラーを見ています。

#1252 で 4 箇所すべてを修正しました。マージ後に main を取り込めばこの CI は緑になるはずです。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant