Skip to content

fix(cuda): advertise enforced embedding batch limits for owned-cuda - #21

Merged
ualtinok merged 1 commit into
cortexkit:masterfrom
Qiiks:fix/owned-cuda-batch-advice
Sep 19, 2026
Merged

ualtinok merged 1 commit into
cortexkit:masterfrom
Qiiks:fix/owned-cuda-batch-advice

Conversation

@Qiiks

@Qiiks Qiiks commented Sep 19, 2026

Copy link
Copy Markdown
Contributor

Summary

recommended_batch_for_engine returns None for owned-cuda, so models.list omits recommended_batch for CUDA models. Consumers that require the field reject the envelope (AFT's batch pager refuses discovery — reproduced end-to-end here: AFT failed model discovery with no usable recommended_batch rows/token_budget, and a normal semantic search made zero embedding calls as a result).

This is the same omission PR #19's review surfaced; it was kept out of #19 deliberately. Alfonso's 4a03d6bf landing covers owned-metal + ane; the owned-cuda arm fell through.

Change

  • One-line producer fix: owned-cuda joins the owned-metal arm — rows=MAX_ENGINE_BATCH_ITEMS (8), token_budget=DEFAULT_ENGINE_BATCH_TOKEN_BUDGET (3072). These are the module-enforced limits, matching what the worker actually admits; no invented values.
  • Regression: extends recommended_batch_policy_uses_engine_constants_and_omits_unknown_advice to assert the CUDA arm serializes with those exact constants (fails pre-fix, passes post-fix).

Verification

  • cargo test -p synapse-module --lib recommended_batch — 1 passed (fails on master: no CUDA arm).
  • cargo clippy -p synapse-module --lib -- -D warnings clean; cargo fmt --all -- --check clean.
  • Live end-to-end: AFT on native Synapse indexed and retrieved (refund_cancelled_purchase, semantic_score 0.5837) only after the deployed module carried this arm; pre-fix it refused discovery.
  • No GPU benchmark; rebased on current master (6015c48); Cargo.lock untouched.

Real consumer proof already posted in the #19 thread (issuecomment-5715898282); this PR is the upstreamable form.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Fixes CUDA models missing recommended_batch in models.list because recommended_batch_for_engine returned None for owned-cuda. The engine now returns the same enforced batch limits as owned-metal (8 rows, 3072 token budget), so consumers that require the field no longer reject the envelope.

  • Adds a regression test asserting the CUDA arm serializes those exact constants.

Written for commit cefe807. Summary will update on new commits.

Review in cubic

Copilot AI lite review requested due to automatic review settings September 19, 2026 00:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@synapse-alfonso

Copy link
Copy Markdown

Verified and landing. Confirmed at source that the advertised values are the enforced ones: the batch planner clamps every engine to MAX_ENGINE_BATCH_ITEMS (8) and DEFAULT_ENGINE_BATCH_TOKEN_BUDGET (3072) with no engine branch, so a CUDA row gets exactly what it advertises. Mutation-checked: with the arm reverted, the extended test fails at the CUDA assertion; with it, passes. Clippy clean, CI green, Cargo.lock untouched.

Thanks for tracing the consumer failure end to end — a producer omission that reads as "no error" on our side and "no embedding calls" on AFT's is exactly the kind of silent gap worth a regression test.

@ualtinok
ualtinok merged commit db7be86 into cortexkit:master Sep 19, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants