Status: Binding
Tests protect product guarantees.
Tests should not require:
- real networks,
- API keys,
- real model calls,
- uncontrolled wall-clock behavior,
- uncontrolled random identifiers.
Inject clocks/IDs or use fakes when nondeterminism affects assertions.
A flaky test is a defect.
Behavior AgentDeck claims across engines/adapters should be expressed as shared contract tests whenever practical.
Do not duplicate the same invariant as unrelated engine-specific tests.
Use goldens when byte-level or schema-level compatibility is itself the contract.
Golden baselines change deliberately.
CI must never silently regenerate a baseline to make itself pass.
Concurrency guarantees deserve explicit tests.
Examples:
- double resume,
- double answer,
- cancel versus resume,
- competing session admission,
- concurrent key claims.
Arrange contention intentionally.
Assert the promised outcome, not that tasks happened to overlap for a particular duration.
Durable guarantees must be tested across restart/recovery boundaries when practical.
Examples:
- terminal state remains terminal,
- sequence/order continues correctly,
- suspended work can be recovered,
- stale ownership does not corrupt active work.
Test hard paths on purpose:
- exceptions during execution,
- cancellation,
- consumer disconnect,
- partial writes,
- adapter failure,
- observer failure,
- store failure.
"Hard to test" is a design signal, not automatically a reason to skip coverage.
Assert behavior and invariants.
Prefer:
exactly one caller wins
over:
both callers were inside the critical section at the same time
Test names state the guarantee:
test_terminal_event_remains_last_after_restart
not:
test_case_4
Do not:
- loosen assertions,
- modify lint/type configs,
- regenerate goldens,
- skip tests,
merely to make a change green.
A failing guardrail is evidence to understand.