Skip to content

jev-confidence-gated-triage-cdk: Confidence-gated ticket triage with TypeSafe Jev - #3310

Open
NithinChandranR-AWS wants to merge 1 commit into
aws-samples:mainfrom
NithinChandranR-AWS:NithinChandranR-AWS-feature-jev-confidence-gated-triage-cdk
Open

NithinChandranR-AWS wants to merge 1 commit into
aws-samples:mainfrom
NithinChandranR-AWS:NithinChandranR-AWS-feature-jev-confidence-gated-triage-cdk

Conversation

@NithinChandranR-AWS

@NithinChandranR-AWS NithinChandranR-AWS commented Sep 20, 2026

Copy link
Copy Markdown
Contributor

Description

Auto-triage support tickets with the TypeSafe Jev System One model: one AWS Lambda function gets typed decisions plus calibrated confidence, and an AWS Step Functions Choice state decides whether to act.

Architecture

Amazon API Gateway (POST /tickets, protected by an API key + usage plan) -> AWS Lambda (one HTTPS call to TypeSafe Jev: a choice for department, a score for urgency, a noul for whether the ticket reports a security incident) -> AWS Step Functions Express. The Choice state is the confidence gate: it auto-routes when the department and urgency confidences clear their thresholds, pages on-call via Amazon SNS when the security answer is true with confidence >= 0.9, and otherwise sends the ticket to an Amazon SQS human-review queue (backed by a dead-letter queue) with the full Jev output attached. The Jev API key is held in AWS Secrets Manager.

Verified

  • Deployed live to us-east-1 and tested end-to-end against the real TypeSafe Jev API (model jev-1.13.0). All three confidence-gate branches fired with real model output: a clear ticket auto-routed (department 0.98, urgency 0.99); a genuinely ambiguous ticket fell through to the Amazon SQS human-review queue (urgency confidence 0.38, below the 0.7 gate) with the full Jev decision attached; a clear security-incident ticket paged on-call via Amazon SNS (security noul true, confidence 0.96, above the 0.9 gate).
  • cdk synth passes; IAM scoped to specific resource ARNs (no application-resource wildcards)
  • Request and response follow the TypeSafe System One contract (questions carry instructions; choice uses a criteria object; score uses an ordered criteria array; noul states a proposition and returns a probability), verified against https://docs.typesafe.ai/api
  • 19/19 unit tests pass, exercising the request contract and the confidence-gate branch logic against Jev's real typed-answer response shape

… pattern

Auto-triage support tickets with the TypeSafe Jev System One model. One
AWS Lambda function sends the ticket plus three typed questions (choice,
score, noul) to Jev in a single call and reads back calibrated confidence
per answer; an AWS Step Functions Choice state branches on that confidence
to auto-route, page security on-call via Amazon SNS, or fall through to an
Amazon SQS human-review queue.

Request and response follow the TypeSafe System One contract: questions
carry instructions, choice lists options in a criteria object, score lists
ordered levels in a criteria array, and noul states a proposition. The
handler normalizes each typed answer into {value, confidence} for the gate,
deriving a noul's confidence from |p - 0.5| * 2.

The ingest endpoint requires an Amazon API Gateway API key (with a usage
plan) so the paid TypeSafe Jev calls and security pages cannot be driven
anonymously.
@NithinChandranR-AWS
NithinChandranR-AWS force-pushed the NithinChandranR-AWS-feature-jev-confidence-gated-triage-cdk branch from 0ce9cc8 to 205c327 Compare September 21, 2026 04:44

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants