Is your feature request related to a problem? Please describe.
Following #1868 (OWASP LLM02 output-side scorer pack: XSS / SQLi / Shell / Path), red-teamers evaluating LLM output have no deterministic, no-LLM scorer in PyRIT for five additional common injection payload families: SSRF, SSTI, XXE, open redirect, and LDAP injection. These gaps mean batch/CI pipelines either skip these checks or fall back to a slower LLM-based scorer for what is pattern-matchable work.
Describe the solution you'd like
Extend the pyrit/score/true_false/regex/ family with five more output-side payload detectors using the same RegexScorer pattern:
- SSRFOutputScorer — cloud-metadata endpoints (169.254.169.254), loopback/RFC1918 URL targets, and SSRF-prone schemes (gopher/dict).
- SSTIOutputScorer — the canonical arithmetic eval probe (
{{7*7}} / ${} / #{}) and Python object-traversal gadget chains (__class__/__mro__/__subclasses__/__globals__). Deliberately excludes ordinary {{ variable }} to keep false positives low.
- XXEOutputScorer — external/parameter entity declarations and DOCTYPE internal subsets.
- OpenRedirectOutputScorer — protocol-relative / encoded-slash redirect parameters and userinfo host confusion.
- LDAPInjectionOutputScorer — filter-break and always-true clause injection markers.
Each mirrors XSSOutputScorer exactly: a _DEFAULT_PATTERNS dict, categories=["security"], OR aggregator, no LLM call (cheap for batch/CI), plus unit tests in tests/unit/score/regex/ covering positives, benign negatives, rationale, custom patterns, and memory.
I've implemented this on a branch already (matching #1868's structure). Before I open the PR — does this scope and the standalone-scorer-per-family placement look right, or would you prefer them grouped differently? Happy to adjust. (Microsoft CLA is already on file from #1868.)
Describe alternatives you've considered
Using the existing LLM-based InsecureCodeScorer — too slow and costly for batch evaluation, and it targets code-quality issues rather than output-side payload detection. A single combined "injection" scorer — possible, but splitting by family matches the existing per-category precedent (#1868) and gives callers fine-grained control.
Additional context
These complement #1868 and round out the common OWASP LLM02 output-side payload families.
Is your feature request related to a problem? Please describe.
Following #1868 (OWASP LLM02 output-side scorer pack: XSS / SQLi / Shell / Path), red-teamers evaluating LLM output have no deterministic, no-LLM scorer in PyRIT for five additional common injection payload families: SSRF, SSTI, XXE, open redirect, and LDAP injection. These gaps mean batch/CI pipelines either skip these checks or fall back to a slower LLM-based scorer for what is pattern-matchable work.
Describe the solution you'd like
Extend the
pyrit/score/true_false/regex/family with five more output-side payload detectors using the sameRegexScorerpattern:{{7*7}}/${}/#{}) and Python object-traversal gadget chains (__class__/__mro__/__subclasses__/__globals__). Deliberately excludes ordinary{{ variable }}to keep false positives low.Each mirrors
XSSOutputScorerexactly: a_DEFAULT_PATTERNSdict,categories=["security"],ORaggregator, no LLM call (cheap for batch/CI), plus unit tests intests/unit/score/regex/covering positives, benign negatives, rationale, custom patterns, and memory.I've implemented this on a branch already (matching #1868's structure). Before I open the PR — does this scope and the standalone-scorer-per-family placement look right, or would you prefer them grouped differently? Happy to adjust. (Microsoft CLA is already on file from #1868.)
Describe alternatives you've considered
Using the existing LLM-based
InsecureCodeScorer— too slow and costly for batch evaluation, and it targets code-quality issues rather than output-side payload detection. A single combined "injection" scorer — possible, but splitting by family matches the existing per-category precedent (#1868) and gives callers fine-grained control.Additional context
These complement #1868 and round out the common OWASP LLM02 output-side payload families.