feat(pii): add opt-in GLiNER NER engine (PII_ENGINE), device-agnostic

[TheodoreSpeaks]

What

Adds GLiNER (zero-shot transformer NER) as an opt-in engine for the PII service, selected by PII_ENGINE (spacy | gliner, default spacy). Of the ~40 supported entity types only 4 are NER — PERSON, LOCATION, NRP, DATE_TIME — and GLiNER replaces exactly those; all ~36 regex/checksum recognizers (locale IDs, credit card, email, VIN, …) are registered identically on both engines.

One image, one tag. The pii image ships both engines (spaCy *_lg models + torch-CPU + gliner + baked GLiNER weights), so switching engines is a pure env flip — no image swap, no tag matrix, and rollback is PII_ENGINE=spacy. Image grows ~6.1GB → ~9.6GB.

Device-agnostic: PII_DEVICE (cpu/cuda) or auto-detect. The same code runs on Fargate CPU today and the EC2-GPU fleet later — no code change to switch. GPU capacity/scheduling is a separate infra task (not touched here).

Design

• apps/pii/engines.py (new): side-effect-free engine builders extracted from server.py (which is now just env dispatch + endpoints; the spacy path is construction-identical to before).
  • One multilingual model (urchade/gliner_multi_pii-v1, override via PII_GLINER_MODEL) shared across the 5 per-language recognizer instances — SharedModelGLiNERRecognizer.load() caches the ~1.2GB model instead of loading it 5×.
  • Its analyze() restricts requested entities to the 4 it owns: presidio passes the full ~40-entity list to every recognizer and stock GLiNERRecognizer would append unknown entities (CREDIT_CARD, VIN, ES_NIF, …) as ad-hoc zero-shot labels — wrong scope and slower.
  • The gliner path uses the small spaCy models (*_sm, ~60MB total) purely for tokenization + lemmas so NlpArtifacts/context-boosts for the regex recognizers behave as before (blank pipelines don't work — presidio tries to pip-download them). SpacyRecognizer is removed from the registry so sm-model NER never competes with GLiNER.
  • Fail-fast: invalid PII_ENGINE dies at import; a custom image missing gliner deps dies with an actionable message instead of presidio's silent pip-download fallback.
• docker/pii.Dockerfile: single linear image. GLiNER weights are baked at build (cached layer) with HF_HUB_OFFLINE=1 so startup never touches the network and a typo'd PII_GLINER_MODEL fails loudly. The GPU variant for the infra follow-up is the same Dockerfile built with --build-arg TORCH_INDEX_URL=https://download.pytorch.org/whl/cu128 (torch pinned to the newest version published on both cpu and cu128 indexes). Bench + tests + pytest ride along in the image.
• CI: unchanged — same single pii build/tags as before.
• Helm: pii.engine (spacy|gliner, default spacy) and pii.device render to PII_ENGINE/PII_DEVICE as chart-owned env keys (pii.env can't override them; PII_GLINER_MODEL stays a plain pii.env passthrough). nvidia.com/gpu requests are deliberately not wired — infra follow-up.
• PII_WORKERS (merged from feat(pii): env-driven uvicorn worker count (PII_WORKERS) #5457) note: each uvicorn worker loads its own GLiNER model copy — into GPU memory on cuda — so GPU deployments generally want PII_WORKERS=1 per GPU (documented in Dockerfile + server.py; behavior unchanged).

Parity & throughput (spaCy vs GLiNER-CPU)

apps/pii/scripts/bench_engines.py runs the same payload through both engines and diffs results; the 32-text multilingual payload (en/es/it/pl/fi with names, locations, dates, NRP, and regex bait: emails, phones, valid-checksum VIN/ES_NIF/IT_FISCAL_CODE/PESEL/HETU) ships in the repo. Numbers below from the CPU (aarch64 docker) run; rerun with docker run --rm <pii-image> python scripts/bench_engines.py:

engine	throughput	p50 / p95 latency
spacy (*_lg)	~305 texts/s	2.9ms / 5.8ms
gliner (CPU)	~3.7 texts/s	272ms / 355ms

CPU-GLiNER is ~80× slower than spaCy — that's the point of this being opt-in now and GPU-bound later.

NER agreement (span IoU ≥ 0.5, same type):

entity	spacy	gliner	matched
PERSON	18	28	17
LOCATION	23	26	19
DATE_TIME	15	24	12
NRP	8	13	6
ORGANIZATION	3	0	— (intentional: not in the app's supported set, no GLiNER mapping)

Regex/checksum entities: identical across engines ✓ (the script exits non-zero on any non-NER diff).

Qualitative diff highlights: GLiNER has higher recall overall and produces wider (often better) spans (45 Queen Street, Toronto vs just Toronto), but zero-shot eagerness tags pronouns (She, he) and once an email as PERSON; spaCy has its own false positives (quarterly as DATE_TIME, credit-card number as ORGANIZATION) and misses GLiNER catches. GLINER_ENTITY_MAPPING label prompts are the tuning lever; the bench is the harness.

Enabling

• Our fleet: infra sets PII_ENGINE=gliner on the pii task def (same image). Realistically waits for the GPU capacity provider given CPU throughput.
• Helm: pii.engine: gliner — nothing else.
• Local: docker run -e PII_ENGINE=gliner ....

Validation

• PII_ENGINE unset ⇒ spaCy, regression-tested /redact_batch en+es byte-identical to today; PII_ENGINE=bogus dies at import with a clear error.
• PII_ENGINE=gliner masks PERSON/LOCATION/email/phone/VIN/ES_NIF; flipping back to spacy on the same image behaves identically to default; startup is fully offline.
• Tests: apps/pii/tests — unit suite (mocked GLiNER; shared-model load-once, label restriction, fail-fast) runs model-free; integration suite (RUN_PII_INTEGRATION=1) ran green in-image for both engines.
• helm unittest 65/65 (incl. new engine/device/override tests), helm lint clean, bun run lint:check green, ruff clean.

Pre-existing quirk noted while validating (not addressed here): /analyze with return_decision_process=true 500s on both engines (presidio's AnalysisExplanation isn't JSON-serializable via to_dict()); the app never sends that flag.

Follow-ups (infra task)

• EC2-GPU capacity provider + nvidia.com/gpu requests; build/publish the cu128 image via TORCH_INDEX_URL build-arg; set PII_WORKERS=1 per GPU.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Up3F97mjCH9HCj1pX4J8VJ

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview, Comment	Jul 8, 2026 12:25am

[cursor]

PR Summary

Medium Risk
Touches PII redaction behavior when gliner is enabled (NER spans can differ from spaCy); default spacy path is intended to be unchanged. Larger image and higher memory/GPU worker caveats affect ops, not app auth.

Overview
Adds an opt-in GLiNER NER path to the PII Presidio service while keeping spaCy as the default (PII_ENGINE=spacy). Analyzer construction moves into new engines.py with shared regex/checksum recognizers (VIN, locale IDs); server.py only dispatches on PII_ENGINE, PII_DEVICE, and PII_GLINER_MODEL.

The gliner engine swaps spaCy NER for one shared multilingual GLiNER model (SharedModelGLiNERRecognizer), uses small spaCy models for tokenization only, strips SpacyRecognizer, and restricts GLiNER prompts to PERSON/LOCATION/NRP/DATE_TIME. The single PII image now bundles torch, gliner deps, small spaCy models, and offline-baked HuggingFace weights (HF_HUB_OFFLINE=1).

Helm exposes pii.engine / pii.device as chart-owned env (users cannot override via pii.env). Adds bench_engines.py for spacy vs gliner throughput/NER parity, plus unit and gated integration tests.

^{Reviewed by Cursor Bugbot for commit bff7626. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR adds GLiNER as an opt-in zero-shot NER engine alongside spaCy, selected via PII_ENGINE at runtime with no image change required. Engine selection is cleanly factored into a new engines.py module with side-effect-free builders; server.py becomes a thin env-dispatch layer.

• engines.py: SharedModelGLiNERRecognizer caches the ~1.2 GB GLiNER model across 5 per-language recognizer instances; its analyze() override scopes GLiNER to only the 4 NER entities it owns, preventing ad-hoc zero-shot labelling of regex entities. Both engine paths register identical regex/checksum recognizers.
• docker/pii.Dockerfile: Single linear image ships spaCy *_lg + *_sm models, torch-CPU, gliner, and baked HF weights; HF_HUB_OFFLINE=1 ensures offline-only startup. GPU variant is the same Dockerfile built with a TORCH_INDEX_URL build-arg.
• Helm / tests: pii.engine/pii.device rendered as chart-owned env vars with omit() guard; 65 Helm unit tests pass; model-free unit suite and in-image integration suite cover both engines.

Confidence Score: 5/5

Safe to merge — the spaCy default path is construction-identical to the code it replaces, and the GLiNER path is gated behind an explicit env var that is unset in all current deployments.

The refactor correctly extracts engine builders into a dedicated module without changing the spaCy path's observable behavior. The shared-model cache, label-scoping override, and fail-fast guards are all correctly implemented and tested. No logic regressions were found in the dispatch flow, the VIN check-digit validator, or the Helm chart rendering.

No files require special attention. The extraEnvVars bypass noted in a previous review thread is a pre-existing Helm pattern limitation, not introduced by this PR.

Important Files Changed

Filename	Overview
apps/pii/engines.py	New module: clean side-effect-free engine builders, shared GLiNER model cache, label-scoping analyze() override, and VIN check-digit validator. Core logic is correct.
apps/pii/server.py	Refactored to env-dispatch thin wrapper; PII_ENGINE validation, logger placement, and module-level startup ordering are all correct.
docker/pii.Dockerfile	Single-image design ships both engines; layer ordering, HF_HUB_OFFLINE bake, and GPU build-arg path all look correct.
apps/pii/tests/test_engines.py	Unit tests cover fail-fast, shared-model load-once, label restriction, and inference skip — all model-free via FakeGLiNER monkeypatch.
apps/pii/tests/test_integration.py	Integration suite adapts to PII_ENGINE and gates itself behind RUN_PII_INTEGRATION; shared + engine-specific assertions cover the main redaction paths.
helm/sim/templates/deployment-pii.yaml	PII_ENGINE/PII_DEVICE rendered as chart-owned env vars with omit() guard on pii.env; extraEnvVars still appended last (covered in previous review thread).
helm/sim/values.schema.json	Added enum-validated engine field and freeform device field under the pii block.
helm/sim/values.yaml	Added pii.engine (default: spacy) and pii.device (default: empty) with clear comments on CPU/GPU trade-offs.
helm/sim/tests/pii_test.yaml	Helm unit tests cover default spacy, explicit gliner+device, and pii.env override protection for the chart-owned keys.
apps/pii/scripts/bench_engines.py	Benchmark + parity harness with throughput, per-text latency, NER IoU diff, and a hard non-NER parity gate that exits non-zero on any mismatch.
apps/pii/requirements-gliner.txt	Pinned gliner, transformers, huggingface_hub; torch intentionally omitted (handled via build-arg in Dockerfile).
apps/pii/requirements-dev.txt	New file: pytest + httpx for model-free unit tests and integration suite.
apps/pii/tests/conftest.py	Inserts apps/pii onto sys.path so tests can import server/engines directly from the repo.
.gitignore	Added pycache and .pytest_cache ignores for the new Python test tooling.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[container start] --> B{PII_ENGINE?}
    B -- "invalid" --> C[ValueError at import\nfail fast]
    B -- "spacy (default)" --> D[build_spacy_analyzer\n5x spaCy lg models]
    B -- "gliner" --> E{deps present?}
    E -- "gliner pkg missing" --> F[RuntimeError\nfail fast]
    E -- "sm models missing" --> G[RuntimeError\nfail fast]
    E -- "ok" --> H[build_gliner_analyzer\n5x sm models for tokenization]
    H --> I[remove SpacyRecognizer\nfrom registry]
    I --> J{assert removal}
    J -- "failed" --> K[RuntimeError\nPresidio layout changed]
    J -- "ok" --> L[add 5x SharedModelGLiNERRecognizer\nshared _shared_models cache]
    D --> M[_register_common_recognizers\nVIN x5 languages + 21 EXTRA_RECOGNIZERS]
    L --> M
    M --> N[AnalyzerEngine ready]
    N --> O[FastAPI app serves /analyze /redact /redact_batch]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[container start] --> B{PII_ENGINE?}
    B -- "invalid" --> C[ValueError at import\nfail fast]
    B -- "spacy (default)" --> D[build_spacy_analyzer\n5x spaCy lg models]
    B -- "gliner" --> E{deps present?}
    E -- "gliner pkg missing" --> F[RuntimeError\nfail fast]
    E -- "sm models missing" --> G[RuntimeError\nfail fast]
    E -- "ok" --> H[build_gliner_analyzer\n5x sm models for tokenization]
    H --> I[remove SpacyRecognizer\nfrom registry]
    I --> J{assert removal}
    J -- "failed" --> K[RuntimeError\nPresidio layout changed]
    J -- "ok" --> L[add 5x SharedModelGLiNERRecognizer\nshared _shared_models cache]
    D --> M[_register_common_recognizers\nVIN x5 languages + 21 EXTRA_RECOGNIZERS]
    L --> M
    M --> N[AnalyzerEngine ready]
    N --> O[FastAPI app serves /analyze /redact /redact_batch]

Loading

_{Reviews (2): Last reviewed commit: "refactor(pii): ship both engines in one ..." | Re-trigger Greptile}

[TheodoreSpeaks]

@greptile review