research: EP review phase 6 — scope analytic (conjugate) likelihood updates

[Jammy2211]

Overview

Phase 6 of the EP framework review (umbrella: PyAutoMind/research/graphical_ep/ep_framework_review.md). EP's cost is dominated by per-factor sampler runs (see PyAutoMind/research/graphical_ep/ep_scoping.md); factors whose tilted-distribution moments exist in closed form should skip the sampler entirely. Phase 1 established the hooks already exist (ExactFactorFit, has_exact_projection / calc_exact_update / calc_exact_projection, auto-selected by EPOptimiser.from_meanfield). This phase maps what exists, what's reachable, which (likelihood form × cavity family) pairs admit analytic updates, the API to expose them, and the expected speed-up.

Autonomy: --auto, effective supervised (research cap). Read-only; parks at the scoping verdict.

Plan

1. Inventory the existing exact-update machinery: every calc_exact_update/calc_exact_projection implementation, the regression tests that exercise them (test_linear_regression.py::test_exact_updates, test_exact.py::test_probit_regression), and what they prove already works.
2. Determine whether the declarative path uses the exact machinery at all — specifically whether PriorFactors in factor_graph.optimise(...) are conjugate-multiplied or needlessly optimised (Phase 3 logs suggest the latter: PriorFactors appear in the per-factor optimisation logs). If so, that is the immediate cheap win: an EP cycle currently pays ~one optimiser run per free parameter.
3. Map the conjugate table: Gaussian×Gaussian (exists), linear-Gaussian factors (z = Ax + b — partially present via jacobians?), probit, Gamma/Beta conjugacies, hierarchical Gaussian parent updates; note which are blocked by open bugs (Priors & messages: 9 confirmed bugs — guidance wanted on 5 decisions #1331-04 truncated log-partition).
4. Ground against the use cases: the IC50 global factor (-0.5‖(latents @ C + c − μ)/σ‖²) is exactly linear-Gaussian — quantify what making it analytic would save.
5. Output: scoping verdict — ranked implementation candidates with API sketch and speed-up estimates, posted here for prioritisation.

[Jammy2211]

Phase 6 scoping verdict — analytic likelihood updates

What already exists (more than expected)

1. Machinery: ExactFactorFit + the has_exact_projection / calc_exact_update / calc_exact_projection hooks, auto-selected by EPOptimiser.from_meanfield. Verified correct in Phase 1; exercised by Phase 3's ep_exact.py.
2. Conjugate message updates: any message-as-factor (AbstractMessage.as_factor) multiplies in exactly (base-class calc_exact_update).
3. A duck-typed user contract, undocumented: the regression tests define factor callables that supply their own analytic tilted projections — ProbitModel (probit-Bernoulli × Gaussian cavity, closed-form moments incl. log_norm) and LinearModel (linear-Gaussian) in test_autofit/graphical/regression/test_exact.py / test_linear_regression.py::test_exact_updates. This is exactly the "analytic likelihood update" API — it exists, works, and is documented nowhere.

The gap (and the immediate win)

The declarative path never uses any of it. AbstractDeclarativeFactor._make_ep_optimiser constructs EPOptimiser(...) directly (not from_meanfield), so exact detection never runs. Worse, it couldn't detect anyway: PriorFactor.__init__ wraps the bound method prior.factor, which strips the hooks — verified live:

pf = PriorFactor(af.GaussianPrior(0, 1))
pf._has_exact_projection  # None
hasattr(pf.prior.message, 'has_exact_projection')  # True

So every factor_graph.optimise(...) run pays one full optimiser run per free parameter per EP cycle for updates that are closed-form conjugate multiplications. The author knew: declarative/factor/prior.py:22 carries # TODO: Consider analytical solution rather than implementing optimisation. In the Phase 3 tutorial graph, 7 of 10 factor fits per cycle are PriorFactors; at IC50 scale it's ~3N+ unnecessary optimiser runs per cycle, each paying the full per-fit wrapper overhead that ep_scoping.md identified as EP's amplified cost.

Ranked implementation candidates

#	Candidate	Effort	Pay-off
1	Exact PriorFactor updates in the declarative path — forward the message's exact hooks through PriorFactor (the as_factor pattern) and switch _make_ep_optimiser to from_meanfield-style auto-detection	~1 day + tests	Removes one optimiser run per free parameter per cycle on every declarative EP fit — the dominant factor count on real graphs; resolves the in-code TODO
2	Document + formalise the analytic-projection contract — promote the duck-typed has_exact_projection/calc_exact_projection protocol to a documented interface (README §3.2 + an example script following ProbitModel)	~1-2 days	Unlocks user-side analytic factors generally; prerequisite for 3
3	First-class linear-Gaussian factor — generalise the regression-test LinearModel into autofit.graphical (with plates). Grounding: the IC50 global factor is exactly this shape (-0.5‖(latents @ C + c − μ)/σ‖², jointly Gaussian in latents and hill_coef) — its per-cycle search, the highest-dimensional fit on that graph, becomes closed-form	~1 week	Eliminates the worst per-cycle fit in the flagship real use case
4	Wider conjugate table (Gamma-Poisson, Beta-Bernoulli)	medium	No current demand; blocked by the Gamma/Beta bugs (#1331 dec. 3, #1332 F2) — sequence after the fix batches
5	Truncated-normal analytic moments	small-medium	Blocked by #1331-04 (log-partition bug)

Recommendation

Implement 1 + 2 as one feature task (they share tests and the README section), then 3 as its own task validated against an IC50-shaped integration script. 4/5 wait for the fix batches. Speed-up verification: extend Phase 4's ep_history.csv timing story or the ep_scoping.md baseline package to compare cycles-to-convergence and wall time with/without exact PriorFactors.

Parked for prioritisation — approving this files the feature/autofit prompt(s) in PyAutoMind per the no-bulk-queue convention.

[Jammy2211]

Implementation plan for all four work packages filed as #1338 (plan only, nothing implemented — WP1 exact PriorFactors + projection contract, WP2 linear-Gaussian factor, WP3 conjugate table [gated on #1331/#1332 fixes], WP4 truncated-normal moments [gated on #1331-04]). Backlog anchor: PyAutoMind/feature/autofit/ep_analytic_updates.md. Pick-up may be a fresh session/model — the issue is self-contained.

[Jammy2211]

Closing — the research/scoping deliverable is complete. Phase 6 scoped the analytic (conjugate) likelihood updates and produced the four-work-package implementation plan on #1338 (exact PriorFactors, projection contract, linear-Gaussian, conjugate table). Per direction this was plan-only (no implementation). #1338 stays open as the implementation backlog; closing this research task as its output lives there.