feat: Milestone 8 — render/bundle layer and end-to-end Generator.generate()

[shaypal5]

Summary

Implements the full observation model (Milestone 8) that transforms hidden simulation output into the canonical output bundle on disk.

• render/relational.py: to_dataframes() — converts SimulationResult + PopulationResult to 9 typed pd.DataFrames, one per relational table
• render/snapshots.py: build_snapshot() — 30-column leakage-free lead snapshot; vectorised touch/session/activity aggregates + account/contact field joins; has_open_opportunity uses pandas .where() to avoid deprecation warnings
• render/tasks.py: write_task_splits() — deterministic 70/15/15 train/valid/test Parquet split + task_manifest.json using seeded shuffle
• render/manifests.py: build_manifest() / write_manifest() — full provenance manifest with row counts and SHA-256 file hashes
• api/bundle.py: write_bundle() — orchestrates all render steps (tables → snapshot → task splits → dataset card → feature dictionary → manifest)
• core/models.py: WorldBundle enriched with population, simulation_result, world_graph; WorldBundle.save(path) lazy-imports write_bundle to avoid circular imports
• api/generator.py: Generator.generate() fully implemented end-to-end
• tests/render/test_render.py: 31 tests covering relational export, snapshot correctness, task splits (disjointness, ratios, determinism), manifest integrity, and full bundle smoke test
• Replaces stale test_generate_not_implemented with a WorldBundle assertion

521 tests passing; ruff + mypy clean.

Test plan

• pytest tests/render/test_render.py — 31 new render tests pass
• pytest — 521 total tests pass (no regressions)
• ruff check . — clean
• ruff format --check . — clean
• mypy leadforge/ — no issues in 63 source files

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Implements Milestone 8’s render/bundle layer so hidden simulation outputs can be rendered into canonical on-disk bundles, and wires Generator.generate() end-to-end to produce a fully populated WorldBundle.

Changes:

• Added render modules to export relational tables, build the lead snapshot, create deterministic task splits, and write provenance manifests.
• Added bundle orchestration (write_bundle) and WorldBundle.save() delegation; implemented Generator.generate() end-to-end.
• Added comprehensive render/bundle tests and updated generator tests to assert generate() returns a populated WorldBundle.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
leadforge/render/relational.py	Converts SimulationResult + PopulationResult into typed relational DataFrames.
leadforge/render/snapshots.py	Builds the canonical leakage-free lead snapshot DataFrame used for ML/task export.
leadforge/render/tasks.py	Writes deterministic train/valid/test Parquet splits plus a task manifest.
leadforge/render/manifests.py	Builds/writes bundle manifest.json with row counts and SHA-256 hashes.
leadforge/api/bundle.py	Orchestrates full bundle writing pipeline (tables → snapshot → tasks → docs → manifest).
leadforge/api/generator.py	Implements Generator.generate() to produce a populated WorldBundle.
leadforge/core/models.py	Extends WorldBundle to hold generation artefacts and adds save() method.
tests/render/test_render.py	Adds end-to-end and unit coverage for render/bundle components.
tests/api/test_generator.py	Updates generator test to expect a populated WorldBundle from generate().
tests/render/__init__.py	Establishes tests.render package.
.agent-plan.md	Updates planning/status notes to reflect Milestone 8 completion and Milestone 9 next steps.

Comments suppressed due to low confidence (2)

leadforge/render/snapshots.py:195

• These account/contact joins use DataFrame.apply(..., axis=1) per field, which is O(n) Python-level work and will become a bottleneck as n_leads grows. Prefer vectorized joins (build accounts_df keyed by account_id and contacts_df keyed by contact_id and merge once) or Series.map from id→value dicts for each field.

    # -------------------------------------------------------------------
    acct_df = pd.DataFrame([a.to_dict() for a in population.accounts])[_ACCOUNT_JOIN_COLS]
    cont_df = pd.DataFrame([c.to_dict() for c in population.contacts])[_CONTACT_JOIN_COLS]
    lead_df = lead_df.merge(acct_df, on="account_id", how="left")
    lead_df = lead_df.merge(cont_df, on="contact_id", how="left")

    # -------------------------------------------------------------------
    # Select and order columns per canonical feature spec; apply dtypes.
    # -------------------------------------------------------------------
    snapshot = lead_df[_SNAPSHOT_COLUMNS].copy()
    for col, dtype in _SNAPSHOT_DTYPES.items():
        if col in snapshot.columns:
            snapshot[col] = snapshot[col].astype(dtype)

    return snapshot

leadforge/render/snapshots.py:208

• The dtype enforcement loop suppresses ValueError/TypeError during astype() and continues silently. If a column can’t be cast to the canonical feature dtype, this will hide schema drift and can break downstream consumers unexpectedly. Consider either letting the exception propagate or raising a clearer error that includes the column name and intended dtype.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Implements Milestone 8’s render/bundle “observation model” so a full Generator.generate() run can be rendered into a canonical on-disk bundle (Parquet tables, snapshot/task splits, and a provenance/integrity manifest).

Changes:

• Add render layer modules for relational export, lead snapshot construction, deterministic task splitting, and manifest hashing/writing.
• Add bundle writer orchestration and implement Generator.generate() end-to-end, enriching WorldBundle to carry simulation outputs.
• Add comprehensive render/bundle tests and update generator tests to assert a populated WorldBundle.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
leadforge/render/relational.py	Converts SimulationResult/PopulationResult into typed DataFrames for each relational table
leadforge/render/snapshots.py	Builds the ML-ready, leakage-free lead snapshot with vectorized aggregates + joins
leadforge/render/tasks.py	Writes deterministic train/valid/test Parquet splits and a task_manifest.json
leadforge/render/manifests.py	Builds/writes manifest.json with row counts and SHA-256 hashes
leadforge/api/bundle.py	Orchestrates full bundle writing pipeline (tables → snapshot/tasks → docs → manifest)
leadforge/api/generator.py	Implements Generator.generate() to produce a populated WorldBundle
leadforge/core/models.py	Extends WorldBundle and adds save() that delegates to bundle writer
leadforge/schema/entities.py	Adds EntityRowProtocol and tightens typing for the entity registry
tests/render/test_render.py	New end-to-end tests for render layer correctness, determinism, and bundle outputs
tests/api/test_generator.py	Updates generator test to validate generate() returns a populated bundle
.agent-plan.md	Updates milestone status/roadmap notes

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[github-actions]

pr-agent-context report:

No unresolved review comments, failing checks, or actionable patch coverage gaps were found on PR
#13. Treat this PR as all clear unless new signals appear.

Run metadata:

Tool ref: v4
Tool version: 4.0.20
Trigger: commit pushed
Workflow run: 25074720814 attempt 1
Comment timestamp: 2026-04-28T20:00:55.614579+00:00
PR head commit: 5629f12f55731fa00f49dfa64bc0ef172d65d5d0