Skip to content

Add CSV I/O for PointDataset (cluster-friendly alternative to JSON) #446

Description

@Jammy2211

Motivation

Strong-lens cluster workflows can involve tens or hundreds of multiply-imaged
background sources, each currently stored as its own PointDataset JSON file.
Editing hundreds of JSON files by hand (to add/remove images, correct
positions, retune noise values, paste in flux/time-delay measurements) is
fiddly and error-prone.

A single CSV — one row per image, grouped by name — is a much more natural
cluster input. It opens cleanly in a spreadsheet, diffs line-by-line in git,
and can be generated by standard astronomy tooling. JSON stays the canonical
exact-round-trip format; CSV becomes the hand-editable form preferred for
clusters.

Proposed schema (one row per image)

| name | y | x | positions_noise | flux | flux_noise | time_delay | time_delay_noise |

  • name, y, x, positions_noise are always required.
  • flux / flux_noise and time_delay / time_delay_noise are optional;
    columns dropped when blank for every row.
  • Rows are grouped by name → one PointDataset per group.

Library changes (PyAutoLens)

  1. autolens/point/dataset.py
    • PointDataset.to_csv(file_path)csv stdlib only, no pandas dependency.
    • classmethod PointDataset.from_csv(file_path, name: Optional[str] = None);
      name= required when the CSV has multiple groups, auto-picked when there
      is exactly one.
    • Module-level output_to_csv(datasets: List[PointDataset], file_path) and
      list_from_csv(file_path) -> List[PointDataset].
    • Fix the stale PointDict reference in the module docstring — the class
      no longer exists; the workflow is now lists of PointDataset.
  2. autolens/__init__.py: re-export output_to_csv and list_from_csv
    alongside the existing output_to_json / from_json re-exports.
  3. test_autolens/point/test_dataset.py: round-trip tests covering
    • positions-only
    • positions + fluxes + fluxes_noise_map
    • positions + time_delays + time_delays_noise_map
    • positions + fluxes + time_delays
    • heterogeneous list (one with fluxes, one without) — verifies
      optional-column handling at the list level

Workspace changes (autolens_workspace) — follow-up PR

  1. scripts/point_source/simulator.py: after every al.output_to_json(...)
    that writes a PointDataset, emit an equivalent al.output_to_csv(...)
    with the matching .csv filename and add narration — JSON is the exact
    round-trip, CSV is the hand-editable spreadsheet form preferred for
    clusters.
  2. New scripts/point_source/features/csv.py:
    • write a synthetic list of ~3 PointDataset objects to a single CSV via
      al.output_to_csv
    • reload with al.list_from_csv
    • build [FitPointDataset, ...] per loaded dataset and print info
  3. Do not change the modeling scripts yet; JSON remains the canonical
    modeling input until the cluster examples land.

Non-goals (explicitly deferred)

  • FitPointDataset.to_csv residual / chi-squared export — stretch goal;
    follow-up issue if worth it.
  • Introducing a PointDatasetList class — keep list-level helpers as plain
    functions for now.
  • Adding pandas as a dependency.

Classification

Library-first. Workspace examples ship in a follow-up once the library PR is
released.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions