Restructure the lecture series: chaining into chapter 2, pixelizations as chapter 3 with a Bayesian formalism lecture, new Scaling Up Galaxies chapter 4

AGENTS.md

@@ -10,8 +10,9 @@ instructions for this repo.
 - `scripts/` — Runnable Python tutorial scripts:
   - `chapter_1_introduction/` — Grids, light profiles, galaxies, data, fitting
   - `chapter_2_modeling/` — Non-linear searches, Bayesian inference, galaxy modeling
-  - `chapter_3_search_chaining/` — Search chaining, prior passing, automated pipelines
-  - `chapter_4_pixelizations/` — Pixelized reconstruction, inversions, regularization
+  - `chapter_3_pixelizations/` — Pixelized reconstruction, inversions, regularization, the Bayesian
+    formalism
+  - `chapter_4_scaling_up_galaxies/` — Extra galaxies, blended multi-galaxy systems, cluster fields
   - `chapter_optional/` — Alternative searches and advanced topics
   - `simulators/` — Simulator scripts that generate the tutorial datasets at runtime
 - `notebooks/` — Jupyter versions, generated from `scripts/` (do not edit directly)
@@ -36,7 +37,7 @@ fast smoke run). **Dataset realism:** automated runs **do** cap datasets —
 workspaces. (This paragraph previously claimed the opposite; the claim was untrue and went unnoticed
 because the chapters that break under the cap were never in the smoke list.) Tutorials must therefore
 work at **both** resolutions: never hardcode an index or a shape derived from the full-resolution
-dataset. `chapter_4_pixelizations/tutorial_3_inversions.py` is the cautionary case — it sized its mesh
+dataset. `chapter_3_pixelizations/tutorial_3_inversions.py` is the cautionary case — it sized its mesh
 from `dataset.shape_native`, giving 10000 mesh pixels at full resolution but 256 under the cap, so its
 fixed `pix_indexes` ran off the end.
 

README.md

@@ -23,11 +23,11 @@ more appropriate — they are concise and assume the concepts taught in **HowToG
 - `chapter_1_introduction` — An introduction to galaxy morphology and **PyAutoGalaxy**: grids, light
   profiles, galaxies, simulated imaging data, and fitting.
 - `chapter_2_modeling` — Bayesian inference, non-linear searches, and how to fit a galaxy model to CCD
-  imaging data with **PyAutoGalaxy**.
-- `chapter_3_search_chaining` — Chaining multiple non-linear searches together to build automated galaxy
-  modeling pipelines for complex systems.
-- `chapter_4_pixelizations` — Pixelized source reconstructions (inversions) for galaxies with irregular
-  morphologies.
+  imaging data with **PyAutoGalaxy**, ending with search chaining and automated pipelines.
+- `chapter_3_pixelizations` — Pixelized reconstructions (inversions) for galaxies with irregular
+  morphologies, including the Bayesian formalism underpinning them.
+- `chapter_4_scaling_up_galaxies` — Scaling galaxy modeling up beyond a single galaxy: extra galaxies,
+  blended multi-galaxy systems, and cluster fields.
 - `chapter_optional` — Optional tutorials on alternative non-linear searches and other advanced topics.
 
 **HowToGalaxy** currently sits at four chapters. Each chapter will take around a day to work through.

llms.txt

@@ -26,8 +26,8 @@ real analysis?" below).
 
 - **Chapter 1 — Introduction** → [scripts/chapter_1_introduction/](scripts/chapter_1_introduction/): Galaxies from first principles: grids & galaxies, light profiles, simulating data, and fitting data. The conceptual foundation.
 - **Chapter 2 — Modeling** → [scripts/chapter_2_modeling/](scripts/chapter_2_modeling/): Fitting a galaxy model to data with a non-linear search — parameter spaces, priors, dealing with failure, linear light profiles, masking, and reading results.
-- **Chapter 3 — Search Chaining** → [scripts/chapter_3_search_chaining/](scripts/chapter_3_search_chaining/): Breaking a hard fit into a sequence of searches (prior passing) and modeling multi-galaxy images.
-- **Chapter 4 — Pixelizations** → [scripts/chapter_4_pixelizations/](scripts/chapter_4_pixelizations/): Reconstructing irregular galaxy structure on a pixel-grid instead of with light profiles — mappers, inversions, Bayesian regularization.
+- **Chapter 3 — Pixelizations** → [scripts/chapter_3_pixelizations/](scripts/chapter_3_pixelizations/): Reconstructing irregular galaxy structure on a pixel-grid instead of with light profiles — mappers, inversions, Bayesian regularization, the Bayesian formalism.
+- **Chapter 4 — Scaling Up Galaxies** → [scripts/chapter_4_scaling_up_galaxies/](scripts/chapter_4_scaling_up_galaxies/): Beyond a single galaxy — extra galaxies, blended multi-galaxy systems, and cluster fields.
 
 ## I want to understand…
 
@@ -37,8 +37,8 @@ real analysis?" below).
 - **How does a non-linear search find a model?** → [scripts/chapter_2_modeling/tutorial_1_non_linear_search.py](scripts/chapter_2_modeling/tutorial_1_non_linear_search.py): Parameter space, priors, MLE / MCMC / nested sampling.
 - **My fit found a wrong (local-maxima) solution — what do I do?** → [scripts/chapter_2_modeling/tutorial_4_dealing_with_failure.py](scripts/chapter_2_modeling/tutorial_4_dealing_with_failure.py): Prior tuning, reducing complexity, and looking harder.
 - **What are linear light profiles / MGE?** → [scripts/chapter_2_modeling/tutorial_5_linear_profiles.py](scripts/chapter_2_modeling/tutorial_5_linear_profiles.py): Solving intensities linearly; the Multi-Gaussian Expansion basis.
-- **How do I model an image with two galaxies?** → [scripts/chapter_3_search_chaining/tutorial_3_x2_galaxies.py](scripts/chapter_3_search_chaining/tutorial_3_x2_galaxies.py): Search chaining to handle multi-galaxy degeneracies.
-- **How do I reconstruct irregular structure on a pixel grid?** → [scripts/chapter_4_pixelizations/tutorial_1_pixelizations.py](scripts/chapter_4_pixelizations/tutorial_1_pixelizations.py) through [tutorial_4_bayesian_regularization.py](scripts/chapter_4_pixelizations/tutorial_4_bayesian_regularization.py).
+- **How do I model an image with two galaxies?** → [scripts/chapter_4_scaling_up_galaxies/tutorial_2_multi_galaxy.py](scripts/chapter_4_scaling_up_galaxies/tutorial_2_multi_galaxy.py): Joint fitting of blended galaxies and their degeneracies.
+- **How do I reconstruct irregular structure on a pixel grid?** → [scripts/chapter_3_pixelizations/tutorial_1_pixelizations.py](scripts/chapter_3_pixelizations/tutorial_1_pixelizations.py) through [tutorial_4_bayesian_regularization.py](scripts/chapter_3_pixelizations/tutorial_4_bayesian_regularization.py).
 
 ## Ready for real analysis?
 

notebooks/README.md

@@ -14,7 +14,7 @@ Welcome to **HowToGalaxy** - The **PyAutoGalaxy** tutorial!
 
 # HOW TO TACKLE HowToGalaxy
 
-The **HowToGalaxy** lecture series current sits at 4 chapters, and each will take a day or so to go through
+The **HowToGalaxy** lecture series currently sits at 4 chapters, and each will take a day or so to go through
 properly. You probably want to be modeling galaxies faster than that! Furthermore, the concepts
 in the later chapters are pretty challenging, and familiarity and modeling is desirable before
 you tackle them.
@@ -85,31 +85,32 @@ fit CCD imaging data of a galaxy with a model. At the end, you'll understand:
 2. How to fit a model to galaxy CCD imaging via a non-linear search.
 3. The trade-off between realism and complexity when choosing a model.
 4. Why an incorrect model may be inferred and how to prevent this from happening.
-5. The challenges that are involved in inferred a robust model in a computationally reasonable run-time.
+5. The challenges that are involved in inferring a robust model in a computationally reasonable run-time.
+6. How to chain non-linear searches together to build automated modeling pipelines with prior passing.
 
 **Once completed, you'll be ready to model your own galaxies with PyAutoGalaxy!**
 
 # OVERVIEW OF CHAPTER 3 (Intermediate)
 
-**Automated Modeling with non-linear search chaining**
+**Using an inversion to perform a pixelized morphology reconstruction**
 
-In chapter 3, we'll learn how to chain multiple non-linear searches together to build automated modeling pipelines
-which can:
-
-1. Break-down the fitting of a model using multiple non-linear searches and prior passing.
-2. Use a custom pipeline to fit data containing multiple galaxy where each galaxy is fitted one at a time.
-3. Fit the global structure of a galaxy, followed by faint morphological features like a bar.
-
-# OVERVIEW OF CHAPTER 4 (Intermediate)
-
-**Using an inverison to perform a pixelized morphology reconstructions**
-
-In chapter 4, we'll learn how to reconstruct morphology features of a galaxy using a pixel-grid, ensuring that we can
-fit an accurate model to sources with complex and irregular morphologies. You'll learn how to:
+In chapter 3, we'll learn how to reconstruct the morphological features of a galaxy using a pixel-grid, ensuring
+that we can fit an accurate model to galaxies with complex and irregular morphologies. You'll learn how to:
 
 1. Pixelize a galaxy reconstruction into pixels.
 2. Perform a linear inversion using this pixelization to reconstruct the galaxy's light.
 3. Apply a smoothness prior on the galaxy reconstruction, called regularization.
 4. Apply smoothing within a Bayesian framework to objectively quantify the reconstruction's complexity.
-5. Use alternative pixelizations, for example a Voronoi mesh.
+5. Write down the linear algebra and Bayesian evidence equations that underpin the whole framework.
 6. Use these features to fit a model via non-linear searches.
+
+# OVERVIEW OF CHAPTER 4 (Advanced)
+
+**Scaling Up Galaxies**
+
+In chapter 4, we'll scale galaxy modeling up beyond a single galaxy, learning how to:
+
+1. Handle extra galaxies near the one being fitted, by scaling their light out of the fit or modeling them explicitly.
+2. Model two or more blended galaxies simultaneously and understand the degeneracies this creates.
+3. Model cluster fields, where the brightest cluster galaxy is fitted richly and the member population is
+   composed from a catalogue.

notebooks/chapter_1_introduction/README.md

@@ -1,4 +1,4 @@
-In chapter 1, we introduce you to strong gravitational lensing and the core **PyAutoGalaxy** API.
+In chapter 1, we introduce you to galaxies, the light they emit and the core **PyAutoGalaxy** API.
 
 **Colab** links to every tutorial are included.
 

notebooks/chapter_1_introduction/tutorial_0_visualization.ipynb

@@ -63,7 +63,7 @@
    "metadata": {},
    "source": [
     "\n",
-    "# from autogalaxy import setup_notebook; setup_notebook()"
+    "from autogalaxy import setup_notebook; setup_notebook()"
    ],
    "outputs": [],
    "execution_count": null
@@ -186,12 +186,12 @@
     "\n",
     "Does the figure display correctly on your computer screen?\n",
     "\n",
-    "If not, you can customize common matplotlib options by passing them directly to `plot_array`:\n",
+    "If not, you can customize common plot options by passing them directly to `plot_array`:\n",
     "\n",
     " - `title=`: Set the figure title.\n",
-    " - `figsize=`: Control the figure size as a `(width, height)` tuple.\n",
     " - `colormap=`: Set the matplotlib colormap name (e.g. `\"jet\"`, `\"gray\"`).\n",
-    " - `xlabel=`, `ylabel=`: Override the default axis labels."
+    " - `use_log10=`: Plot the image on a log10 scale, revealing faint features.\n",
+    " - `vmin=`, `vmax=`: Override the minimum and maximum values of the colormap."
    ]
   },
   {
@@ -210,32 +210,30 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "Many matplotlib options can be customized, but for now we're only concerned with making sure figures display clear in\n",
+    "Many plot options can be customized, but for now we're only concerned with making sure figures display clearly in\n",
     "your Jupyter Notebooks. Nevertheless, a comprehensive API reference guide of all available plot arguments can\n",
     "be found in the `autogalaxy_workspace/*/guides/plot` package. You should check this out once you are more familiar with\n",
     "**PyAutoGalaxy**.\n",
     "\n",
-    "Ideally, we would not specify a `figsize` every time we plot an image. Fortunately, default values can be fully\n",
+    "Ideally, we would not specify these options every time we plot an image. Fortunately, default values can be fully\n",
     "customized via the config files.\n",
     "\n",
     "Checkout the `general.yaml` file in `autogalaxy_workspace/config/visualize`.\n",
     "\n",
-    "All default matplotlib values are here. There are a lot of entries, so lets focus on whats important for displaying\n",
-    "figures:\n",
+    "All default visualization values are here. There are a lot of entries, so lets focus on whats important for\n",
+    "displaying figures:\n",
     "\n",
-    " - mat_wrap.yaml -> Figure -> figure: -> figsize\n",
-    " - mat_wrap.yaml -> YLabel -> figure: -> fontsize\n",
-    " - mat_wrap.yaml -> XLabel -> figure: -> fontsize\n",
-    " - mat_wrap.yaml -> TickParams -> figure: -> labelsize\n",
-    " - mat_wrap.yaml -> YTicks -> figure: -> labelsize\n",
-    " - mat_wrap.yaml -> XTicks -> figure: -> labelsize\n",
+    " - general.yaml -> general -> backend: the matplotlib backend used for visualization.\n",
+    " - general.yaml -> colormap: the default colormap of 2D plots.\n",
+    " - general.yaml -> subplot_shape_to_figsize_factor: controls the size of subplot figures.\n",
+    " - general.yaml -> colorbar: the font sizes of colorbar labels.\n",
     "\n",
     "Don't worry about all the other files or options listed for now, as they'll make a lot more sense once you are familiar\n",
     "with **PyAutoGalaxy**.\n",
     "\n",
     "If you had to change any of the above settings to get the figures to display clearly, you should update their values\n",
     "in the corresponding config files above (you will need to reset your Jupyter notebook server for these changes to\n",
-    "take effect, so make sure you have the right values using the `figsize` argument in the cell above beforehand!).\n",
+    "take effect, so make sure you have the right values using the arguments in the cell above beforehand!).\n",
     "\n",
     "__Subplots__\n",
     "\n",
@@ -260,11 +258,12 @@
    "source": [
     "__Visuals__\n",
     "\n",
-    "Visuals can be added to any figure by passing them as keyword arguments directly to `plot_array`.\n",
+    "Visuals can be added to any figure by passing them as keyword arguments directly to `plot_array`, for\n",
+    "example `grid=`, `positions=` and `lines=`. The `autogalaxy_workspace/*/guides/plot` examples illustrate every\n",
+    "overlay argument.\n",
     "\n",
-    "For example, we can plot a mask on the image above by passing `mask=mask`.\n",
-    "\n",
-    "The `visuals` example illustrates every overlay argument, for example `mask=`, `grid=`, `positions=`, `lines=`, etc."
+    "Some visuals appear automatically. For example, if we apply a mask to the dataset, figures of it zoom into the\n",
+    "unmasked region and only show the unmasked data \u2014 as shown by the plot below, which applies an annular mask."
    ]
   },
   {
@@ -278,7 +277,9 @@
     "    outer_radius=3.0,\n",
     ")\n",
     "\n",
-    "aplt.plot_array(array=dataset.data, title=\"Data\")"
+    "dataset = dataset.apply_mask(mask=mask)\n",
+    "\n",
+    "aplt.plot_array(array=dataset.data, title=\"Data With Annular Mask\")"
    ],
    "outputs": [],
    "execution_count": null