BDMS-470: Migrate Stratigraphy

[jirhiker]

Why

This PR addresses the following problem / context:

• Staging lacks a first-class legacy stratigraphy table, so Stratigraphy.csv rows aren’t preserved outside the association transfer.
• Without a dedicated transfer + tests, stratigraphy CSV ingestion can regress silently.
• Alembic/test/transfer flows don’t share a single schema reset path for stratigraphy legacy data.

How

Implementation summary - the following was changed / added / removed:

• Added Alembic revision 1d2c3b4a5e67_create_nma_stratigraphy_table.py plus the Stratigraphy model (NMA_Stratigraphy table) and wired Thing’s relationship so legacy intervals can be queried off wells.
• Built transfers/stratigraphy_legacy.py with a batch upsert pipeline, hooked it into the main transfer worker (guarded by TRANSFER_NMA_STRATIGRAPHY), and exposed metrics for the new table.
• Created tests/test_stratigraphy_legacy.py to cover the model FK linkage and the transfer’s CSV ingestion path end to end.

Notes

Any special considerations, workarounds, or follow-up work to note?

• Run the new transfer after wells load so PointID→Thing resolutions succeed.
• The Alembic revision must be applied before running transfers/tests to ensure NMA_Stratigraphy exists.

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[Copilot]

Pull request overview

This PR adds support for ingesting legacy stratigraphy data from Stratigraphy.csv into a dedicated NMA_Stratigraphy table. Previously, stratigraphy rows weren't preserved as first-class records outside of association transfers.

Changes:

• Added NMA_Stratigraphy table model with relationship to Thing
• Implemented batch upsert transfer pipeline for Stratigraphy.csv with environment flag control
• Added tests covering model relationships and end-to-end CSV ingestion

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
alembic/versions/1d2c3b4a5e67_create_nma_stratigraphy_table.py	Creates NMA_Stratigraphy table schema with indexes and foreign key to Thing
db/nma_legacy.py	Defines Stratigraphy model with columns for lithology log data
db/thing.py	Adds stratigraphy_logs relationship to Thing model
transfers/stratigraphy_legacy.py	Implements StratigraphyLegacyTransferer with CSV parsing and batch upsert logic
transfers/transfer.py	Integrates stratigraphy legacy transfer into parallel and sequential execution paths
transfers/metrics.py	Adds nma_stratigraphy_metrics method for tracking transfer results
tests/test_stratigraphy_legacy.py	Tests model relationships and transfer functionality

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

[Copilot]

The _string_value method converts non-string, non-None values to strings, which may silently coerce unexpected types. Consider adding validation or logging when converting non-string values to help catch data quality issues in the CSV.

Suggested change

	return str(value)
	# Log when coercing non-string values to string to help catch data-quality issues.
	logger.warning(
	"Coercing non-string value to string in StratigraphyLegacyTransferer._string_value: "
	"value=%r (type=%s)",
	value,
	type(value).__name__,
	)
	try:
	coerced = str(value)
	except Exception:
	logger.exception(
	"Failed to coerce value to string in StratigraphyLegacyTransferer._string_value: "
	"value=%r (type=%s)",
	value,
	type(value).__name__,
	)
	return None
	trimmed = coerced.strip()
	return trimmed or None

[Copilot]

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 3 comments.

[Copilot]

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.