Add Dagster+ serverless code location + deploy paths#70
Merged
jirhiker merged 3 commits intoJun 24, 2026
Conversation
Make the repo show up in Dagster+ as a serverless code location. - dagster_cloud.yaml: location die-orchestration, module orchestration.definitions, build dir orchestration (its pyproject carries the dagster deps + nmuwd path-source). - GitHub Actions: prod deploy on push to main + per-PR branch deployments, PEX fast deploy (no system GDAL needed; only shapely, which ships a bundled-GEOS wheel). - orchestration/deploy_serverless.sh: manual deploy via dagster-cloud serverless deploy-python-executable, same build definition as CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Verified via cross-reference (grep + framework-entry-point check) before removal; kept Flask/FastAPI route handlers, click commands, and inherited base-class slots that vulture flags as "unused". Removed: - Dead functions/classes: bounding_polygons get_* helpers (6), geo_utils utm/lonlat converters (+ orphaned PROJECTIONS), converter UnitConverter Protocol, logger Loggable, strategies OutputStrategy, source BaseContainerSource/BaseFileSource, transformer convert_units (deprecated), unifier get_sources_in_polygon/generate_site_bounds/ *_unification_test/get_datastream(s) + stale commented call block. - orchestration/assets/wells.py (build_wells_asset, imported nowhere). - 63 unused imports across backend/frontend/orchestration (autoflake). flake8 (E9,F63,F7,F82) + mypy clean; 427 tests collect with no import errors; local persister tests pass. ckan connector appears entirely unwired (config uses st2 instead) — flagged separately, not removed here. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the repo show up in Dagster+ as a serverless code location.
What
dagster_cloud.yaml— code locationdie-orchestration,module_name: orchestration.definitions, build dirorchestration(itspyproject.tomldeclares the dagster deps + thenmuwdpath-source the PEX builder recurses into forbackend/).dagster-cloud-deploy.yml(prod deploy on push tomain) +dagster-cloud-branch-deployments.yml(per-PR branch deployments). PEX fast deploy.orchestration/deploy_serverless.sh— manual deploy viadagster-cloud serverless deploy-python-executable, same build definition as CI.Why PEX, not Docker
shapely(bundled-GEOS wheel) — no fiona/pyogrio/geopandas, so no system GDAL. Docker build would be pure overhead.deploy-docker's bundled Dockerfile runspip install .on a barepyproject.toml(no README/source copied yet) → fails on this repo's hatchling layout.requirements.txt; an editable-e .line breaks it — so no rootrequirements.txt.CI and manual share one build definition. No drift.
Required before deploys run
GitHub repo secrets:
DAGSTER_CLOUD_API_TOKEN,ORGANIZATION_ID,DAGSTER_CLOUD_URL. Manual script needsDAGSTER_CLOUD_ORGANIZATION+DAGSTER_CLOUD_API_TOKENenv vars + local Docker.Known wart
Root pyproject
dev/geoserverextras get pulled into the deps PEX (builder concatenates all optional-dependency groups). Functional, just image bloat.🤖 Generated with Claude Code