feat: add --environment flag to sqlmesh janitor for scoped cleanup#5810
Open
mday-io wants to merge 2 commits into
Open
feat: add --environment flag to sqlmesh janitor for scoped cleanup#5810mday-io wants to merge 2 commits into
mday-io wants to merge 2 commits into
Conversation
df7b55c to
00992be
Compare
00992be to
e3ca817
Compare
7ad7a4c to
fa4cd32
Compare
Collaborator
|
@mday-io Not a blocker for merge, but a few things I noticed:
Let me know if I'm missing anything! |
StuffbyYuki
approved these changes
May 31, 2026
Signed-off-by: Michael Day <michael.day@cloudkitchens.com> Signed-off-by: mday-io <mdaytn@gmail.com>
fa4cd32 to
3fa6132
Compare
…nvalidate test - Extend _create_expiration_filter_expr to accept optional name= param, consolidating the repeated AND-name filter from get_expired_environments and delete_expired_environments into a single place - Add test_invalidate_environment_sync_calls_cleanup_with_name: verifies that invalidate_environment(..., sync=True) passes name= to _cleanup_environments, guarding the --sync bug fix at the Context level - Add test_invalidate_environment_no_sync_skips_cleanup: verifies that sync=False does not trigger delete_expired_environments
3fa6132 to
e3e0b07
Compare
Contributor
Author
Not tackling the notebooks wrapper for now, but created a new commit for the other comments. Thanks for the review! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Add an
--environment/-eflag tosqlmesh janitorthat scopes cleanup to a single named environment, and fixsqlmesh invalidate <env> --syncto only clean up the named environment rather than all currently-expired environments.New flag:
sqlmesh janitor --environment <name>removes views, schemas, and state records for the named environment only. If the environment is not expired or does not exist, a warning is emitted. Global snapshot expiration and interval compaction are skipped when this flag is set — those operations are cross-environment and cannot be safely scoped to a single one.Bug fix:
sqlmesh invalidate <env> --syncpreviously called_cleanup_environments()with no filter after marking the target environment expired, which caused all other currently-expired environments to be deleted as a side effect. It now scopes the cleanup to the named environment only.Test Plan
--gateway duckdb_persistentwith two environments (dev1,dev2). Invalidatingdev1and runningsqlmesh janitor --environment dev1cleaned updev1while leavingdev2untouched.test_state_sync.py).invalidate_environment(..., sync=True)passesname=to_cleanup_environmentsand thatsync=Falseskips cleanup entirely (test_context.py).Checklist
make styleand fixed any issuesmake fast-test)git commit -s) per the DCO