acc: write output.txt and out.requests.txt outside the bundle sync root - #6629
Merged
Merged
Conversation
Collaborator
Integration test reportCommit: f7176ae
Top 11 slowest tests (at least 2 minutes):
|
janniklasrose
approved these changes
Sep 11, 2026
denik
enabled auto-merge
September 11, 2026 13:39
The acceptance harness runs the CLI with the test tmpdir as both cwd and the bundle root, so it swept its own output.txt and out.requests.txt into the bundle sync. Both are rewritten continuously during the run, so their mtime kept advancing past the sync snapshot and they were re-uploaded on every deploy. The sync counts changed files (len(change.put)), so the "Files: N uploaded" line was non-deterministic on coarse-mtime filesystems and under CI load. Write both files to a per-subtest outputDir (a t.TempDir() sibling of the bundle root) and copy them back for comparison. Scripts and helpers read the relocated requests file via $OUT_REQUESTS. Removed the now-redundant `echo "*" > .gitignore` churn hack from the 16 tests where it only suppressed these harness files; kept it in the 13 that also write their own out.*/LOG.* files, which are not relocated. Co-authored-by: Isaac <no-reply@databricks.com>
OUT_REQUESTS is always set by the harness, so the "out.requests.txt" fallback was dead. Read it directly with os.environ[...] and drop the literal filename. Co-authored-by: Isaac <no-reply@databricks.com>
Committing the harness's raw request recording pulled in bootstrap noise (the /.well-known/databricks-config GET, scim/Me) and every response field, and it was the last file still forcing out.requests.txt back into the synced test dir. Replace the 37 committed recordings with a print_requests.py call that records just what each test asserts: --get // where the request flow and headers are the point (auth, oauth-in-run, the recorder selftests), a path filter for tests that assert one operation, and nothing for tests that already assert via print_requests.py/jq into output.txt. The harness now copies only output.txt back from the per-subtest output dir; out.requests.txt stays out of the sync entirely. Co-authored-by: Isaac <no-reply@databricks.com>
Follow-up to writing output.txt outside the sync root: rather than copying it back into tmpDir so the existing comparison finds it, compare it directly from outputDir. output.txt is produced on every run, so compare it explicitly and skip it in the tmpDir loop. Co-authored-by: Isaac <no-reply@databricks.com>
… on Windows
A script-wide MSYS_NO_PATHCONV=1 stops Git Bash from converting the print_requests.py
helper's own path, so python can't find it ("can't open file ...print_requests.py").
The tests that now assert via print_requests.py under a global setting (selftest/server,
lakeview/publish, workspace/repos/*) set MSYS_NO_PATHCONV=1 per-command on the calls
that need it instead, leaving print_requests.py on the normal path. No-op off Windows.
Co-authored-by: Isaac <no-reply@databricks.com>
…ded on main shouldSkip never matches output.txt (it has no variant infix), so compare it unconditionally. Rebase pulled in model_services and other tests whose Files counts still assumed output.txt/out.requests.txt were synced; regenerate them. Co-authored-by: Isaac <no-reply@databricks.com>
denik
force-pushed
the
denik/test-toml-stability
branch
from
September 11, 2026 14:35
3a9af9d to
e4ceb04
Compare
…tale) My earlier regen captured a local-only "AttributeError: 'Resources' object has no attribute 'add_model_service'" because the local pydabs build predates the feature. This test has no Files: line, so the PR doesn't affect it — restore main's golden. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 11, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 11, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
Collaborator
Integration test reportCommit: af185aa
197 interesting tests: 167 MISS, 13 flaky, 9 FAIL, 7 KNOWN, 1 SKIP
Top 50 slowest tests (at least 2 minutes):
|
denik
added a commit
that referenced
this pull request
Sep 13, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 13, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
This was referenced Sep 14, 2026
denik
added a commit
that referenced
this pull request
Sep 14, 2026
…ests These tests zeroed the bundle sync with a catch-all .gitignore to keep the flaky `Files: N uploaded` line stable. Since output.txt and out.requests.txt were moved out of the sync root (#6629), the remaining synced files are stable (written once), so the workaround is no longer needed. Remove it and let the deploy summary show the real, stable counts. Left in place: resource_deps/create_error (the .gitignore pins an injected 500 to jobs/create rather than the file upload) and volumes/uppercase-name (captures its own deploy output into a synced file, which re-uploads across deploys). Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
…#6665) Follow-up to #6629. Those tests used a catch-all `echo "*" > .gitignore` to zero the bundle sync and keep the flaky `Files: N uploaded` line stable. Now that `output.txt`/`out.requests.txt` live outside the sync root, the remaining synced files are written once and stable, so the workaround is obsolete — remove it and let the deploy summary show the real counts. Not touched: `resource_deps/create_error` (its `.gitignore` pins an injected 500 to `jobs/create` instead of the file upload) and `resources/volumes/uppercase-name` (it captures its own deploy output into a synced file that re-uploads across deploys). This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
#6629 moved output.txt/out.requests.txt out of the bundle sync root and added a dedicated doComparison call for output.txt; pass sortLines to it so SortLines still applies to the deploy action lines there (also fixes the build). Regenerate two resource_deps error-path goldens for the new Files counts / inline line. Co-authored-by: Isaac <no-reply@databricks.com>
denik
added a commit
that referenced
this pull request
Sep 14, 2026
The prior regen's -run pattern had '/' inside alternation groups, so Go's segment-by-segment matching skipped these families too. Clean per-family regen picks up #6629's lower Files counts (output.txt/out.requests.txt no longer synced) and the inline per-resource line. permissions/504/create verified stable across 10 runs. Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
…ot (#6629) The acceptance harness runs the CLI with the test tmpdir as both cwd and the bundle root, so its own `output.txt` and `out.requests.txt` got swept into the bundle sync. Both are rewritten continuously during the run, so they re-uploaded on every deploy and made the `Files: N uploaded` count non-deterministic. Write both to a per-subtest temp dir outside the bundle root; scripts read the requests file via `$OUT_REQUESTS`. While here, stop committing the raw `out.requests.txt` recording as a golden (37 tests). It captured bootstrap noise and every response field; each test now asserts via `print_requests.py` recording just what it cares about. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
janniklasrose
pushed a commit
that referenced
this pull request
Sep 15, 2026
…#6665) Follow-up to #6629. Those tests used a catch-all `echo "*" > .gitignore` to zero the bundle sync and keep the flaky `Files: N uploaded` line stable. Now that `output.txt`/`out.requests.txt` live outside the sync root, the remaining synced files are written once and stable, so the workaround is obsolete — remove it and let the deploy summary show the real counts. Not touched: `resource_deps/create_error` (its `.gitignore` pins an injected 500 to `jobs/create` instead of the file upload) and `resources/volumes/uppercase-name` (it captures its own deploy output into a synced file that re-uploads across deploys). This pull request and its description were written by Isaac. Co-authored-by: Isaac <no-reply@databricks.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The acceptance harness runs the CLI with the test tmpdir as both cwd and the bundle root, so its own
output.txtandout.requests.txtgot swept into the bundle sync. Both are rewritten continuously during the run, so they re-uploaded on every deploy and made theFiles: N uploadedcount non-deterministic. Write both to a per-subtest temp dir outside the bundle root; scripts read the requests file via$OUT_REQUESTS.While here, stop committing the raw
out.requests.txtrecording as a golden (37 tests). It captured bootstrap noise and every response field; each test now asserts viaprint_requests.pyrecording just what it cares about.This pull request and its description were written by Isaac.