refactor(asf): unify Dropwizard service bootstrap into common/auth#5983
Open
Ma77Ball wants to merge 4 commits into
Open
refactor(asf): unify Dropwizard service bootstrap into common/auth#5983Ma77Ball wants to merge 4 commits into
Ma77Ball wants to merge 4 commits into
Conversation
github-actions
Bot
added
dependencies
Contributor
Automated Reviewer SuggestionsBased on the
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #5983 +/- ##
============================================
- Coverage 56.89% 56.86% -0.04%
- Complexity 3058 3075 +17
============================================
Files 1129 1127 -2
Lines 43802 43272 -530
Branches 4743 4668 -75
============================================
- Hits 24922 24607 -315
+ Misses 17449 17267 -182
+ Partials 1431 1398 -33
*This pull request uses carry forward flags. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
|
| config | throughput | MB/s | latency | max Δ latest / 7d | |
|---|---|---|---|---|---|
| 🔴 | bs=10 sw=10 sl=64 | 348 | 0.212 | 28,136/39,430/39,430 us | 🔴 +8.4% / 🔴 +158.0% |
| 🔴 | bs=100 sw=10 sl=64 | 789 | 0.481 | 126,175/144,019/144,019 us | 🔴 +5.5% / 🔴 +32.9% |
| ⚪ | bs=1000 sw=10 sl=64 | 892 | 0.544 | 1,119,170/1,167,585/1,167,585 us | ⚪ within ±5% / 🔴 +12.6% |
Baseline details
Latest main 24b587f from same runner
| config | metric | PR | latest main | 7d avg | Δ latest | Δ 7d |
|---|---|---|---|---|---|---|
| bs=10 sw=10 sl=64 | throughput | 348 tuples/sec | 352 tuples/sec | 770.95 tuples/sec | -1.1% | -54.9% |
| bs=10 sw=10 sl=64 | MB/s | 0.212 MB/s | 0.215 MB/s | 0.471 MB/s | -1.4% | -54.9% |
| bs=10 sw=10 sl=64 | p50 | 28,136 us | 28,836 us | 12,775 us | -2.4% | +120.2% |
| bs=10 sw=10 sl=64 | p95 | 39,430 us | 36,380 us | 15,286 us | +8.4% | +158.0% |
| bs=10 sw=10 sl=64 | p99 | 39,430 us | 36,380 us | 18,795 us | +8.4% | +109.8% |
| bs=100 sw=10 sl=64 | throughput | 789 tuples/sec | 805 tuples/sec | 976.93 tuples/sec | -2.0% | -19.2% |
| bs=100 sw=10 sl=64 | MB/s | 0.481 MB/s | 0.491 MB/s | 0.596 MB/s | -2.0% | -19.3% |
| bs=100 sw=10 sl=64 | p50 | 126,175 us | 122,644 us | 102,557 us | +2.9% | +23.0% |
| bs=100 sw=10 sl=64 | p95 | 144,019 us | 136,507 us | 108,383 us | +5.5% | +32.9% |
| bs=100 sw=10 sl=64 | p99 | 144,019 us | 136,507 us | 115,249 us | +5.5% | +25.0% |
| bs=1000 sw=10 sl=64 | throughput | 892 tuples/sec | 899 tuples/sec | 1,009 tuples/sec | -0.8% | -11.6% |
| bs=1000 sw=10 sl=64 | MB/s | 0.544 MB/s | 0.549 MB/s | 0.616 MB/s | -0.9% | -11.7% |
| bs=1000 sw=10 sl=64 | p50 | 1,119,170 us | 1,115,967 us | 997,695 us | +0.3% | +12.2% |
| bs=1000 sw=10 sl=64 | p95 | 1,167,585 us | 1,161,588 us | 1,036,731 us | +0.5% | +12.6% |
| bs=1000 sw=10 sl=64 | p99 | 1,167,585 us | 1,161,588 us | 1,069,334 us | +0.5% | +9.2% |
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,575.18,200,128000,348,0.212,28135.50,39430.33,39430.33
1,100,10,64,20,2535.78,2000,1280000,789,0.481,126175.29,144019.14,144019.14
2,1000,10,64,20,22424.65,20000,12800000,892,0.544,1119170.36,1167584.92,1167584.92
per-service init/run paths
Raise patch coverage for the bootstrap-unification
refactor, which had
left the extracted glue uncovered (Codecov patch was
~29%).
- ServiceBootstrapSpec: exercise initDatabase (the
shared SQL pool setup),
tolerating the no-DB case so it runs whether or
not a Postgres is reachable.
- Add initialize()/run() tests to each service
RunSpec so the bootstrap
wiring (configure, initDatabase, config-file path,
auth stack, HTTP path)
is executed and asserted.
initDatabase mutates the JVM-wide SqlServer
singleton that the DB-backed
suites in auth/access-control/notebook-migration
rely on, so mark those
three modules Test / parallelExecution := false to
keep the suites hermetic.
Contributor
|
hmm, @Ma77Ball two questions:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this PR?
ServiceBootstrapincommon/authwith three shared helpers:configure(YAML env-var substitution +DefaultScalaModuleregistration),initDatabase(open the SQL connection pool fromStorageConfig), andconfigFilePath(resolve$TEXERA_HOME/<service>/src/main/resources/<file>).ServiceBootstrap, removing the duplicatedinitialize()/initConnection/main()boilerplate.WorkflowCompilingServiceat the sharedRequestLoggingFilter.registerinstead of its hand-rolled inline servlet filter, andNotebookMigrationServiceat the sharedAuthFeatures.registerinstead of its own identicalregisterAuthFeatures.jackson-module-scalaas aprovideddependency ofcommon/authforServiceBootstrap'sDefaultScalaModule.Any related issues, documentation, discussions?
Closes: #5982
How was this PR tested?
ServiceBootstrapSpec: runsbt -J-Dnet.bytebuddy.experimental=true "Auth/testOnly *ServiceBootstrapSpec", expect it to verifyconfigurewraps the config source provider and registers the Scala module, and thatconfigFilePathreturns an absolute path ending in the conventional<service>/src/main/resources/<file>suffix.sbt "ConfigService/testOnly *RunSpec" "AccessControlService/testOnly *RunSpec" "WorkflowCompilingService/testOnly *RunSpec" "ComputingUnitManagingService/testOnly *RunSpec" "NotebookMigrationService/testOnly *RunSpec"plusAuthFeaturesSpec, confirming each service still registers the same resources and auth stack.sbt "Auth/compile" "ConfigService/compile" "AccessControlService/compile" "FileService/compile" "ComputingUnitManagingService/compile" "WorkflowCompilingService/compile" "NotebookMigrationService/compile".-J-Dnet.bytebuddy.experimental=trueandFileServiceRunSpeccould not be instrumented locally; all of these run normally on CI's JDK 21.Was this PR authored or co-authored using generative AI tooling?
Co-authored with Claude Opus 4.8 in compliance with ASF