Invalidate only package-dependent files when a Composer package changes#5933
Conversation
b98431a to
30bebd3
Compare
|
Pushed a fix for the two red checks:
The message change is a side effect of a fix this PR needs. If you'd rather C1 not touch that shared message, I can instead do the normalized |
|
Ohh I like this but there's a problem - often there might be dependencies between vendor packages. A model situation:
We need to inspect whether file with class A gets invalidated (and most importantly files depending on class A, like one that calls a method on class A) when only package c/foo is updated. |
Will dive into this! |
|
Good question, this was the case I was most worried about too, so I built your exact model and tested it. Two vendor packages via path repos: It's recorded, via exactly the tree recursion you guessed. The result cache: Both record Then I simulated a
The reason it's sound: these are the same reflection dependencies PHPStan already collects for file-to-file incremental analysis. C1 adds no new tracking; it keeps the vendor entries The one transitive case that isn't recorded: a package used only inside another vendor package's method bodies, never in its reflected API (say Happy to share the reproducer. |
Today any change to composer.lock or installed.json marks the whole result cache as outdated and forces a full re-analysis, even when the bumped package is a dev tool or a transitive dependency that no analysed file uses. This records, per file, the project-vendor packages whose code the file reflected during analysis. The edges reuse the reflection dependencies NodeDependencies already collects for file-to-file incremental invalidation (instead of discarding the vendor entries at NodeDependencies::getFileDependencies), so the no-under-invalidation property is inherited: if a file's analysis reflected a package's code, the file is recorded against that package. Packages are resolved through the project's installed.php install paths and scoped to the analysed project's vendor, so PHPStan's own bundled dependencies (already covered by CACHE_VERSION and the PHP version) are excluded. When the only meta difference is composerLocks/composerInstalled, the result cache now diffs the per-package versions, seeds only the files that depend on a changed package, and falls through into the existing incremental loop, which still adds content-changed files and propagates dependents on a signature change. Any other meta difference still triggers a full re-analysis, and an unreadable or unexpectedly shaped installed.php falls back to the full nuke, so the change never under-invalidates. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
30bebd3 to
8101b09
Compare
|
Super nice! I'm thinking about other situations where the result cache invalidation on composer change is important. But I couldn't figure out a scenario where it'd matter in this context. I'm going to merge this. Please be ready for some feedback after this releases (it might not be immediately), it's possible we'll break some scenario we didn't think of. |
|
Thank you. |
Great! I've saved all related context and learnings while working on this, ready for feedback when it comes in. Would be amazing if this feature can survive the tests :) |
Today any change to
composer.lock/installed.jsonmarks the whole result cache as outdated, so the next run re-analyses every file. That happens even when the bumped package is a dev tool or a transitive dependency that no analysed file actually uses, which turns a routinecomposer updateinto a full cold run.This makes that invalidation package-granular: a lock change re-analyses only the files that depend on a package whose version actually changed, and falls back to the current full re-analysis whenever it can't be sure.
How it works
NodeDependencies), and atgetFileDependencies()it drops the ones that resolve intovendorbefore storing the file→file edges. C1 keeps those dropped entries and maps each to its Composer package, scoped to the analysed project's vendor directory. PHPStan's own bundled dependencies are excluded, because they track the PHPStan version (already covered byCACHE_VERSIONand the PHP-version meta key), not the project's lock file. The package name is resolved throughinstalled.php's per-package install paths rather than avendor/<v>/<p>/guess, so custom installer paths, path repositories and symlinks resolve correctly.FileAnalyserResult→AnalyserResult(including the parallel-worker IPC) and persisted in the result cache alongside the existing per-file dependencies. A per-package version map frominstalled.phpgoes into the cache meta.composerLocks/composerInstalled, the cache diffs the stored per-package versions against the currentinstalled.php, seeds$filesToAnalysewith the files that depend on a changed package, and then falls through into the existing incremental loop. That loop still adds content-hash-changed files and still propagates dependents on an exported-signature change, so nothing about the existing invalidation is weakened.Correctness
The package edges are the same reflection dependencies PHPStan already trusts for file-to-file incremental analysis. If a file's analysis reflected a package's code, the file is recorded against that package, so a change to the package re-analyses the file by the same principle as the existing machinery. C1 adds no new dependency tracking; it keeps a portion of the data it was discarding.
Anything outside that is conservative:
installed.phpis unreadable or not shaped as expected, it falls back to the full nuke. Over-invalidation is always safe; the change is built so it can only ever fall back to today's behaviour, never under-invalidate.I verified the seeded file set against a full cold run after a dependency bump and got byte-identical results across an unused-package bump (0 files), a narrow real dependency, a wide DI-interface package, and a purely-transitive package that no project file directly uses.
Measured impact
Tempest (1519 files), warm cache, then a single-package bump. CPU = user+sys, the load-robust metric:
league/flysystem)psr/container, a DI interface)The frequent case (a dev, transitive or unused bump) becomes a warm run. Even the widest core package stays around 10% of the project, because a ref-only bump with no code change re-analyses just the direct package-dependents and does not propagate further. Second corpus rector-src (1151 files, an AST tool) shows the same shape; only the identity of the one or two genuinely-wide domain packages differs per project (
nikic/php-parserthere,psr/containeron Tempest).Cost of recording the package edges, measured on the same corpus against the no-C1 base:
So it adds about 2% to a cold run and 1% to the cache, nothing to the warm path, and in return turns the common dependency bump from a full re-analysis into a warm run.
Notes
The package reasoning is unit-tested in
PackageDependencyResolverTest: the path→package resolution and theinstalled.phpversion diff that decides which packages changed. The seeding itself reuses the existing incremental loop.Beyond the synthetic cases above, I checked end-to-end that a dependency bump produces a byte-identical error set to a full cold run on two real projects (~1k and ~4k analysed files): an unused dev dependency turns the run warm (0 files re-analysed, identical errors), and a pervasive runtime dependency re-analyses its dependents and still matches the cold result. Happy to add a committed end-to-end invalidation test in whatever shape you'd prefer.