Skip to content

Invalidate only package-dependent files when a Composer package changes#5933

Merged
ondrejmirtes merged 1 commit into
phpstan:2.2.xfrom
SanderMuller:c1-package-granular-invalidation
Jun 25, 2026
Merged

Invalidate only package-dependent files when a Composer package changes#5933
ondrejmirtes merged 1 commit into
phpstan:2.2.xfrom
SanderMuller:c1-package-granular-invalidation

Conversation

@SanderMuller

Copy link
Copy Markdown
Contributor

Today any change to composer.lock / installed.json marks the whole result cache as outdated, so the next run re-analyses every file. That happens even when the bumped package is a dev tool or a transitive dependency that no analysed file actually uses, which turns a routine composer update into a full cold run.

This makes that invalidation package-granular: a lock change re-analyses only the files that depend on a package whose version actually changed, and falls back to the current full re-analysis whenever it can't be sure.

How it works

  • Record. During analysis, each file already collects the reflection dependencies it used (NodeDependencies), and at getFileDependencies() it drops the ones that resolve into vendor before storing the file→file edges. C1 keeps those dropped entries and maps each to its Composer package, scoped to the analysed project's vendor directory. PHPStan's own bundled dependencies are excluded, because they track the PHPStan version (already covered by CACHE_VERSION and the PHP-version meta key), not the project's lock file. The package name is resolved through installed.php's per-package install paths rather than a vendor/<v>/<p>/ guess, so custom installer paths, path repositories and symlinks resolve correctly.
  • Store. The per-file package list is threaded through FileAnalyserResultAnalyserResult (including the parallel-worker IPC) and persisted in the result cache alongside the existing per-file dependencies. A per-package version map from installed.php goes into the cache meta.
  • Restore. When the only meta difference is composerLocks / composerInstalled, the cache diffs the stored per-package versions against the current installed.php, seeds $filesToAnalyse with the files that depend on a changed package, and then falls through into the existing incremental loop. That loop still adds content-hash-changed files and still propagates dependents on an exported-signature change, so nothing about the existing invalidation is weakened.

Correctness

The package edges are the same reflection dependencies PHPStan already trusts for file-to-file incremental analysis. If a file's analysis reflected a package's code, the file is recorded against that package, so a change to the package re-analyses the file by the same principle as the existing machinery. C1 adds no new dependency tracking; it keeps a portion of the data it was discarding.

Anything outside that is conservative:

  • Any meta difference other than the lock keys (config, PHP version, extensions, stubs, ...) still triggers a full re-analysis.
  • If installed.php is unreadable or not shaped as expected, it falls back to the full nuke. Over-invalidation is always safe; the change is built so it can only ever fall back to today's behaviour, never under-invalidate.
  • Stub-overridden vendor classes resolve to the stub file, not the vendor file, so a vendor-source change to a stubbed class correctly does not re-analyse (the stub is authoritative).

I verified the seeded file set against a full cold run after a dependency bump and got byte-identical results across an unused-package bump (0 files), a narrow real dependency, a wide DI-interface package, and a purely-transitive package that no project file directly uses.

Measured impact

Tempest (1519 files), warm cache, then a single-package bump. CPU = user+sys, the load-robust metric:

bump files re-analysed CPU vs cold
cold (full) 1519 1x (56.6s)
unused dep (dev / transitive) 0 ~67x less
narrow dep (league/flysystem) 17 ~36x less
wide dep (psr/container, a DI interface) 147 ~6x less

The frequent case (a dev, transitive or unused bump) becomes a warm run. Even the widest core package stays around 10% of the project, because a ref-only bump with no code change re-analyses just the direct package-dependents and does not propagate further. Second corpus rector-src (1151 files, an AST tool) shows the same shape; only the identity of the one or two genuinely-wide domain packages differs per project (nikic/php-parser there, psr/container on Tempest).

Cost of recording the package edges, measured on the same corpus against the no-C1 base:

metric no-C1 C1 overhead
cold CPU 55.32s 56.41s ~2%
warm CPU (no lock change) 0.53s 0.53s 0%
result-cache size 16.40 MB 16.60 MB +1.2%

So it adds about 2% to a cold run and 1% to the cache, nothing to the warm path, and in return turns the common dependency bump from a full re-analysis into a warm run.

Notes

The package reasoning is unit-tested in PackageDependencyResolverTest: the path→package resolution and the installed.php version diff that decides which packages changed. The seeding itself reuses the existing incremental loop.

Beyond the synthetic cases above, I checked end-to-end that a dependency bump produces a byte-identical error set to a full cold run on two real projects (~1k and ~4k analysed files): an unused dev dependency turns the run warm (0 files re-analysed, identical errors), and a pervasive runtime dependency re-analyses its dependents and still matches the cold result. Happy to add a committed end-to-end invalidation test in whatever shape you'd prefer.

@SanderMuller
SanderMuller force-pushed the c1-package-granular-invalidation branch from b98431a to 30bebd3 Compare June 25, 2026 09:22
@SanderMuller

Copy link
Copy Markdown
Contributor Author

Pushed a fix for the two red checks:

  • Coding Standard: an import-ordering nit in AnalyserTest.php (the new use wasn't alphabetically placed). Fixed.
  • Result cache E2E (result-cache-scanned): the "metadata do not match" message now lists scannedFiles instead of projectConfig, scannedFiles, so I updated the assertion.

The message change is a side effect of a fix this PR needs. isMetaDifferent() already normalizes the current projectConfig (ksort + Neon::encode) before comparing, because the cached value is stored as a Neon-encoded string while the current one is a raw array. getMetaKeyDifferences() did not normalize, so it compared the encoded string against the raw array and always reported projectConfig as changed whenever a config file existed and any other key differed (that's why the scanned-file case showed projectConfig even though the config was identical). C1's lock-only detection reads getMetaKeyDifferences() to decide whether only composerLocks/composerInstalled changed, so the stray projectConfig would otherwise prevent the granular path; I couldn't simply ignore projectConfig there without risking missing a real config change. Aligning getMetaKeyDifferences() with isMetaDifferent() fixes both, and as a bonus the diagnostic no longer lists a key that didn't actually change.

If you'd rather C1 not touch that shared message, I can instead do the normalized projectConfig comparison privately inside the lock-only check and leave getMetaKeyDifferences() as is. Let me know which you prefer.

@ondrejmirtes

Copy link
Copy Markdown
Member

Ohh I like this but there's a problem - often there might be dependencies between vendor packages.

A model situation:

  1. class A (ours, analysed) extends class B from vendor/b/foo
  2. Class B from vendor/b/foo extends class C from vendor/c/foo
  3. This means that class A has a dependency on c/foo package
  4. But this isn't recorded in the result cache I think? (But it might be given we recurse the whole tree).

We need to inspect whether file with class A gets invalidated (and most importantly files depending on class A, like one that calls a method on class A) when only package c/foo is updated.

@SanderMuller

Copy link
Copy Markdown
Contributor Author

Ohh I like this but there's a problem - often there might be dependencies between vendor packages.

A model situation:

  1. class A (ours, analysed) extends class B from vendor/b/foo
  2. Class B from vendor/b/foo extends class C from vendor/c/foo
  3. This means that class A has a dependency on c/foo package
  4. But this isn't recorded in the result cache I think? (But it might be given we recurse the whole tree).

We need to inspect whether file with class A gets invalidated (and most importantly files depending on class A, like one that calls a method on class A) when only package c/foo is updated.

Will dive into this!

@SanderMuller

Copy link
Copy Markdown
Contributor Author

Good question, this was the case I was most worried about too, so I built your exact model and tested it. Two vendor packages via path repos: c/foo has class C { public function methodFromC(): int }, b/foo has class B extends C, and the analysed project has class A extends B plus class D { public function run(A $a): int { return $a->methodFromC(); } }.

It's recorded, via exactly the tree recursion you guessed. The result cache:

src/A.php => ["b/foo", "c/foo"]
src/D.php => ["b/foo", "c/foo"]

Both record c/foo directly. And it's not limited to files that call a C-derived member: a class that only type-hints A (class E { public function take(A $a): void {} }) also records ["b/foo", "c/foo"]. Reflecting A resolves its whole ancestry down to C, so C's file lands in the dependency set of any file that references A.

Then I simulated a c/foo update (changed methodFromC(): int to string in the installed vendor/c/foo, bumped its reference in installed.php / installed.json) and re-ran:

Composer packages changed (c/foo); re-analysing only the files depending on them.
Result cache restored. 2 files will be reanalysed.

A and D both re-analysed and the output was byte-identical to a full cold run (both report src/D.php:3 Method App\D::run() should return int but returns string). So a c/foo-only bump correctly invalidates A and the files using it.

The reason it's sound: these are the same reflection dependencies PHPStan already collects for file-to-file incremental analysis. C1 adds no new tracking; it keeps the vendor entries NodeDependencies was dropping (mapping each to its package) instead of discarding them. So if analysing a file reflected C's code, that file depends on c/foo, by the same principle as today.

The one transitive case that isn't recorded: a package used only inside another vendor package's method bodies, never in its reflected API (say b/foo's B::doStuff() body calls c/foo's CHelper, but B's signature doesn't mention it). There the project file records only b/foo. I checked that case too: changing that internal c/foo code and bumping it leaves the analysis byte-identical, because B's declared API is unchanged, so the change can't affect what your code sees. Not re-analysing there is correct, the same basis as the current incremental machinery.

Happy to share the reproducer.

Today any change to composer.lock or installed.json marks the whole
result cache as outdated and forces a full re-analysis, even when the
bumped package is a dev tool or a transitive dependency that no analysed
file uses.

This records, per file, the project-vendor packages whose code the file
reflected during analysis. The edges reuse the reflection dependencies
NodeDependencies already collects for file-to-file incremental
invalidation (instead of discarding the vendor entries at
NodeDependencies::getFileDependencies), so the no-under-invalidation
property is inherited: if a file's analysis reflected a package's code,
the file is recorded against that package. Packages are resolved through
the project's installed.php install paths and scoped to the analysed
project's vendor, so PHPStan's own bundled dependencies (already covered
by CACHE_VERSION and the PHP version) are excluded.

When the only meta difference is composerLocks/composerInstalled, the
result cache now diffs the per-package versions, seeds only the files
that depend on a changed package, and falls through into the existing
incremental loop, which still adds content-changed files and propagates
dependents on a signature change. Any other meta difference still
triggers a full re-analysis, and an unreadable or unexpectedly shaped
installed.php falls back to the full nuke, so the change never
under-invalidates.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@SanderMuller
SanderMuller force-pushed the c1-package-granular-invalidation branch from 30bebd3 to 8101b09 Compare June 25, 2026 10:04
@ondrejmirtes

Copy link
Copy Markdown
Member

Super nice! I'm thinking about other situations where the result cache invalidation on composer change is important. But I couldn't figure out a scenario where it'd matter in this context.

I'm going to merge this. Please be ready for some feedback after this releases (it might not be immediately), it's possible we'll break some scenario we didn't think of.

@ondrejmirtes
ondrejmirtes merged commit 29140da into phpstan:2.2.x Jun 25, 2026
670 of 671 checks passed
@ondrejmirtes

Copy link
Copy Markdown
Member

Thank you.

@SanderMuller

Copy link
Copy Markdown
Contributor Author

Super nice! I'm thinking about other situations where the result cache invalidation on composer change is important. But I couldn't figure out a scenario where it'd matter in this context.

I'm going to merge this. Please be ready for some feedback after this releases (it might not be immediately), it's possible we'll break some scenario we didn't think of.

Great! I've saved all related context and learnings while working on this, ready for feedback when it comes in. Would be amazing if this feature can survive the tests :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants