[2.2.3 perf regression fix] PackageDependencyResolver: memoize resolvePackage per file#5961
Conversation
|
how to reproduce the problem? |
resolvePackage() maps a file to its owning Composer package by scanning every install path with str_starts_with(). It is called from NodeDependencies::getPackageDependencies() once per dependency file of every analysed file, so the same vendor files are resolved over and over, and each call is linear in the number of installed packages. The install-path map is already memoized, but the per-file lookup was not, so on projects with a large dependency tree this scan dominates cold analysis. Memoize the result per file, including the null result for files that belong to no project package (PHPStan's own bundled dependencies), which otherwise pay a full scan on every call. The resolver is a singleton service, so the cache persists across all files in a worker. Same approach as 200a203 (DependencyResolver: reduce duplicate work). The result is unchanged: the cache is keyed by file, and a file's owning package cannot change during a run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
da143ca to
b8713b5
Compare
Cold analysis of a project with a large dependency tree, comparing 2.2.2 to 2.2.3. On a project here (~265 installed packages, ~1800 files, level 7), interleaved cold runs (result cache cleared before each, repeated to be robust to machine load):
The same holds single-process ( On the memory report in #14892: this PR is a CPU fix. It does not reduce the stored There is also a residual ~7% above 2.2.2 that remains after this PR and is not |
|
Thank you! |
Follow-up to #5933, which introduced
PackageDependencyResolver.Problem
resolvePackage()maps a file to its owning Composer package by scanning every install path withstr_starts_with(). It is called fromNodeDependencies::getPackageDependencies()once per dependency file of every analysed file, so the same vendor files are resolved over and over, and each call is linear in the number of installed packages. The install-path map (getInstallPathToPackage()) is already memoized, but the per-file lookup was not.A cold profile of a project with a large dependency tree (~265 installed packages, ~1800 files) shows 236,905
resolvePackage()calls and 62.9Mstr_starts_with()comparisons, all new in 2.2.3.Fix
Memoize the result per file. Most dependency files belong to no project package (they are PHPStan's own bundled dependencies), so the common
nullresult is cached too, since it otherwise pays a full scan on every call.PackageDependencyResolveris a singleton service, so the cache persists across all files in a worker. Same approach as 200a203 (DependencyResolver: reduce duplicate work).Output is unchanged: the cache is keyed by file, and a file's owning package cannot change during a run.
Result
Cold analysis of the same project, result cache cleared before each run, interleaved across builds and repeated to be robust to machine load:
This recovers about 85% of the regression. A residual ~7% above 2.2.2 remains after this change and is not
resolvePackage(memoizing it does not move it further); it is elsewhere in the 2.2.2..2.2.3 range and I am narrowing it down separately.The full test suite, self-analysis and the coding standard pass. The result-cache e2e scenarios behave identically with and without the change, as expected for a pure memoization.