Summary
During analysis, DeclarationReferenceGenerator._getPackageName(sourceFile) calls PackageJsonLookup.tryLoadNodePackageJsonFor(sourceFile.fileName) once per external source file. That path (PackageJsonLookup._tryLoadNodePackageJsonInner) calls FileSystem.getRealPath() (Node fs.realpathSync → an lstat per path segment, following symlinks) before the content cache can help, and the upward folder walk in _tryGetPackageFolderFor does it at each directory level. In pnpm workspaces (a node_modules symlink farm) this produces a large lstat storm and shows up as a significant chunk of api-extractor CPU time in profiles.
This was noticed while profiling the type-check optimization in #5891 — after removing the whole-program semantic check, realpathSync→lstat inside FileSystem.getRealPath → tryLoadNodePackageJsonFor was the next-largest chunk.
Insight
TypeScript already resolved every one of these modules during program construction and recorded the answer. Each ResolvedModuleFull carries:
resolvedFileName — the already-realpath'd target (TS paid the lstat cost once, cached), and
packageId.name — the bare package name, which is exactly what _getPackageName reconstructs via the filesystem walk (packageId.subModuleName holds the sub-path separately).
Proof of concept
Probing the @rushstack/mcp-server program via the (internal) program.forEachResolvedModule(cb):
resolutions: total=1111, withPackageId=1103, uniqueFiles=258
@modelcontextprotocol/sdk <= @modelcontextprotocol/sdk/dist/esm/server/mcp.d.ts
zod <= zod/index.d.cts
...
99.3% of resolutions carry packageId, and packageId.name came back as the bare package name even for non-index files.
Proposed design
- Build a
Map<resolvedFileName, packageId.name> once (e.g. off Collector/AstSymbolTable) via the internal program.forEachResolvedModule(...). Type-reference-directive resolutions (forEachResolvedTypeReferenceDirective) also carry packageId and should be included.
- In
_getPackageName, consult the map first (keyed by sourceFile.fileName) — zero filesystem access for the common case.
- Fall back to
tryLoadNodePackageJsonFor on a miss (the ~0.7% without packageId, or files reached via other means), preserving today's exact behavior.
forEachResolvedModule / getResolvedModule are TypeScript internals (not in the public .d.ts), but api-extractor already accesses TS internals via TypeScriptInternals.ts, so this is consistent with existing patterns.
Caveats to validate
packageId.name semantics — the TS doc comment hints it may include a subpath in some cases; the PoC showed bare names, but this needs verification against @types packages and deep imports vs. the existing packageJson.name behavior.
- Key matching —
resolvedFileName must equal the program's sourceFile.fileName (generally true; preserveSymlinks is the edge case).
- Correctness — must produce byte-identical API reports across all acceptance-test projects. The fast-path-with-fallback design keeps this low-risk.
Related
Summary
During analysis,
DeclarationReferenceGenerator._getPackageName(sourceFile)callsPackageJsonLookup.tryLoadNodePackageJsonFor(sourceFile.fileName)once per external source file. That path (PackageJsonLookup._tryLoadNodePackageJsonInner) callsFileSystem.getRealPath()(Nodefs.realpathSync→ anlstatper path segment, following symlinks) before the content cache can help, and the upward folder walk in_tryGetPackageFolderFordoes it at each directory level. In pnpm workspaces (anode_modulessymlink farm) this produces a largelstatstorm and shows up as a significant chunk of api-extractor CPU time in profiles.This was noticed while profiling the type-check optimization in #5891 — after removing the whole-program semantic check,
realpathSync→lstatinsideFileSystem.getRealPath→tryLoadNodePackageJsonForwas the next-largest chunk.Insight
TypeScript already resolved every one of these modules during program construction and recorded the answer. Each
ResolvedModuleFullcarries:resolvedFileName— the already-realpath'd target (TS paid thelstatcost once, cached), andpackageId.name— the bare package name, which is exactly what_getPackageNamereconstructs via the filesystem walk (packageId.subModuleNameholds the sub-path separately).Proof of concept
Probing the
@rushstack/mcp-serverprogram via the (internal)program.forEachResolvedModule(cb):99.3% of resolutions carry
packageId, andpackageId.namecame back as the bare package name even for non-index files.Proposed design
Map<resolvedFileName, packageId.name>once (e.g. offCollector/AstSymbolTable) via the internalprogram.forEachResolvedModule(...). Type-reference-directive resolutions (forEachResolvedTypeReferenceDirective) also carrypackageIdand should be included._getPackageName, consult the map first (keyed bysourceFile.fileName) — zero filesystem access for the common case.tryLoadNodePackageJsonForon a miss (the ~0.7% withoutpackageId, or files reached via other means), preserving today's exact behavior.forEachResolvedModule/getResolvedModuleare TypeScript internals (not in the public.d.ts), but api-extractor already accesses TS internals viaTypeScriptInternals.ts, so this is consistent with existing patterns.Caveats to validate
packageId.namesemantics — the TS doc comment hints it may include a subpath in some cases; the PoC showed bare names, but this needs verification against@typespackages and deep imports vs. the existingpackageJson.namebehavior.resolvedFileNamemust equal the program'ssourceFile.fileName(generally true;preserveSymlinksis the edge case).Related
SourceMapper.getRealPathusage for the same treatment.