Skip to content

[api-extractor] Reuse TypeScript's module-resolution results to avoid realpath/lstat storm when resolving external package names #5892

Description

@dmichon-msft

Summary

During analysis, DeclarationReferenceGenerator._getPackageName(sourceFile) calls PackageJsonLookup.tryLoadNodePackageJsonFor(sourceFile.fileName) once per external source file. That path (PackageJsonLookup._tryLoadNodePackageJsonInner) calls FileSystem.getRealPath() (Node fs.realpathSync → an lstat per path segment, following symlinks) before the content cache can help, and the upward folder walk in _tryGetPackageFolderFor does it at each directory level. In pnpm workspaces (a node_modules symlink farm) this produces a large lstat storm and shows up as a significant chunk of api-extractor CPU time in profiles.

This was noticed while profiling the type-check optimization in #5891 — after removing the whole-program semantic check, realpathSynclstat inside FileSystem.getRealPathtryLoadNodePackageJsonFor was the next-largest chunk.

Insight

TypeScript already resolved every one of these modules during program construction and recorded the answer. Each ResolvedModuleFull carries:

  • resolvedFileName — the already-realpath'd target (TS paid the lstat cost once, cached), and
  • packageId.name — the bare package name, which is exactly what _getPackageName reconstructs via the filesystem walk (packageId.subModuleName holds the sub-path separately).

Proof of concept

Probing the @rushstack/mcp-server program via the (internal) program.forEachResolvedModule(cb):

resolutions: total=1111, withPackageId=1103, uniqueFiles=258
  @modelcontextprotocol/sdk  <=  @modelcontextprotocol/sdk/dist/esm/server/mcp.d.ts
  zod                        <=  zod/index.d.cts
  ...

99.3% of resolutions carry packageId, and packageId.name came back as the bare package name even for non-index files.

Proposed design

  1. Build a Map<resolvedFileName, packageId.name> once (e.g. off Collector/AstSymbolTable) via the internal program.forEachResolvedModule(...). Type-reference-directive resolutions (forEachResolvedTypeReferenceDirective) also carry packageId and should be included.
  2. In _getPackageName, consult the map first (keyed by sourceFile.fileName) — zero filesystem access for the common case.
  3. Fall back to tryLoadNodePackageJsonFor on a miss (the ~0.7% without packageId, or files reached via other means), preserving today's exact behavior.

forEachResolvedModule / getResolvedModule are TypeScript internals (not in the public .d.ts), but api-extractor already accesses TS internals via TypeScriptInternals.ts, so this is consistent with existing patterns.

Caveats to validate

  • packageId.name semantics — the TS doc comment hints it may include a subpath in some cases; the PoC showed bare names, but this needs verification against @types packages and deep imports vs. the existing packageJson.name behavior.
  • Key matchingresolvedFileName must equal the program's sourceFile.fileName (generally true; preserveSymlinks is the edge case).
  • Correctness — must produce byte-identical API reports across all acceptance-test projects. The fast-path-with-fallback design keeps this low-risk.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status
    Needs triage

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions