Skip to content

feat: offer inline-script env setup as an unresolved-import quick fix - #29

Closed
StellaHuang95 wants to merge 1 commit into
mainfrom
stellahuang-microsoft-inline-script-code-action
Closed

StellaHuang95 wants to merge 1 commit into
mainfrom
stellahuang-microsoft-inline-script-code-action

Conversation

@StellaHuang95

Copy link
Copy Markdown
Owner

Problem

The PEP 723 inline-script feature has exactly one discovery surface — a CodeLens — and provideCodeLenses returns [] whenever the document is dirty (src/features/inlineScript/codeLens.ts:51).

So the CodeLens is absent at the one moment a user most needs it: right after typing import requests and seeing the red squiggle. Users who don't already know PEP 723 never discover the feature at all.

Change

Register a CodeActionProvider (src/features/inlineScript/setupCodeAction.ts) that offers "Set up this script's Python environment" as a quick fix when:

  1. the inline-scripts feature flag is on,
  2. an unresolved-import diagnostic is present at the invocation range,
  3. the file is a local .py that could carry an inline-script environment,
  4. it isn't already set up (routing.shouldRoute(uri) is false), and
  5. it declares a # /// script block.

Gates are ordered cheapest-first because provideCodeActions can fire on cursor movement — the context.diagnostics scan happens before any parsing, so the common case (no unresolved import) costs nothing but a short array scan.

Metadata is parsed from the in-memory buffer, so unlike the CodeLens this works while the document is dirty.

Honest labeling — wording and mechanics

The title says what the action does, not that the squiggle will clear. It may well not: the unresolved module might be undeclared in the block, or declared under a different distribution name (PIL vs pillow). Correspondingly:

  • action.diagnostics is left unset — populating it tells VS Code the action resolves those diagnostics and opts it into fix-all affordances.
  • action.isPreferred is left unset, so it never pre-empts a genuine import fix such as "add import".

There is deliberately no import-name → package-name mapping. Setup installs the block's declared dependencies verbatim; the import name is never an input.

Diagnostic codes

Matched on code only, never source — Pyrefly-backed Pylance reports its source as the literal string pylance + pyrefly (pylance-internal/src/common/diagnosticCodeMapper.ts:102), so any source allow-list would be wrong somewhere. Diagnostic.code is string | number | {value, target}; the union is normalized and lowercased before comparison.

Checker Codes Source
Pyright / Pylance / basedpyright reportMissingImports, reportMissingModuleSource
Ty unresolved-import, possibly-missing-import diagnosticCodeMapper.ts:164-165
Pyrefly missing-import, missing-source, missing-source-for-stubs diagnosticCodeMapper.ts:380-382
mypy import-not-found, import-untyped ms-python.mypy-type-checker

reportMissingModuleSource is included on purpose

Pylance's own isMissingImportDiagnostic() excludes it (diagnosticCodeMapper.ts:72-78) — and rightly so for their fix, since a stub-resolved module is already spelled correctly and has nothing for a "change spelling" action to suggest. For us the meaning is the opposite kind of useful: a stub was found but the source was not, i.e. the package isn't installed — precisely what setting up the environment fixes. The divergence is commented in the code.

Reappears after a block edit — intentional

Once the script is set up the action disappears. If the user then edits the # /// script block (e.g. to add the dependency that was missing), it comes back by itself: InlineScriptRoutingRegistry.setMetadata resets validatedAssociation to false when the metadata identity changes (src/common/inlineScript/routingRegistry.ts:71-73). This is desired behavior and is covered by a test.

Save before setup — and a race it closes

InlineScriptEnvManager.create resolves the block from disk via readInlineScriptMetadataFromFile (src/managers/builtin/inlineScript/envManager.ts:308). Since the quick fix is offered on a dirty buffer, the handler now saves first.

Saving alone wasn't enough. setUpInlineScriptEnvironment compares the metadata identity before and after create, and silently skips the association if it changed mid-setup. For a block the user had just typed, routing metadata is still undefined (the lazy detector only clears on block-touching edits, src/features/inlineScript/lazyDetector.ts:278-291) — the detector's own save handler would then populate it during create, the identity would go undefined → X, and the association would be silently skipped. The user would wait for an environment and get nothing.

saveScriptBeforeSetup() closes this by re-reading the file it just wrote and seeding routing.setMetadata(). The detector's later write computes the same identity from the same bytes, so the identity stays stable, and an unchanged identity preserves any existing validated association.

Telemetry

New event inlineScript.setupInvoked with a low-cardinality trigger (codelens | codeaction | bulk) and outcome (created | notCreated | error), emitted at the setup entry points. Measuring whether the code action actually improves adoption is the entire premise of this change.

A new event rather than threading trigger through the existing inlineScript.envCreated/envReuseHit/envError events: those fire deep inside InlineScriptEnvManager, and plumbing a trigger down would mean changing the public CreateEnvironmentOptions / EnvironmentManager.create API surface for a telemetry detail. outcome: 'notCreated' intentionally lumps cancelled/skipped/failed together, since the failure taxonomy already ships on envError's category. normalizeSetupTrigger() coerces anything unrecognized to codelens so the property stays low-cardinality even if the command is invoked with junk. The __GDPR__ block in src/common/telemetry/constants.ts is updated.

Tests

22 new tests in setupCodeAction.unit.test.ts plus an 11-test setupInlineScriptEnvironmentHandler suite in setupEnvironment.unit.test.ts. Coverage includes every gating branch, all four diagnostic dialects, the {value, target} object form, case-insensitivity, the pylance + pyrefly source, diagnostics/isPreferred left unset, reappear-after-block-edit, in-memory parse while dirty, and the header byte bound.

Guard tests verified to fail without their guards

Per the convention established in microsoft#1772, each guard was removed one at a time and the covering test confirmed to fail — not merely to pass today. All 11 were detected:

Guard Covering test
feature-flag gate offers nothing when the inline-scripts feature flag is off
unresolved-import diagnostic gate offers nothing when no diagnostic reports an unresolved import
routing-key (local .py) gate offers nothing for a file that cannot carry an inline-script environment
already-set-up (shouldRoute) gate offers nothing once the script is already set up
PEP 723 metadata gate offers nothing when the file has no PEP 723 block
header byte budget bound ignores a metadata block past the header byte budget setup reads
action.diagnostics left unset leaves diagnostics unset so VS Code is not told the action resolves them
save-before-setup saves a dirty document before setup
seed routing metadata after save seeds routing metadata so setup does not misread it as a mid-setup edit
trigger normalization coerces an unknown trigger so telemetry stays low-cardinality
companion-prompt failure isolation does not report a failing companion-extension prompt as a setup failure

That last one caught a real bug introduced in this PR: promptUpdateExtensionsForInlineScripts() sat inside the setup try, so a prompt failure emitted a second telemetry event with outcome: 'error' and showed the user a "setup failed" message even though the environment had been created successfully. It's now outside the try with its own .catch, mirroring what the bulk path already did.

Notes

  • python-envs.setupInlineScriptEnv remains out of package.json by design; it's invoked only by the two UI surfaces.
  • No changes to ms-python.python or Pylance are needed.
  • One nuance worth a reviewer's eye: isInlineScriptsFeatureEnabled() is re-read per call in the provider, whereas registration is latched at activation. Flipping the setting mid-session without a reload makes the code action go inert while the CodeLens keeps showing. That's the safer direction, and the brief's gate ordering requires the per-call check.

Validation

npm run lint ✓ · npm run compile-tests ✓ · npm run unittest ✓ (2267 passing, 6 pending, 0 failing)

Nothing here is user-visible: the whole surface stays behind the undeclared internal flag python-envs.inlineScripts.enabled, which defaults to false.

The PEP 723 inline-script feature's only discovery surface is a CodeLens,
and provideCodeLenses returns [] while document.isDirty (codeLens.ts:51).
That hides it at the one moment a user most needs it: right after typing
`import requests` and seeing the squiggle.

Add a CodeActionProvider that offers "Set up this script's Python
environment" on an unresolved-import diagnostic in a .py file that
declares a `# /// script` block and has no environment yet. It parses the
in-memory buffer, so it works on an unsaved edit.

The action makes no promise it cannot keep: the title says what it does
rather than that the squiggle will clear, and `diagnostics`/`isPreferred`
are both left unset so VS Code is not told the action resolves anything.

Setup reads the block from disk (envManager.ts:308), so the handler now
saves a dirty document first and seeds routing metadata from the saved
bytes, closing a race where a just-typed block would be misread as a
mid-setup edit and have its association silently skipped.

Adds an `inlineScript.setupInvoked` telemetry event with a low-cardinality
`trigger` (codelens | codeaction | bulk) to measure whether the new
surface actually improves adoption.

Everything stays behind `python-envs.inlineScripts.enabled`, so none of
this is user-visible yet.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@StellaHuang95

Copy link
Copy Markdown
Owner Author

Closing for now - not ready to send.

@StellaHuang95
StellaHuang95 deleted the stellahuang-microsoft-inline-script-code-action branch September 15, 2026 00:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant