fix(knowledge): accept relative internal file URLs in document processor

[TheodoreSpeaks]

Summary

• KB document processing sporadically failed with Unsupported fileUrl scheme: only data: URIs and http(s):// URLs are allowed on knowledge-process-document runs.
• Root cause: some ingestion paths store an app-relative internal URL (/api/files/serve/...?context=workspace). The download layer (downloadFileFromUrl) already resolves these directly against storage, but parseWithFileParser and downloadFileForBase64 applied a stricter scheme guard that threw before the URL ever reached the downloader.
• Fix at the consumer: both guards now accept internal URLs via isInternalFileUrl(...) and route them through the existing downloader, matching how the OCR path already behaves. Still fails fast on genuinely unsupported schemes.
• Consumer-side fix covers every producer and lets already-failed documents recover on retry with no data migration. Security is unchanged — the downloader still authorizes the resolved storage key against the billed actor and derives context from the trusted key prefix.

Type of Change

• Bug fix

Testing

Tested manually. Ran bun run lint (no changes) and bun run check:api-validation:strict (passed). Typecheck clean.

Checklist

• Code follows project style guidelines
• Self-reviewed my changes
• Tests added/updated and passing
• No new warnings introduced
• I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment

Project	Deployment	Actions	Updated (UTC)
docs	Skipped		Jul 8, 2026 12:43am

[cursor]

PR Summary

Low Risk
Narrow guard change reusing existing internal URL handling and authorization in the downloader; no new schemes or security model changes.

Overview
Fixes sporadic knowledge document processing failures when ingestion stores an app-relative internal path (/api/files/serve/...) instead of a full http(s) URL.

downloadFileForBase64 and parseWithFileParser now treat internal serve paths like HTTP URLs by checking isInternalFileUrl and delegating to the existing downloader/parser path. Unsupported schemes still fail fast; error text now mentions internal paths. Behavior matches the OCR flow and downloadFileFromUrl—no new authorization surface.

^{Reviewed by Cursor Bugbot for commit cb44c01. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

This PR fixes a sporadic failure in the knowledge-base document processor where relative internal file URLs (/api/files/serve/...) triggered an "Unsupported fileUrl scheme" error in parseWithFileParser and downloadFileForBase64. Both guards now check isInternalFileUrl(fileUrl) alongside the existing http(s):// regex, routing the URL through the existing downloadFileFromUrl downloader that already handles access authorization.

• The two affected scheme guards in downloadFileForBase64 and parseWithFileParser are updated to accept internal relative paths, matching how the OCR path (handleFileForOCR) already behaved.
• Authorization semantics are unchanged: downloadFileFromUrl still requires a userId for internal URLs (fails closed without one), infers context from the trusted storage-key prefix rather than the ?context= query parameter, and calls verifyFileAccess before reading any bytes.

Confidence Score: 5/5

Safe to merge — the change is a minimal, targeted fix that adds a well-tested predicate to two scheme guards, with no changes to authorization logic.

The two modified code paths now delegate internal URLs to the same downloadFileFromUrl function that was already proven correct in the OCR path. That function enforces access authorization (requires userId, derives context from the trusted storage-key prefix, calls verifyFileAccess) before reading any bytes, and fails closed when userId is absent. No new trust surface is introduced.

No files require special attention.

Important Files Changed

Filename	Overview
apps/sim/lib/knowledge/documents/document-processor.ts	Two scheme guards updated to accept internal /api/files/serve/ relative paths alongside http(s):// URLs; routing delegates to the existing downloadFileWithTimeout→downloadFileFromUrl chain that already handles authorization correctly.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[fileUrl received] --> B{data: URI?}
    B -- yes --> C[parseDataURI / decode base64]
    B -- no --> D{https?:// URL?}
    D -- yes --> E{isInternalFileUrl?}
    E -- no --> F[External: DNS/SSRF validate]
    E -- yes --> G[Internal path NEW branch]
    D -- no --> G2{isInternalFileUrl? NEW CHECK}
    G2 -- yes --> G
    G2 -- no --> H[throw: Unsupported scheme]
    G --> I[downloadFileFromUrl]
    I --> J{userId provided?}
    J -- no --> K[throw: Access denied]
    J -- yes --> L[extractStorageKey]
    L --> M[inferContextFromKey trusted prefix]
    M --> N[verifyFileAccess]
    N -- denied --> O[throw: Access denied]
    N -- granted --> P[downloadFile from storage]

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    A[fileUrl received] --> B{data: URI?}
    B -- yes --> C[parseDataURI / decode base64]
    B -- no --> D{https?:// URL?}
    D -- yes --> E{isInternalFileUrl?}
    E -- no --> F[External: DNS/SSRF validate]
    E -- yes --> G[Internal path NEW branch]
    D -- no --> G2{isInternalFileUrl? NEW CHECK}
    G2 -- yes --> G
    G2 -- no --> H[throw: Unsupported scheme]
    G --> I[downloadFileFromUrl]
    I --> J{userId provided?}
    J -- no --> K[throw: Access denied]
    J -- yes --> L[extractStorageKey]
    L --> M[inferContextFromKey trusted prefix]
    M --> N[verifyFileAccess]
    N -- denied --> O[throw: Access denied]
    N -- granted --> P[downloadFile from storage]

Loading

_{Reviews (2): Last reviewed commit: "fix(knowledge): accept relative internal..." | Re-trigger Greptile}

[TheodoreSpeaks]

@greptile review