feat(query,viewer): add viewer_external_links read model for /api/links?type=external#157
Merged
YusukeHirao merged 3 commits intoJul 3, 2026
Merged
Conversation
…ink listing listExternalLinks ran a live anchors JOIN + GROUP BY + COUNT(DISTINCT ...) per request, twice (once for the total, once for the page of data). SQLite is known to build a separate b-tree for COUNT(DISTINCT) instead of reusing an index, so this scaled poorly and had no measurement or caching unlike every other heavy read in this package. Add viewer_external_links, populated once per buildViewerReadModel run by computeExternalLinkRows (the aggregation logic lifted verbatim from listExternalLinks so referrer-counting semantics stay in lockstep with getPageDetail.inboundLinks). listViewerExternalLinks serves reads from the pre-aggregated table via plain paginateQuery, with no keyset cursor needed since the REST contract is already offset-based. listExternalLinks itself is untouched and remains the fallback for archives without a current read model. Bump VIEWER_READ_MODEL_SCHEMA_VERSION 4->5 for the new table.
…fast path /api/links?type=external now dispatches to listViewerExternalLinks when the viewer_external_links read model is current, falling back to the legacy listExternalLinks otherwise -- the same two-layer pattern register-pages-route.ts already uses for /api/pages. No unsupported filter forces a legacy fallback here, since urlPattern/status both map directly onto viewer_external_links columns.
…onale Explain why listExternalLinks's live GROUP BY + COUNT(DISTINCT ...) query was moved into the viewer_external_links read model: SQLite is known to build a separate b-tree for COUNT(DISTINCT) instead of reusing an index, and expression-based indexes only help when the indexed expression matches the query verbatim -- neither escape hatch was reliable here, so materialising the aggregate once (the pattern this package already uses for viewer_pages/viewer_directory_nodes) was the safer fix.
This was referenced Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
listExternalLinksran a liveanchorsJOIN +GROUP BY+COUNT(DISTINCT ...)per request, twice (once for the total, once for the page of data) — SQLite is known to build a separate b-tree forCOUNT(DISTINCT)instead of reusing an index, and this query had no measurement or caching unlike every other heavy read in this package.viewer_external_links, populated once perbuildViewerReadModelrun bycomputeExternalLinkRows(aggregation logic lifted verbatim fromlistExternalLinks, soreferrerCountstays in lockstep withgetPageDetail.inboundLinks, fix(crawler): http と https の同一 URL が別ページとして二重登録される #71).listViewerExternalLinksserves reads from the pre-aggregated table via plainpaginateQuery— no keyset cursor needed since the REST contract is already offset-based.register-links-route.tsnow dispatches to the fast path when the read model is current, falling back to the untouchedlistExternalLinksotherwise — the same two-layer patternregister-pages-route.tsalready uses for/api/pages.VIEWER_READ_MODEL_SCHEMA_VERSION4→5; existing archives rebuild automatically on next open.Test plan
yarn build(+ directtscper-package check to rule out sandboxed no-op builds)yarn test(2385 tests, full suite)yarn lint(0 errors)compute-external-link-rows.spec.ts,list-viewer-external-links.spec.ts(mirrorslist-external-links.spec.ts's full case set),build-viewer-read-model.spec.tsadditions,register-links-route.spec.ts(fast-path/legacy-fallback dual fixture)@examplefixed) + doc pass (lint clean)🤖 Generated with Claude Code