Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 20 additions & 13 deletions backend/src/api/public/v1/akrites-external/openapi.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@ info:
Packages, Advisories and Contacts endpoints are implemented. Blast Radius
submit (2a) and poll (2b) are both implemented, backed by a 4-stage
Temporal pipeline (intel, dependents, reachability, report) for npm
packages; other ecosystems fail fast with ECOSYSTEM_NOT_SUPPORTED. The
7-day result cache is specced separately and not yet built.
packages; other ecosystems fail fast with ECOSYSTEM_NOT_SUPPORTED. Submit
reuses a 'done' analysis for the same advisory/package/ecosystem if it
completed within the last day (configurable, see the `force` field below)
instead of starting a new Temporal workflow.


TODO: scopes below (read:packages, read:stewardships) are the existing
Expand Down Expand Up @@ -57,10 +59,11 @@ tags:
status and, once done, results. Bulk submit (jobs:batch) is capped at
20 jobs per request (10 recommended as the default batch size) — each
entry starts its own workflow — and stays behind the same strict rate
limiter as the single-job route; bulk poll
(jobs:batch/poll) is read-only and capped at 100 like the other batch
endpoints. Same interim scope note as Advisories applies (read:packages,
pending a dedicated read:advisories scope).
limiter as the single-job route, defaulting to 50 requests/hour
(configurable via AKRITES_BLAST_RADIUS_RATE_LIMIT_MAX / _WINDOW_MS);
Comment on lines 59 to +63
bulk poll (jobs:batch/poll) is read-only and capped at 100 like the
other batch endpoints. Same interim scope note as Advisories applies
(read:packages, pending a dedicated read:advisories scope).

components:
securitySchemes:
Expand Down Expand Up @@ -406,7 +409,10 @@ components:
force:
type: boolean
default: false
description: Bypasses the 7-day cache and always triggers a new run. Use sparingly.
description: >
Bypasses the advisory cache (a 'done' analysis for the same
advisory/package/ecosystem completed within the last day, by
default) and always triggers a new run. Use sparingly.

BlastRadiusJobEntry:
type: object
Expand All @@ -433,8 +439,10 @@ components:
type: string
enum: [pending, running, done, failed]
description: >
Pending in the single-job submit response, always returned before the
Temporal workflow runs. In the batch submit response, a job whose
Pending when a job is freshly submitted, returned before the Temporal
workflow runs. Done when the advisory cache is reused instead — see
force above — in which case analysisId is the cached analysis's own
id, already completed. In the batch submit response, a job whose
workflow failed to start comes back as failed instead — the rest of
the batch is unaffected.

Expand Down Expand Up @@ -1067,10 +1075,9 @@ paths:
Starts a Temporal workflow running the 4-stage reachability pipeline
(intel, dependents, reachability, report) for npm; other ecosystems
fail fast with ECOSYSTEM_NOT_SUPPORTED. Poll status/results via
GET /jobs/{analysisId}.


Not yet implemented: the 7-day result cache and force-bypass semantics.
GET /jobs/{analysisId}. Reuses a 'done' analysis for the same
advisory/package/ecosystem completed within the last day (by
default) instead of starting a new workflow, unless force is true.
tags: [Blast Radius]
security:
- M2MBearer:
Expand Down
57 changes: 54 additions & 3 deletions backend/src/api/public/v1/packages/blastRadius.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
import { z } from 'zod'

import * as blastRadiusDal from '@crowd/data-access-layer/src/packages/blastRadius'
import { QueryExecutor } from '@crowd/data-access-layer/src/queryExecutor'

// The reachability pipeline is npm-only for now — every other ecosystem (including
// a missing one) is rejected by the schema below before the Temporal workflow is
// triggered.
Expand All @@ -10,6 +13,16 @@ export const SUPPORTED_BLAST_RADIUS_ECOSYSTEMS = ['npm'] as const
// so it is NOT run through purlFieldSchema/normalizePurl like the other endpoints.
const ADVISORY_ID_PATTERN = /^(GHSA-[0-9a-zA-Z]{4}-[0-9a-zA-Z]{4}-[0-9a-zA-Z]{4}|CVE-\d{4}-\d{4,})$/

// How recent a 'done' analysis for the same (advisoryId, package, ecosystem) has to
// be for submit to reuse it instead of starting a new Temporal workflow — see
// getRecentDoneAnalysis. Configurable via env so it can be tuned without a redeploy;
// defaults to 1 day. force=true on the request always bypasses this cache.
const blastRadiusCacheMaxAgeDaysEnv = Number(process.env.AKRITES_BLAST_RADIUS_CACHE_MAX_AGE_DAYS)
export const BLAST_RADIUS_CACHE_MAX_AGE_DAYS =
Number.isSafeInteger(blastRadiusCacheMaxAgeDaysEnv) && blastRadiusCacheMaxAgeDaysEnv > 0
? blastRadiusCacheMaxAgeDaysEnv
: 1

export const blastRadiusJobRequestSchema = z.object({
advisoryId: z
.string()
Expand All @@ -36,19 +49,57 @@ export interface BlastRadiusJobEntry {
status: BlastRadiusJobStatus
}

// Builds the 2a response body. The pipeline isn't implemented yet, so every freshly
// submitted job comes back pending — see analyzeBlastRadius in packages_worker.
// Builds the 2a response body. status defaults to 'pending' (a freshly submitted
// job — see analyzeBlastRadius in packages_worker) but a cache hit passes the
// cached analysis's own status (always 'done' — see getRecentDoneAnalysis) so the
// caller doesn't need to poll a job that's already finished.
export function toBlastRadiusJobEntry(params: {
analysisId: string
advisoryId: string
package: string | null
ecosystem: BlastRadiusJobEcosystem
status?: BlastRadiusJobStatus
}): BlastRadiusJobEntry {
return {
analysisId: params.analysisId,
advisoryId: params.advisoryId,
package: params.package,
ecosystem: params.ecosystem,
status: 'pending',
status: params.status ?? 'pending',
}
}

// Shared by submitBlastRadiusJob and submitBlastRadiusJobBatch — looks up a
// recent 'done' analysis for the same (advisoryId, package, ecosystem) and, if
// found, builds the job entry for it. Returns null on a cache miss or when
// force=true (which bypasses the cache entirely).
export async function getCachedJobEntry(
qx: QueryExecutor,
params: {
advisoryId: string
package: string | null
ecosystem: BlastRadiusJobEcosystem
force: boolean
},
): Promise<BlastRadiusJobEntry | null> {
if (params.force) {
return null
}

const cached = await blastRadiusDal.getRecentDoneAnalysis(
qx,
{ advisoryOsvId: params.advisoryId, packageName: params.package, ecosystem: params.ecosystem },
BLAST_RADIUS_CACHE_MAX_AGE_DAYS,
)
if (!cached) {
return null
}

return toBlastRadiusJobEntry({
analysisId: cached.id,
advisoryId: params.advisoryId,
package: params.package,
ecosystem: params.ecosystem,
status: 'done',
})
}
11 changes: 2 additions & 9 deletions backend/src/api/public/v1/packages/getBlastRadiusJobBatch.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { Request, Response } from 'express'

import { groupBy } from '@crowd/common'
import * as blastRadiusDal from '@crowd/data-access-layer/src/packages/blastRadius'

import { getPackagesQx } from '@/db/packagesDb'
Expand Down Expand Up @@ -34,15 +35,7 @@ export async function getBlastRadiusJobBatch(req: Request, res: Response): Promi
blastRadiusDal.getDependentsExcludedByRangeCountBatch(qx, doneIds),
])

const verdictsByAnalysisId = new Map<string, typeof verdictRows>()
for (const row of verdictRows) {
const bucket = verdictsByAnalysisId.get(row.analysisId)
if (bucket) {
bucket.push(row)
} else {
verdictsByAnalysisId.set(row.analysisId, [row])
}
}
const verdictsByAnalysisId = groupBy(verdictRows, (row) => row.analysisId)
const excludedByRangeCountByAnalysisId = new Map(
excludedByRangeCounts.map(({ analysisId, count }) => [analysisId, count]),
)
Expand Down
64 changes: 63 additions & 1 deletion backend/src/api/public/v1/packages/submitBlastRadiusJob.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ import { describe, expect, it, vi } from 'vitest'

import { submitBlastRadiusJob } from './submitBlastRadiusJob'

const { start, createAnalysis, failAnalysis } = vi.hoisted(() => ({
const { start, createAnalysis, failAnalysis, getRecentDoneAnalysis } = vi.hoisted(() => ({
start: vi.fn().mockResolvedValue(undefined),
createAnalysis: vi.fn().mockResolvedValue(undefined),
failAnalysis: vi.fn().mockResolvedValue(undefined),
getRecentDoneAnalysis: vi.fn(),
}))

vi.mock('@/db/packagesTemporal', () => ({
Expand All @@ -20,12 +21,15 @@ vi.mock('@/db/packagesDb', () => ({
vi.mock('@crowd/data-access-layer/src/packages/blastRadius', () => ({
createAnalysis,
failAnalysis,
getRecentDoneAnalysis,
}))

function mockReqRes(body: unknown) {
start.mockClear()
createAnalysis.mockClear()
failAnalysis.mockClear()
getRecentDoneAnalysis.mockClear()
getRecentDoneAnalysis.mockResolvedValue(null)

const req = { body } as unknown as Request

Expand Down Expand Up @@ -152,4 +156,62 @@ describe('submitBlastRadiusJob', () => {
})
expect(errorMessage).toBe('temporal unreachable')
})

it('reuses a recent done analysis instead of starting a workflow', async () => {
const { req, res, start, status, json } = mockReqRes({
advisoryId: 'GHSA-jf85-cpcp-j695',
ecosystem: 'npm',
})
getRecentDoneAnalysis.mockResolvedValue({
id: 'cached-analysis-id',
advisory_osv_id: 'GHSA-jf85-cpcp-j695',
package_name: null,
ecosystem: 'npm',
status: 'done',
error: null,
candidates_considered: 5,
started_at: '2026-07-01T00:00:00.000Z',
completed_at: '2026-07-01T01:00:00.000Z',
})

await submitBlastRadiusJob(req, res)

expect(createAnalysis).not.toHaveBeenCalled()
expect(start).not.toHaveBeenCalled()
expect(status).toHaveBeenCalledWith(202)
expect(json).toHaveBeenCalledWith(
expect.objectContaining({
analysisId: 'cached-analysis-id',
advisoryId: 'GHSA-jf85-cpcp-j695',
package: null,
ecosystem: 'npm',
status: 'done',
}),
)
})

it('bypasses the cache and starts a new workflow when force is true, even with a recent done analysis', async () => {
const { req, res, start } = mockReqRes({
advisoryId: 'GHSA-jf85-cpcp-j695',
ecosystem: 'npm',
force: true,
})
getRecentDoneAnalysis.mockResolvedValue({
id: 'cached-analysis-id',
advisory_osv_id: 'GHSA-jf85-cpcp-j695',
package_name: null,
ecosystem: 'npm',
status: 'done',
error: null,
candidates_considered: 5,
started_at: '2026-07-01T00:00:00.000Z',
completed_at: '2026-07-01T01:00:00.000Z',
})

await submitBlastRadiusJob(req, res)

expect(getRecentDoneAnalysis).not.toHaveBeenCalled()
expect(createAnalysis).toHaveBeenCalledTimes(1)
expect(start).toHaveBeenCalledTimes(1)
})
})
25 changes: 22 additions & 3 deletions backend/src/api/public/v1/packages/submitBlastRadiusJob.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,24 +8,43 @@ import { getPackagesQx } from '@/db/packagesDb'
import { getPackagesTemporalClient } from '@/db/packagesTemporal'
import { validateOrThrow } from '@/utils/validation'

import { blastRadiusJobRequestSchema, toBlastRadiusJobEntry } from './blastRadius'
import {
blastRadiusJobRequestSchema,
getCachedJobEntry,
toBlastRadiusJobEntry,
} from './blastRadius'

// 2a — submit a blast-radius analysis job. Always exactly one job per request.
// Every submission gets a fresh analysisId and status pending.
// Every submission gets a fresh analysisId and status pending, unless a 'done'
// analysis for the same (advisoryId, package, ecosystem) is still within the
// advisory cache window — see BLAST_RADIUS_CACHE_MAX_AGE_DAYS — in which case that
// cached analysis is returned instead, with no new row and no workflow start.
// force=true on the request skips this cache entirely.
export async function submitBlastRadiusJob(req: Request, res: Response): Promise<void> {
const body = validateOrThrow(blastRadiusJobRequestSchema, req.body)

const jobPackage = body.package ?? null
const jobEcosystem = body.ecosystem

const qx = await getPackagesQx()
const cached = await getCachedJobEntry(qx, {
advisoryId: body.advisoryId,
package: jobPackage,
ecosystem: jobEcosystem,
force: body.force,
})
if (cached) {
res.status(202).json(cached)
return
}

const analysisId = generateUUIDv4()

// Create the pending row synchronously, before starting the workflow — otherwise a
// client that polls GET /jobs/:analysisId immediately after this 202 can race
// blastRadiusStart's own createAnalysis call and get a 404 for a job that was, in
// fact, accepted. blastRadiusStart's createAnalysis upserts the same row, so this
// is safe to run again from the workflow.
const qx = await getPackagesQx()
await blastRadiusDal.createAnalysis(qx, {
id: analysisId,
advisoryOsvId: body.advisoryId,
Expand Down
Loading
Loading