Skip to content

Test#692

Merged
sweetmantech merged 19 commits into
mainfrom
test
Jun 20, 2026
Merged

Test#692
sweetmantech merged 19 commits into
mainfrom
test

Conversation

@sweetmantech

@sweetmantech sweetmantech commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

Summary by cubic

Adds POST /api/connectors/files to stage an image (by public URL) into Composio storage and return a { name, mimetype, s3key } descriptor for use in file_uploadable connector actions (e.g., LinkedIn/X post images).

  • New Features
    • POST /api/connectors/files: accepts { url, toolSlug }, uploads via Composio, and returns { success, name, mimetype, s3key }.
    • Auth required via x-api-key or Authorization: Bearer; no account_id is used.
    • Derives toolkitSlug from the action toolSlug to scope the upload correctly.
    • Status codes: 200 on success; 400 for invalid/malformed body; 401 for unauthenticated; 502 when the upstream fetch/store fails.

Written for commit cdb34d0. Summary will update on new commits.

Review in cubic

Summary by CodeRabbit

  • New Features
    • Added API endpoint for uploading connector files via URL
    • Supports CORS requests and requires authentication
    • Performs comprehensive request validation on upload parameters
    • Returns uploaded file metadata and detailed error responses

sweetmantech and others added 19 commits June 16, 2026 11:40
…ckfill drain (#671)

Two chat#1796 refinements on the historical (Songstats) path:

1. Free-tier card-on-file link. The gate was issuing the paid subscription
   checkout ($99/mo after a 30-day trial). New createCardOnFileSession uses
   Stripe Checkout `mode: "setup"` — collects a card for $0, no subscription,
   no Stripe product. The account then pays only for metered usage via credits.

2. Instant drain. After enqueuing a historical job, fire-and-forget
   start(songstatsBackfillWorkflow) so the backfill begins immediately instead
   of waiting up to 24h for the cron. Safe by reuse: the workflow's budget gate
   (limit − reserve − rolling-30d ledger) caps it to the Songstats quota and
   SKIP LOCKED prevents double-claiming with the daily cron, which stays as the
   backstop. Only kicks when something was actually enqueued.

26 new/updated unit tests; research+stripe+workflows suite 453 green; tsc/lint/format clean.
…t#1797) (#673)

Pacing/backoff + per-step logging for the Songstats backfill drain (chat#1797 bullets 1 & 3). Bounded exponential backoff (fetchSongstatsWithBackoff, 502/503/504/408/429), defer-to-pending past the bound with claimed-batch release, per-step + per-batch logging.
…97) (#674)

Bullet 2 of chat#1797 (code half). Songstats is the rate authority — removes getBackfillBudgetStep, the budget gate, and insertSongstatsQuotaLedger/selectSongstatsQuotaSpent. The drain now claims+processes regardless of the ledger (un-stalls the backfill); the songstats_quota_ledger table is dropped in recoupable/database#35 (apply AFTER this deploys).
…t) (#677)

* feat: POST /api/catalogs create + materialize from valuation snapshot

Creates a catalog owned by the authenticated account (account derived
from credentials via validateAuthContext, never the body). With
from.snapshot_id, materializes the catalog from a completed valuation
snapshot: creates the catalogs row, links account_catalogs, adds the
snapshot's measured ISRCs as catalog_songs, and records the catalog on
the snapshot. Re-claiming the same snapshot is idempotent.

TDD: validateCreateCatalogBody (6 tests) + createCatalogHandler (8 tests),
red->green. New supabase wrappers: insertCatalog, selectCatalogById,
insertAccountCatalog, updateSnapshotCatalog.

Implements recoupable/chat#1801 Phase 2. Matches docs contract recoupable/docs#243.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor: re-anchor POST /api/catalogs to merged contract + review fixes

- Flatten request to the merged docs#243 contract: from:{snapshot_id} -> a
  root snapshot field (validator + handler + tests). Error copy follows.
- DRY/SRP: drop the inline success() helper; use the shared successResponse().
- KISS rename: materializeSnapshotCatalog.ts -> createSnapshotCatalog.ts.
- DRY: delete the redundant updateSnapshotCatalog helper; reuse the existing
  updatePlaycountSnapshot(id, fields).

Validator change done red->green. lib/catalog: 24 tests pass; tsc + eslint clean.

Addresses review on PR #677.

* fix: materialize catalog songs from song_measurements, not snapshot.isrcs

Testing the full materialize path surfaced a real bug: a valuation snapshot
is album_ids-scoped, so its own isrcs column is null — createSnapshotCatalog
read snapshot.isrcs and would link an EMPTY catalog. The measured ISRCs live
in song_measurements (snapshot lineage), so source them there.

New selectSnapshotIsrcs(snapshotId) helper (distinct song_measurements.song
for the snapshot). createSnapshotCatalog now uses it.

TDD: new createSnapshotCatalog.test.ts (3 tests) red->green; lib/catalog 27 pass.

Addresses PR #677 verification.

* refactor: reuse selectSongMeasurements (snapshot filter) instead of a new helper

KISS/DRY per review: drop selectSnapshotIsrcs; add an optional snapshot
filter to the existing selectSongMeasurements, and derive distinct ISRCs
in createSnapshotCatalog. lib/catalog + song_measurements: 36 tests pass.

Addresses review on PR #677.

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e hidden) (#681)

* fix: LEFT-join artists in catalog-songs read so materialized tracks surface

selectCatalogSongsWithArtists used song_artists!inner -> accounts!inner, so
valuation-captured tracks (which have songs + song_measurements but no
song_artists yet) were filtered out — a materialized catalog read back as 0
songs (verified live on api#677). Drop the two !inner so artist-less songs
return with artists: []; songs!inner stays (catalog_songs.song FK guarantees it).

Closes the read-path half of the song_artists follow-up in recoupable/chat#1801.
Longer-term (option a): the capture pipeline should also write song_artists.

* Update lib/supabase/catalog_songs/selectCatalogSongsWithArtists.ts
…(chat#1793) (#679)

* feat: add X (Twitter) + LinkedIn to the Composio connector whitelist (chat#1793)

Expand the existing whitelist pattern to two new platforms — no
architecture changes:
- SUPPORTED_TOOLKITS (getConnectors.ts) + ENABLED_TOOLKITS (getComposioTools.ts)
- CONNECTOR_DISPLAY_NAMES: twitter → "X (Twitter)", linkedin → "LinkedIn"
- buildAuthConfigs() reads COMPOSIO_TWITTER_AUTH_CONFIG_ID +
  COMPOSIO_LINKEDIN_AUTH_CONFIG_ID
- document both env vars in .env.example

TDD: new buildAuthConfigs unit + expanded getConnectors / handler /
ENABLED_TOOLKITS assertions, RED before GREEN. Full lib/composio suite
green (157 tests).

Implements the contract from docs#244.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: fix lint/format — relocate ENABLED_TOOLKITS test block, reformat toolkit array

- Move the ENABLED_TOOLKITS describe block below the imports (import/first)
- Prettier-format the expanded toolkits array in getConnectors.test.ts

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…680)

* feat: allow artists to connect X (Twitter); keep LinkedIn label-only (chat#1793)

Add `twitter` to ALLOWED_ARTIST_CONNECTORS — artist-facing social, same
class as tiktok/instagram/youtube. `linkedin` is intentionally left out
(label/owner-only).

TDD: isAllowedArtistConnector.test.ts asserts twitter allowed + linkedin
excluded, RED before GREEN. Full lib/composio suite green (157 tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat: allow artists to connect LinkedIn too (chat#1793)

Reversal of the earlier "LinkedIn label/owner-only" call: per owner
decision 2026-06-18, LinkedIn is now an artist-facing connector like
the others. Add `linkedin` to ALLOWED_ARTIST_CONNECTORS.

TDD: flipped the linkedin assertions (now allowed/included), RED before
GREEN. Full lib/composio suite green (159 tests).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* chore: remove unused ALLOWED_ARTIST_CONNECTORS from api (chat#1793)

The api copy of the artist connector allow-list had no runtime consumer —
only its definition, test, and an (also-unused) barrel re-export. The
connector routes are unopinionated (allow any connector for any account);
the allow-list that actually drives the artist Connectors tab lives in
`chat` (`lib/composio/allowedArtistConnectors.ts`). Removing the dead code.

Supersedes the earlier plan to add twitter/linkedin to this api constant
(decision: owner, 2026-06-18) — the artist allow-list is chat-only.

Deletes isAllowedArtistConnector.ts + its test, and the barrel re-export.
lib/composio suite green (149); no new tsc errors vs test (198 baseline).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… in the catalog (#684)

* fix: enrich captured songs with artists + notes (root cause)

The valuation capture path created songs rows from the Spotify track
lookup but discarded track.artists and never ran the manual flow's
enrichment, so captured songs had no song_artists and no notes -> the
chat catalog view's isCompleteSong filter (artist + notes required, on by
default) hid every valuation track (count shown, list empty).

mapUnmappedAlbumTracks now carries track.artists through and runs the
same enrichment as processSongsInput: linkSongsToArtists (auto-creates
the artist account) + queueRedisSongs (queues note generation).

TDD: new test asserts artists are linked + queued; lib/research/playcounts
+ lib/songs 109 tests pass.

Root-cause follow-up on recoupable/chat#1801.

* style: prettier-format the capture-enrichment test

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#689)

GET /api/tasks scopes every lookup to the caller's own account. A lookup
by `id` alone therefore returns nothing when the caller's key doesn't own
the task, which blocks the background worker (customer-prompt-task) from
loading a customer's scheduled task config with a shared admin key.

When an admin caller queries by `id` with no `account_id` param, drop the
account scope so the single task is returned regardless of owner. Non-admin
id lookups stay scoped to the authenticated account (no cross-account leak).

ValidatedGetTasksQuery.account_id is now optional; selectScheduledActions
already filters by account_id only when present.

TDD: RED (admin id lookup not cross-account, non-admin not scoped) -> GREEN.

Fixes part of recoupable/chat#1810.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…dIn/X posts (#691)

* feat(connectors): add POST /api/connectors/files (stage image for posts)

Connector actions with file_uploadable fields (e.g.
LINKEDIN_CREATE_LINKED_IN_POST.images[], TWITTER_CREATION_OF_A_POST) need a
Composio { name, mimetype, s3key } descriptor whose s3key already lives in
Composio storage. The execute path forwards parameters verbatim and never
stages the file, so any s3key 404s.

Add POST /api/connectors/files: given { url, toolSlug }, stage the image via
composio.files.upload() and return flat { success, name, mimetype, s3key }.
The caller passes that descriptor into parameters.images[] on the existing
POST /api/connectors/actions. No change to the execute path (Option A).

- uploadConnectorFile: calls composio.files.upload({ file: url, toolSlug,
  toolkitSlug }) where toolkitSlug is derived from the action slug.
- validate body (zod { url, toolSlug }) + request (validateAuthContext gate;
  no account_id — upload is scoped by tool/toolkit, not connection).
- handler returns 200 on success, 400 invalid body, 401 unauth, 502 upstream.

URL-only input by decision; generic across file_uploadable toolkits
(linkedin, twitter). TDD RED→GREEN; connectors suite green (129 tests).

Implements recoupable/chat#1809. Docs: recoupable/docs#246.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style: prettier-format connectors file-upload tests

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(connectors): use shared safeParseJson in file-upload validator

Address review (DRY): replace the raw `await request.json()` with the
shared `safeParseJson` helper (lib/networking/safeParseJson), matching the
other validators. Malformed JSON now yields a clean 400 via body validation
instead of throwing into the handler's 502 path.

TDD: added a malformed-JSON test (RED on request.json() throw) → GREEN.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Jun 20, 2026

Copy link
Copy Markdown
Contributor

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
api Ready Ready Preview Jun 20, 2026 10:19pm

Request Review

@sweetmantech sweetmantech merged commit 291dbe7 into main Jun 20, 2026
6 of 7 checks passed
@coderabbitai

coderabbitai Bot commented Jun 20, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: aee3d37c-44f6-4fba-acf0-563b88b26f92

📥 Commits

Reviewing files that changed from the base of the PR and between 2f1847b and cdb34d0.

⛔ Files ignored due to path filters (5)
  • lib/composio/connectors/__tests__/deriveToolkitSlug.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
  • lib/composio/connectors/__tests__/uploadConnectorFile.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
  • lib/composio/connectors/__tests__/uploadConnectorFileHandler.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
  • lib/composio/connectors/__tests__/validateUploadConnectorFileBody.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
  • lib/composio/connectors/__tests__/validateUploadConnectorFileRequest.test.ts is excluded by !**/*.test.*, !**/__tests__/** and included by lib/**
📒 Files selected for processing (6)
  • app/api/connectors/files/route.ts
  • lib/composio/connectors/deriveToolkitSlug.ts
  • lib/composio/connectors/uploadConnectorFile.ts
  • lib/composio/connectors/uploadConnectorFileHandler.ts
  • lib/composio/connectors/validateUploadConnectorFileBody.ts
  • lib/composio/connectors/validateUploadConnectorFileRequest.ts

📝 Walkthrough

Walkthrough

A new POST /api/connectors/files Next.js route is added, along with its full supporting library stack. The endpoint authenticates the request, validates a url and toolSlug body via Zod, derives a Composio toolkit slug from the tool slug, uploads the file to Composio storage, and returns the resulting file descriptor (name, mimetype, s3key). An OPTIONS handler provides CORS preflight support.

Changes

Connector File Upload Endpoint

Layer / File(s) Summary
Request body schema and validation
lib/composio/connectors/validateUploadConnectorFileBody.ts, lib/composio/connectors/validateUploadConnectorFileRequest.ts
Zod schema enforces url (valid URL) and toolSlug (non-empty); validateUploadConnectorFileBody returns a 400 NextResponse with CORS headers on failure or typed data on success. validateUploadConnectorFileRequest chains auth via validateAuthContext, JSON parsing via safeParseJson, and body validation, returning early NextResponse errors or UploadConnectorFileParams.
Toolkit slug derivation and Composio file upload
lib/composio/connectors/deriveToolkitSlug.ts, lib/composio/connectors/uploadConnectorFile.ts
deriveToolkitSlug lowercases the first underscore-delimited segment of the action slug. uploadConnectorFile obtains a Composio client, calls composio.files.upload, and returns the UploadedConnectorFile descriptor (name, mimetype, s3key).
Handler orchestration and route entry point
lib/composio/connectors/uploadConnectorFileHandler.ts, app/api/connectors/files/route.ts
uploadConnectorFileHandler applies CORS headers, validates the request, calls uploadConnectorFile, returns a 200 success JSON payload, and catches unexpected errors returning a 502 with CORS headers. The route module exports OPTIONS for preflight and POST delegating to the handler.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

Poem

📁 A URL and a slug walk into a route,
The validator checks them — no room for doubt.
The toolkit slug splits on the underscore line,
Composio stores it, returns fields just fine.
200 OK — the file's in the cloud,
Clean, single-responsibility, standing proud. ✨

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch test

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

sweetmantech added a commit that referenced this pull request Jun 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant