Skip to content

feat(bigquery): validate integration + add dataset/table lifecycle tools#5480

Merged
waleedlatif1 merged 5 commits into
stagingfrom
validate/bigquery-integration
Jul 7, 2026
Merged

feat(bigquery): validate integration + add dataset/table lifecycle tools#5480
waleedlatif1 merged 5 commits into
stagingfrom
validate/bigquery-integration

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Validated the existing 5 BigQuery tools (query, list_datasets, list_tables, get_table, insert_rows) against the live BigQuery REST v2 API docs — no critical issues found
  • Added 6 new tools, all covered by the existing bigquery OAuth scope (no new scopes requested): create dataset, delete dataset, create table, delete table, list table data, get query results
  • Wired new operations into the block (dropdown, subblocks, conditions, tools.config), tool registry, and barrel exports

Type of Change

  • New feature

Testing

  • bun run lint:check clean
  • bun run type-check clean (bigquery-related)
  • bun run check:api-validation passed

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

…et/table lifecycle + query-result tools

- Cross-checked existing query/list_datasets/list_tables/get_table/insert_rows tools against BigQuery REST v2 docs; no critical issues found
- Added 6 new tools covered by the existing bigquery OAuth scope: create/delete dataset, create/delete table, list table data, get query results
- Wired new operations into the block (subblocks, conditions, tools.config), registry, and barrel exports
@vercel

vercel Bot commented Jul 7, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 7, 2026 6:51pm

Request Review

@cursor

cursor Bot commented Jul 7, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
New delete-dataset/table operations can destroy production data if misused in workflows; otherwise changes follow existing OAuth BigQuery patterns with schema validation on create table.

Overview
Expands the Google BigQuery block and tool layer with six new REST-backed operations: create/delete dataset, create/delete table, list table data, and get query results (job pagination and follow-up to run query). Existing OAuth scopes are unchanged.

The block gains new operation dropdown entries, operation-grouped subblocks (including deleteContents, job timeoutMs, schema/rows as code inputs with wand helpers), and smarter param mapping (newDatasetId / newTableId → API ids). Types, registry, and exports are updated; block meta adds a provision-dataset-and-table skill template.

Reviewed by Cursor Bugbot for commit 6ca7101. Configure here.

Comment thread apps/sim/blocks/blocks/google_bigquery.ts Outdated
Comment thread apps/sim/tools/google_bigquery/list_table_data.ts
@greptile-apps

greptile-apps Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds 6 new BigQuery tools (create/delete dataset, create/delete table, list table data, get query results) and wires them into the block, tool registry, and barrel exports. It also deduplicates the block's condition arrays into shared constants (EXISTING_DATASET_OPS, EXISTING_TABLE_OPS, LOCATION_OPS) and addresses prior review findings around datasetId/tableId trimming.

  • New tools cover the full dataset/table lifecycle using the existing bigquery OAuth scope; all URL-path identifiers are trimmed before encodeURIComponent, and the block params() function maps newDatasetId/newTableId to the correct canonical datasetId/tableId keys.
  • Shared constants cleanly replace the repeated inline arrays in condition/required and params(), making it easier to add operations in the future.
  • list_table_data returns positional row arrays (no column names) because the underlying tabledata.list API does not include schema; the description advises pairing with get_table, which is the correct workaround given the API constraints.

Confidence Score: 5/5

Safe to merge — all six new tools are correctly wired end-to-end and all previously identified trimming bugs have been addressed.

All URL-path identifiers are trimmed before encoding. The block's params() function correctly maps newDatasetId→datasetId and newTableId→tableId without leaking stale selector values. Shared constants cleanly replace duplicated condition arrays. The schema validation in create_table handles malformed JSON with a clear error. No logic gaps or data-loss paths were found across the full chain from block → params → tool → API.

No files require special attention.

Important Files Changed

Filename Overview
apps/sim/blocks/blocks/google_bigquery.ts Adds 6 new operations; deduplicates condition arrays into shared constants; params() correctly routes newDatasetId→datasetId and newTableId→tableId; all conditional logic and tool mapping look correct.
apps/sim/tools/google_bigquery/create_dataset.ts datasetId trimmed in body's datasetReference; URL path does not include datasetId (POST to /datasets); error handling and output shape are correct.
apps/sim/tools/google_bigquery/create_table.ts datasetId and tableId trimmed in both URL path and tableReference body; schema JSON.parse wrapped in try/catch with actionable error; all previously flagged issues are resolved.
apps/sim/tools/google_bigquery/delete_dataset.ts datasetId trimmed in URL; deleteContents mapped to query param; 204 No Content handled correctly (response.json() only called on error paths).
apps/sim/tools/google_bigquery/delete_table.ts Both datasetId and tableId trimmed in URL; error handling consistent with delete_dataset pattern.
apps/sim/tools/google_bigquery/get_query_results.ts Uses correct BigQuery REST v2 queries/{jobId} endpoint; jobId trimmed; optional params validated before appending to URL; rows mapped to named objects using schema fields.
apps/sim/tools/google_bigquery/list_table_data.ts Returns rows as positional arrays (no column names) consistent with tabledata.list API limitations; tool description correctly advises pairing with get_table; URL params properly constructed.
apps/sim/tools/google_bigquery/types.ts New param and response interfaces match tool implementations; no type inconsistencies found.
apps/sim/tools/google_bigquery/index.ts All 6 new tools exported correctly; barrel export is consistent with registry additions.
apps/sim/tools/registry.ts All 6 new tools imported and registered with correct string keys; entries adjacent to existing BigQuery tools.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    Block["GoogleBigQueryBlock\n(operation dropdown)"] --> Switch{"params.operation"}

    Switch -->|query| QT["google_bigquery_query\nPOST /jobs (runQuery)"]
    Switch -->|get_query_results| GQR["google_bigquery_get_query_results\nGET /queries/{jobId}"]
    Switch -->|list_datasets| LD["google_bigquery_list_datasets\nGET /datasets"]
    Switch -->|create_dataset| CD["google_bigquery_create_dataset\nPOST /datasets\ndatasetId.trim() in body"]
    Switch -->|delete_dataset| DD["google_bigquery_delete_dataset\nDELETE /datasets/{id.trim()}\n?deleteContents="]
    Switch -->|list_tables| LT["google_bigquery_list_tables\nGET /datasets/{id}/tables"]
    Switch -->|get_table| GT["google_bigquery_get_table\nGET /datasets/{id}/tables/{id}"]
    Switch -->|create_table| CT["google_bigquery_create_table\nPOST /datasets/{id.trim()}/tables\nSchema JSON validated + parsed"]
    Switch -->|delete_table| DT["google_bigquery_delete_table\nDELETE /datasets/{id}/tables/{id.trim()}"]
    Switch -->|list_table_data| LTD["google_bigquery_list_table_data\nGET /datasets/{id}/tables/{id}/data\nRows: positional arrays"]
    Switch -->|insert_rows| IR["google_bigquery_insert_rows\nPOST /datasets/{id}/tables/{id}/insertAll"]

    Params["params() mapping\nnewDatasetId → datasetId\nnewTableId → tableId\ndatasetLocation → location"] --> Switch
    Constants["Shared constants\nEXISTING_DATASET_OPS\nEXISTING_TABLE_OPS\nLOCATION_OPS"] --> Params
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
flowchart TD
    Block["GoogleBigQueryBlock\n(operation dropdown)"] --> Switch{"params.operation"}

    Switch -->|query| QT["google_bigquery_query\nPOST /jobs (runQuery)"]
    Switch -->|get_query_results| GQR["google_bigquery_get_query_results\nGET /queries/{jobId}"]
    Switch -->|list_datasets| LD["google_bigquery_list_datasets\nGET /datasets"]
    Switch -->|create_dataset| CD["google_bigquery_create_dataset\nPOST /datasets\ndatasetId.trim() in body"]
    Switch -->|delete_dataset| DD["google_bigquery_delete_dataset\nDELETE /datasets/{id.trim()}\n?deleteContents="]
    Switch -->|list_tables| LT["google_bigquery_list_tables\nGET /datasets/{id}/tables"]
    Switch -->|get_table| GT["google_bigquery_get_table\nGET /datasets/{id}/tables/{id}"]
    Switch -->|create_table| CT["google_bigquery_create_table\nPOST /datasets/{id.trim()}/tables\nSchema JSON validated + parsed"]
    Switch -->|delete_table| DT["google_bigquery_delete_table\nDELETE /datasets/{id}/tables/{id.trim()}"]
    Switch -->|list_table_data| LTD["google_bigquery_list_table_data\nGET /datasets/{id}/tables/{id}/data\nRows: positional arrays"]
    Switch -->|insert_rows| IR["google_bigquery_insert_rows\nPOST /datasets/{id}/tables/{id}/insertAll"]

    Params["params() mapping\nnewDatasetId → datasetId\nnewTableId → tableId\ndatasetLocation → location"] --> Switch
    Constants["Shared constants\nEXISTING_DATASET_OPS\nEXISTING_TABLE_OPS\nLOCATION_OPS"] --> Params
Loading

Reviews (5): Last reviewed commit: "fix(bigquery): stop leaking stale datase..." | Re-trigger Greptile

Comment thread apps/sim/tools/google_bigquery/create_table.ts Outdated
Comment thread apps/sim/tools/google_bigquery/create_table.ts
Comment thread apps/sim/tools/google_bigquery/create_dataset.ts
- create_table: guard JSON.parse on the schema field with a clear error, trim tableReference IDs in the request body
- create_dataset: trim datasetId in the request body
- block: stop leaking a stale query "location" value into create_dataset when datasetLocation is empty; clarify pageToken output description covers list_table_data and get_query_results too
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile @cursor please re-review — pushed fixes for all findings from the first round (schema JSON.parse guard, ID trimming in create_table/create_dataset bodies, stale query-location leak into create_dataset, and clarified pageToken output docs).

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/tools/google_bigquery/create_table.ts
Comment thread apps/sim/tools/google_bigquery/get_query_results.ts
- create_table: validate parsed schema is a non-empty array of field objects with a name, not just syntactically valid JSON
- get_query_results: guard timeoutMs with Number.isFinite before appending to the query string, matching maxResults
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile please re-review; bugbot run

…dation

- Add missing projectId block output (returned by get_table/create_table/create_dataset but was silently dropped from the block's output schema)
- Switch query/rows/schema subBlocks from long-input to code, matching the JSON/SQL field convention used by every other DB integration (clickhouse, mongodb, postgresql, supabase)
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile please re-review; bugbot run — pushed a final alignment pass (added missing projectId block output, switched query/rows/schema subblocks from long-input to code to match repo convention for JSON/SQL fields).

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

bugbot run

Comment thread apps/sim/blocks/blocks/google_bigquery.ts Outdated
…et/create_table

Same class of bug as the earlier location leak: params() spread hidden
datasetId/tableId from ...rest unconditionally, so a stale value from a
previously-selected operation could survive into create_dataset/create_table
if the dedicated new-ID field was somehow empty. Gate datasetId/tableId (and
location) to the operations that actually use them, via shared op-list
constants also reused by the corresponding subblock conditions.
@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

@greptile please re-review; bugbot run — fixed a stale datasetId/tableId leak into create_dataset/create_table (same class as the earlier location leak), and deduped the operation-list conditions into shared constants.

@waleedlatif1

Copy link
Copy Markdown
Collaborator Author

bugbot run

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 6ca7101. Configure here.

@waleedlatif1 waleedlatif1 merged commit 8174182 into staging Jul 7, 2026
18 checks passed
@waleedlatif1 waleedlatif1 deleted the validate/bigquery-integration branch July 7, 2026 19:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant