[pull] master from cube-js:master#624
Merged
Merged
Conversation
…lapse trivial pass-through subqueries; e2e specs for cross-cube rolling measures over rollups (#11346) * test(schema-compiler): e2e spec for cross-cube rolling measures with a shared calc-group switch Adds an integration spec covering rollup pre-aggregation matching for multi-stage case entrypoint measures dispatched by a `type: switch` dimension (calc group) across two joined fact cubes combined in a view: - when each cube declares its own switch dimension, a query combining rolling measures from both cubes pins only one switch with a filter, the other stays unresolved, and no pre-aggregation can match (anti-pattern case, asserted); - when the switch lives on a shared single-row calc-group cube joined `1 = 1` into both fact cubes, a single rolling_window filter resolves every case measure and the query is served from both rollups, verified end to end against Postgres; - rolling window and time-shift arithmetic is pinned with deterministic value assertions on a rollup-free copy of the model. Two currently unsupported paths are documented as skipped tests: querying with a granular time dimension fails with "Alias not found for partition_by dimension" (the calc-group dimension is not projected into the multi-stage window input CTE), and a date-range-only time dimension stops matching the rollups even with allow_non_strict_date_range_match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * test(cubejs-testing): birdbox smoke test for shared calc-group rollups in Cube Store Adds a birdbox e2e smoke test running the shared calc-group model against a real Cube server with rollups stored and queried in Cube Store (CUBEJS_ROLLUP_ONLY, Tesseract planner), covering single-cube, cross-cube, and full multi-stage rolling-window queries dispatched by a shared switch dimension. Unlike the schema-compiler integration spec (Postgres-backed rollups), this exercises the Cube Store execution path used in production. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * test(cubejs-testing): deepen smoke query with growth percentage layer Adds a growth-percentage entrypoint (an extra multi-stage layer over the same rolling leaves) and a four-measure query matching the production query shape, to exercise the deep FullKeyAggregate plan that can overflow Cube Store's serialized-plan decode recursion limit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * test(cubejs-testing): skip deep multi-stage Cube Store query, document decode limit The four-measure query reproduces the Cube Store failure: the deep FullKeyAggregate plan overflows the serialized-plan protobuf decode recursion limit ("Error decoding expr as protobuf: ... recursion limit reached"). Skipped with a FIXME until Cube Store can decode plans of this depth; the three-measure query continues to run as a regression guard for the working path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * feat(tesseract): collapse trivial pass-through subqueries in physical plan Multi-stage planning wraps every CTE reference into a derived table of the form (SELECT * FROM cte_n AS cte_n) AS alias. Each wrapper is a semantic no-op but costs two plan nodes (projection + subquery alias) per usage in engines that inline CTE bodies at every reference, such as Cube Store / DataFusion; deep multi-stage queries can overflow Cube Store's serialized-plan decode recursion limit because of them. Adds a physical-plan optimizer pass that rewrites (SELECT * FROM <table reference> AS x) AS alias into a direct <table reference> AS alias everywhere in the plan: the top-level select, CTE bodies, join sides, unions and nested subqueries. Only pure pass-through selects are collapsed: SELECT * projection with no filter, grouping, having, ordering, distinct, limit, offset or own CTEs, over a single table reference. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * test(tesseract): cover trivial pass-through subquery collapse Asserts that multi-stage SQL no longer contains (SELECT * FROM <table>) wrappers, that collapsed CTE references keep their outer alias, and that executed results are unchanged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * test(cubejs-testing): unskip deep multi-stage Cube Store query With the trivial-subquery collapse optimizer in the Tesseract physical plan builder, the four-measure query's plan fits under Cube Store's serialized-plan decode recursion limit and executes from the rollups. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * fix(tesseract): rustfmt optimizer, adapt Oracle alias assertions to collapsed SQL - cargo fmt on trivial_subquery.rs (CI fmt/clippy + lint failures) - OracleQuery alias assertions now accept a bare CTE reference before the alias, since the trivial-subquery collapse renders join sides as `cte_0 "q_0"` instead of `(SELECT ...) "q_0"`; the no-AS-keyword invariant they guard is unchanged - symmetric rollup assertions in the per-cube switch anti-pattern spec Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * chore: retrigger CI after Docker Hub registry and runner network incidents Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * feat(tesseract): match pre-aggregations without stored calc-group dimensions Calc-group (switch) dimensions are virtual enumerations: they carry no stored data, so a rollup gains nothing from materializing them — the build SQL just CROSS JOINs the value list, multiplying rollup size by the number of values while adding zero information. Until now they still had to be listed in every rollup definition purely because the pre-aggregation matcher treated them like ordinary dimensions. - dimension_matcher: a calc-group dimension now always matches (MatchState::Full) regardless of the rollup's stored dimensions. - query processor: calc-group resolution (filter-pinned value rendered as a literal, otherwise a virtual values cross-join) now also applies when the query source is a pre-aggregation, covering exactly the calc groups NOT stored in the rollup. Stored ones keep resolving to the rollup column, so existing rollups behave as before. Literal render references are re-added after the pre-aggregation render-reference reset so they survive it. - specs: new slim-rollup model + test in the shared-calc-group Postgres suite; the per-cube-switch test now asserts rollups match (calc groups no longer block matching); birdbox fixtures drop the switch dimension from both rollups and all Cube Store smoke tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY * test: pin rollup sets and demonstrate per-cube switch divergence Address review feedback: the anti-pattern block asserted only rollup selection after calc groups stopped blocking matching, so the semantic bug it warns about was no longer demonstrated. A new deterministic test on a no-pre-aggs variant of the per-cube switch model shows the unresolved share_metrics switch emitting one row per enumeration value (a duplicate P1 row with a null share change from the YTD branch) instead of the single 0.3 row the shared-switch model returns. The Cube Store smoke tests now assert the exact set of rollup tables used instead of some() containment: usedPreAggregations keys are per-usage (__usage_N suffix), so they are deduped to distinct rollup tables and compared with toEqual, catching any unexpected extra pre-aggregation being served. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QxbxLWpBwunw3FypC8zfnY --------- Co-authored-by: Claude <noreply@anthropic.com>
…-only data-model writes (#11351)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )