Skip to content

feat(clickhouse): add cloud_mode connection option - #6037

Open
cookieark7 wants to merge 1 commit into
SQLMesh:mainfrom
cookieark7:fix/clickhouse-cloud-mode-option
Open

feat(clickhouse): add cloud_mode connection option#6037
cookieark7 wants to merge 1 commit into
SQLMesh:mainfrom
cookieark7:fix/clickhouse-cloud-mode-option

Conversation

@cookieark7

Copy link
Copy Markdown
Contributor

Description

Closes #5785.

Cloud mode could only be turned on by having clickhouse.cloud in the host name. Self-hosted deployments using the replicated database engine have the same constraint — no single-step CREATE TABLE ... AS SELECT — but no way to enable the mode, so the workaround in the issue was adding a CNAME to make the host resolve as clickhouse.cloud.my.domain.net.

This adds a cloud_mode connection option. It is a tri-state: unset (the default) keeps the existing host-name detection, true forces cloud mode on, false forces it off. Existing configurations are unaffected since nothing sets it.

The cloud_mode property had to be renamed to _resolved_cloud_mode — a Pydantic field and a property of the same name silently return the property object rather than the field value, which is truthy, so keeping both would have flipped every existing standalone user into cloud mode.

Test Plan

test_clickhouse_cloud_mode in tests/core/test_connection_config.py. The assertions go through _extra_engine_config and the adapter's engine_run_mode rather than reading the field back, since what matters is that the value reaches the adapter and selects the two-step CTAS.

I checked the tests actually fail against broken implementations rather than just passing against this one. Five mutations, all caught: ignoring the explicit setting, dropping the host-name fallback, removing cloud_mode from the insert_quorum gate, breaking the config-to-adapter wiring, and inverting the value. An earlier version of the tests missed the third one, because the case I had written also set cluster, which satisfies that gate on its own — added a cloud-mode-without-cluster case to cover it.

Also verified against a local ClickHouse that setting cloud_mode: true on a plain host changes the emitted SQL from a single CREATE ... AS SELECT to CREATE ... EMPTY AS SELECT plus INSERT, and that the rows still land.

Locally: make clickhouse-test 168 passed, make fast-test 2622 passed, make style clean.

I don't have a replicated-database deployment, so the end-to-end benefit for that setup is unverified — @cams-data if you're still on the CNAME workaround, this would be worth a check.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

🤖 Generated with Claude Code

Signed-off-by: cookieark <aishwary.kantode@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Need to Make ClickHouse Cloud-Mode an Option

1 participant