Conversation
ColinLeeo
force-pushed
the
feat/native-package-artifacts
branch
from
September 17, 2026 08:22
54b42b0 to
8341c6b
Compare
The native package version tests hardcoded the fork's 2.5.0 development version, so they failed on any checkout that declares a different cpp/CMakeLists.txt version. Read the version under test instead and keep only the unit cases that intentionally pin an input version.
ColinLeeo
force-pushed
the
feat/native-package-artifacts
branch
from
September 17, 2026 10:52
8341c6b to
72abe30
Compare
- pom.xml: add the with-python-only profile plus the shared check default - python/pom.xml: resolve tsfile.cpp.build, pass it to setup.py via TSFILE_CPP_BUILD, and validate the C++ artifacts at the validate phase - python/check_cpp_build.py: fail fast with an actionable message - wheels.yml: drop the explicit -Denable.lzma2=ON so Python wheels follow the single LZMA2 default owned by cpp/pom.xml
Let the build tree produced by '-Pwith-cpp package' also generate the CPack configuration, so native packages can be produced from that same tree without reconfiguring or rebuilding it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a native packaging pipeline for the C++ implementation and ties it to a single Maven-driven build hierarchy. The workflow builds the C++ SDK once per platform, reuses those SDK artifacts for downstream Go and Python checks, produces the native package families, and assembles one checksummed release bundle.
The pipeline deliberately stops before publishing: it produces verified artifacts plus a machine-readable manifest, and leaves JFrog uploads, mutable
latestpointers, tags, signing, and final-release behavior to a separate change.Background
TsFile already has Python wheel workflows, but there was no single CI path that produced publishable C++ package artifacts and also verified the install/consume contract for Linux, macOS, and Windows. This change makes Maven the only native build entry point and makes artifact flow explicit between build, test, and aggregation jobs.
Related: #914 (earlier native packaging work in this direction).
What this changes
Maven build hierarchy
-Pwith-cpp packageis the C++ build entry point for native packaging.-Pwith-python-only packagebuilds the Python extension and wheel from an existing C++ SDK instead of rebuilding the C++ core.-Dtsfile.version.sync.skip=true, soVersionUpdater.groovydoes not rewritecpp/CMakeLists.txtor Python metadata during package builds. Package identity therefore remains tied to the checkout-derived version generated bynative_package_versions.py.TSFILE_ARCHIVE_VERSION,TSFILE_DEBIAN_PACKAGE_VERSION,TSFILE_RPM_PACKAGE_VERSION, andTSFILE_RPM_PACKAGE_RELEASE.-Denable.cpack=ONexposes CPack generation through Maven without forcing CPack on ordinary developer builds.CMake install and consume contract
cpp/cmake/TsFileConfig.cmake.in) and pkg-config metadata (cpp/cmake/libtsfile.pc.in).cpp/cmake/TsFilePublicHeaders.cmake) mirroring the includes the current C++ implementation uses; narrowing to a stable public API is intentionally deferred.TsFileStaticDependencies.cmake(.in)re-exports bundled codec archives soTsFile::tsfilecarries the full link line.CPack packaging
tsfile,tsfile-dev(tsfile-develon RPM), andtsfile-tools.Native packaging workflow
.github/workflows/native-packages.yml:workflow_dispatchonly,permissions: contents: read, no secrets, and no publishing credentials.preparederives one build identity fromTsFile_CPP_VERSION(which must matchMAJOR.MINOR.PATCH.dev), the UTC build date,run_number,run_attempt, and the short commit SHA./MT, also producing the combined ZIP.go vet,go test, andgo test -raceagainst the Ubuntu SDK.-Pwith-python-onlyagainst the Ubuntu SDK.assemblejob depends on all validation jobs and fails unless every required artifact family is present. It emitstsfile-native-packages-<archive_version>containing:SHA256SUMSandmanifest.jsonwith family, platform, size, SHA-256, target repository/path/properties, source identity, and generated versions.${{ github.repository }}with the immutable workflow commit.Docs
packaging/README.mddocuments the Maven entry points, SDK layout, downstream consumer flow, per-family packaging commands, and the final bundle contract.Version identity
For a checkout declaring
TsFile_CPP_VERSIONof2.3.2.dev:2.3.2.dev0+20260910.123.1.gabcdef12.3.2~dev0+20260910.123.1.gabcdef1-12.3.2/ release0.dev0.20260910.123.1.gabcdef1.el92.3.2-dev0.20260910.123.1.gabcdef12.3.2.dev0.20260910.123.1.gabcdef1Not included
How it was tested
python3 -m unittest discover -s packaging/tests -p 'test_*.py' -vpasses 34 tests (5 installation tests skip unlessTSFILE_RUN_INSTALL_TESTS=1and a dependency cache are provided).ruby packaging/tests/check_native_workflow.rbpasses and checks the manual/read-only workflow contract, Maven entry points, artifact consumers, final aggregation, and shell syntax.-Pwith-cppand-Pwith-python-onlyskip source-version rewrites when-Dtsfile.version.sync.skip=true.-Pwith-cpp generate-test-sources -Denable.cpack=ONconfirms all four version properties reach CMake and that the generatedCPackConfig.cmakeuses the expected archive version.GitHub only exposes
workflow_dispatchfor workflows that already exist on the default branch, so a pre-merge run cannot be dispatched through the workflow definition added by this PR. After this workflow lands ondevelop, run Build native package artifacts and attach the resulting run as post-merge validation.Reviewer notes
packaging/**/*.pyis outside the Pythonspotlessscope, so those files are not currently Black-formatted.cpp/target/buildbecause the current cgo bridge hardcodes that layout. Building against an installedtsfile-devpackage through pkg-config remains a follow-up.