feat!: align integrations with IoTDB 2.0.11 and JDK 17 - #132
Open
CritasWang wants to merge 7 commits into
Open
CritasWang wants to merge 7 commits into
CritasWang wants to merge 7 commits into
Conversation
Align client dependencies and connector APIs with the released IoTDB 2.0.11 baseline. Add Spring Boot 4 compatibility and audit starter pool configuration, including the default fetch size of 5000. Add MyBatis query and type adapters, bounded batch generation, nullable composite keys, and IoTDB-specific MyBatis-Plus operations and templates. Update English documentation and the existing Chinese counterparts. Validation: the full JDK 17 reactor, MyBatis offline and live ms suites, and Spring Boot 3/4 compatibility checks on JDK 17/21 passed. Document the upstream us/ns time-filtered DELETE issue without weakening tests. BREAKING CHANGE: Java modules require JDK 17. Regenerate MyBatis mapper interfaces and XML together; TIME/TIMESTAMP fields use Long, and the MyBatis-Plus example exposes only IoTDB-supported mapper operations.
The service container only listened on its loopback because the image defaults dn_rpc_address to 127.0.0.1, so the published port accepted the TCP handshake and the JDBC session was reset. Bind it to all interfaces, as the Testcontainers ITs already do, and replace the TCP probe with a real client round trip through start-cli: the port is reachable several seconds before the DataNode can open a session.
The MyBatis-Plus example had no parent since the Spring Boot starter parent was dropped, so Spotless ran from a hand-copied configuration and Checkstyle, RAT and the reactor's plugin management never touched it. Inherit the examples parent and re-point the inherited Spring, Jackson and Mockito management at the selected Boot line for both the default Boot 3.5.1 build and the spring-boot4 profile; surefire moves to a JUnit Platform 6 capable version for Boot 4. The Spring Boot starter example now uses the reactor's google-java-format 1.28.0 / Spotless 2.44.5 instead of the pins the reactor moved away from.
IoTDB 2.0.11 UPDATE rejects time in its predicate and FIELD columns in SET, so MBG's key-based UPDATE statements can never run. IoTDBKeyPlugin now disables them with a generator warning instead of emitting SQL that fails at runtime, and gains unit coverage for the nullable-TAG predicate rewrite it already performed. The mybatis-support runtime jar is required by every generated mapper but was only reachable through mvn install; the new with-mybatis distribution profile packages it next to the generator plugin.
The IoTDB push source opened its subscription on the worker thread, so a missing topic, a tsfile-format topic, an unreachable broker or a server without subscription support only produced one log line while the task stayed registered and silently never delivered. Validate the topic format and subscribe on the calling thread so start() reports the cause, and let PushSourceTask/PullSourceTask propagate a start failure even when closing the source succeeds, matching SinkTask and ProcessorTask.
The container ITs provisioned a fresh database per test and never dropped it; on an 8 GB Docker host the DataNode ran out of region memory after about a dozen databases and later tests timed out on their first write. Track the databases each test creates and drop them afterwards. apache/iotdb#18300 (grouped MAX of non-positive floats) ships in 2.0.11, so the DAO comment no longer claims it is unreleased; the -MIN(-x) projection stays for older servers. Remaining 2.0.8 references in the README and migration guide now name the verified 2.0.11 behavior.
The starter README notes that connection-timeout-in-ms now defaults to 0 instead of failing startup with a NullPointerException when unset. The compile.version property in the Flink and Hive connector poms had no consumer.
| } | ||
|
|
||
| private void requireRecordFormatTopic() throws Exception { | ||
| try (final SubscriptionTreeSession session = new SubscriptionTreeSession(host, port)) { |
Comment on lines
+212
to
+213
| new XMLMapperBuilder( | ||
| new StringReader(xml), configuration, "mapper.xml", configuration.getSqlFragments()) |
Comment on lines
+118
to
+122
| new XMLMapperBuilder( | ||
| new StringReader(new DefaultXmlFormatter().getFormattedContent(document)), | ||
| configuration, | ||
| "mapper.xml", | ||
| configuration.getSqlFragments()) |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
One or more issues must be addressed before approval.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Aligns Java integrations with IoTDB 2.0.11 and JDK 17, while adding MyBatis runtime/generator support and updating starter, collector, ThingsBoard, and CI integrations.
Changes:
- Centralizes IoTDB 2.0.11 dependencies and JDK 17 requirements.
- Adds MyBatis handlers, interceptor, generator plugins, and examples.
- Updates Spring Boot starter defaults, collector subscription handling, and ThingsBoard integration tests.
File summaries
| File | Description |
|---|---|
| README.md | Updated as part of this pull request. |
| README-zh.md | Updated as part of this pull request. |
| mybatis-support/src/test/java/org/apache/iotdb/mybatis/type/IoTDBTypeHandlerTest.java | Updated as part of this pull request. |
| mybatis-support/src/test/java/org/apache/iotdb/mybatis/IoTDBQueryInterceptorTest.java | Updated as part of this pull request. |
| mybatis-support/src/main/java/org/apache/iotdb/mybatis/type/IoTDBLocalDateTypeHandler.java | Updated as part of this pull request. |
| mybatis-support/src/main/java/org/apache/iotdb/mybatis/type/IoTDBBlobTypeHandler.java | Updated as part of this pull request. |
| mybatis-support/src/main/java/org/apache/iotdb/mybatis/IoTDBQueryInterceptor.java | Updated as part of this pull request. |
| mybatis-support/README.md | Updated as part of this pull request. |
| mybatis-support/pom.xml | Updated as part of this pull request. |
| mybatis-generator/src/test/java/org/apache/iotdb/mybatis/plugin/IoTDBKeyPluginTest.java | Updated as part of this pull request. |
| mybatis-generator/src/main/resources/generatorConfig.xml | Updated as part of this pull request. |
| mybatis-generator/src/main/java/org/apache/iotdb/mybatis/plugin/LombokPlugin.java | Updated as part of this pull request. |
| mybatis-generator/src/main/java/org/apache/iotdb/mybatis/plugin/IoTDBKeyPlugin.java | Updated as part of this pull request. |
| mybatis-generator/src/main/java/org/apache/iotdb/mybatis/plugin/BatchInsertPlugin.java | Updated as part of this pull request. |
| mybatis-generator/README.md | Updated as part of this pull request. |
| mybatis-generator/README-zh.md | Updated as part of this pull request. |
| mybatis-generator/pom.xml | Updated as part of this pull request. |
| Jenkinsfile | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/test/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTtlIT.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/test/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDaoTest.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/test/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDaoIT.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/test/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesAggregationIT.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/test/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableLatestDaoIT.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/test/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableIngestionBenchmarkIT.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/test/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableAttributesDaoIT.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableTimeseriesDao.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/src/main/java/org/apache/iotdb/extras/thingsboard/table/IoTDBTableConfiguration.java | Updated as part of this pull request. |
| iotdb-thingsboard-table/README.md | Updated as part of this pull request. |
| iotdb-thingsboard-table/pom.xml | Updated as part of this pull request. |
| iotdb-thingsboard-table/docs/user-guide.md | Updated as part of this pull request. |
| iotdb-thingsboard-table/docs/migration-guide.md | Updated as part of this pull request. |
| iotdb-thingsboard-table/docs/benchmarks/README.md | Updated as part of this pull request. |
| iotdb-thingsboard-table/docker-compose.test.yml | Updated as part of this pull request. |
| iotdb-thingsboard-table/docker-compose.bench.yml | Updated as part of this pull request. |
| iotdb-thingsboard-table/CI-NOTES.md | Updated as part of this pull request. |
| iotdb-spring-boot-starter/src/main/resources/META-INF/additional-spring-configuration-metadata.json | Updated as part of this pull request. |
| iotdb-spring-boot-starter/src/main/java/org/apache/iotdb/session/IoTDBSessionPool.java | Updated as part of this pull request. |
| iotdb-spring-boot-starter/src/main/java/org/apache/iotdb/config/IoTDBSessionProperties.java | Updated as part of this pull request. |
| iotdb-spring-boot-starter/README.md | Updated as part of this pull request. |
| iotdb-spring-boot-starter/pom.xml | Updated as part of this pull request. |
| iotdb-collector/collector-core/src/main/java/org/apache/iotdb/collector/runtime/task/source/push/PushSourceTask.java | Updated as part of this pull request. |
| iotdb-collector/collector-core/src/main/java/org/apache/iotdb/collector/runtime/task/source/pull/PullSourceTask.java | Updated as part of this pull request. |
| iotdb-collector/collector-core/src/main/java/org/apache/iotdb/collector/plugin/builtin/source/iotdb/IoTDBPushSource.java | Updated as part of this pull request. |
| iotdb-collector/collector-core/src/main/java/org/apache/iotdb/collector/plugin/builtin/sink/event/PipeRawTabletInsertionEvent.java | Updated as part of this pull request. |
| iotdb-collector/collector-core/pom.xml | Updated as part of this pull request. |
| examples/rocketmq/readme.md | Updated as part of this pull request. |
| examples/rabbitmq/readme.md | Updated as part of this pull request. |
| examples/pom.xml | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/test/java/org/apache/iotdb/CodeGeneratorTest.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/test/java/org/apache/iotdb/ApplicationTest.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/resources/templates/iotdb-mapper.xml.vm | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/resources/templates/iotdb-mapper.java.vm | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/resources/templates/iotdb-entity.java.vm | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/resources/mappers/Table2Mapper.xml | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/resources/mappers/Table1Mapper.xml | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/resources/application.yml | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/xml/Table2Mapper.xml | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/xml/Table1Mapper.xml | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/service/Table2Service.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/service/Table1Service.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/service/impl/Table2ServiceImpl.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/service/impl/Table1ServiceImpl.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/mapper/Table2Mapper.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/mapper/Table1Mapper.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/mapper/IoTDBTableMapper.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/Main.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/entity/Table2.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/entity/Table1.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/src/main/java/org/apache/iotdb/config/IoTDBMybatisConfiguration.java | Updated as part of this pull request. |
| examples/mybatisplus-generator/pom.xml | Updated as part of this pull request. |
| examples/mybatis-generator/src/test/java/org/apache/iotdb/mybatis/MapperConfigurationTest.java | Updated as part of this pull request. |
| examples/mybatis-generator/src/main/resources/mybatis-config.xml | Updated as part of this pull request. |
| examples/mybatis-generator/src/main/resources/generatorConfigByExample.xml | Updated as part of this pull request. |
| examples/mybatis-generator/src/main/resources/generatorConfig.xml | Updated as part of this pull request. |
| examples/mybatis-generator/src/main/java/org/apache/iotdb/mybatis/plugin/xml/MixMapper.xml | Updated as part of this pull request. |
| examples/mybatis-generator/src/main/java/org/apache/iotdb/mybatis/plugin/model/Mix.java | Updated as part of this pull request. |
| examples/mybatis-generator/src/main/java/org/apache/iotdb/mybatis/plugin/mapper/MixMapper.java | Updated as part of this pull request. |
| examples/mybatis-generator/src/main/java/org/apache/iotdb/mybatis/Main.java | Updated as part of this pull request. |
| examples/mybatis-generator/README.md | Updated as part of this pull request. |
| examples/mybatis-generator/pom.xml | Updated as part of this pull request. |
| examples/kafka/readme.md | Updated as part of this pull request. |
| examples/iotdb-spring-boot-start/src/main/resources/application.properties | Updated as part of this pull request. |
| examples/iotdb-spring-boot-start/src/main/java/org/apache/iotdb/iotdbspringbootstartexample/service/IoTDBService.java | Updated as part of this pull request. |
| examples/iotdb-spring-boot-start/README.md | Updated as part of this pull request. |
| examples/iotdb-spring-boot-start/pom.xml | Updated as part of this pull request. |
| distributions/src/assembly/mybatis-generator-plugin.xml | Updated as part of this pull request. |
| distributions/pom.xml | Updated as part of this pull request. |
| connectors/spark-iotdb-table-connector/pom.xml | Updated as part of this pull request. |
| connectors/spark-iotdb-connector/pom.xml | Updated as part of this pull request. |
| connectors/hive-connector/src/main/java/org/apache/iotdb/hive/TsFileSerDe.java | Updated as part of this pull request. |
| connectors/hive-connector/pom.xml | Updated as part of this pull request. |
| connectors/grafana-plugin/docker-compose.yaml | Updated as part of this pull request. |
| connectors/grafana-connector/pom.xml | Updated as part of this pull request. |
| connectors/flink-tsfile-connector/src/test/java/org/apache/iotdb/flink/util/TSFileConfigUtilCompletenessTest.java | Updated as part of this pull request. |
| connectors/flink-tsfile-connector/src/main/java/org/apache/iotdb/flink/tsfile/util/TSFileConfigUtil.java | Updated as part of this pull request. |
| connectors/flink-iotdb-connector/pom.xml | Updated as part of this pull request. |
| .github/workflows/compile-check.yml | Updated as part of this pull request. |
Review details
Suppressed comments (2)
iotdb-collector/collector-core/src/main/java/org/apache/iotdb/collector/plugin/builtin/source/iotdb/IoTDBPushSource.java:132
- This does not actually require the topic's
formatto equalTopicConstant.FORMAT_RECORD_HANDLER_VALUE; it only rejects a string containingtsfilehandler. Any other non-record format/legacy representation passes startup, but the worker below rejects its message type asynchronously, so the task can be created and then silently stop instead of failing creation as intended. Parse the serialized topic attributes and require an exact record-format value (while explicitly rejecting all other formats).
mybatis-generator/src/main/resources/generatorConfig.xml:33 - The default generator configuration still omits the
jdbcType.FLOAToverride.IoTDBJavaTypeResolveronly replaces TIMESTAMP and otherwise inherits MBG's default FLOAT mapping, so users running this checked-in config will continue to generateDoublefields instead of the newFloatcontract. Add the same FLOAT override already present in the example configurations.
- Files reviewed: 102/104 changed files
- Comments generated: 2
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| } | ||
| // Like SinkTask/ProcessorTask: a source that cannot start must fail task creation | ||
| // instead of being swallowed when its cleanup succeeds. | ||
| throw e; |
| } | ||
| // Like SinkTask/ProcessorTask: a source that cannot start must fail task creation | ||
| // instead of being swallowed when its cleanup succeeds. | ||
| throw e; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Align the Java integrations with the released IoTDB 2.0.11 baseline (TsFile 2.4.0, Thrift 0.23.0) and move the reactor to JDK 17. Version properties in the root pom follow the
iotdb-parent:2.0.11values;iotdb-jdbc,iotdb-subscriptionandisessionare now managed centrally.Breaking changes
.java-9-and-above/.java-11-below/.java-16profiles are gone.Long(raw server-precision ticks), FLOAT toFloat, DATE toLocalDate, BLOB tobyte[]. Regenerate mapper interfaces and XML together;batchInsertis now a bounded default method over an internalbatchInsertRowsstatement.IoTDBTableMapperexposes only operations the IoTDB table model supports (insert, selectList, selectByKey, deleteByKey, upsertFields, updateAttributes) through a restrictedISqlInjector.fetch-sizedefaults to the client's 5000 (was 1024);connection-timeout-in-msdefaults to 0 instead of failing startup when unset;sql-dialectis deprecated and both pool types are always exposed.Spring Boot starter
IoTDBSessionPoolbecomes an@AutoConfigurationwith@ConditionalOnMissingBeanpool beans that are closed with the context. The compression flags follow the renamed 2.0.11 builder methods (enableThriftRpcCompactionfor the tree pool,enableThriftCompressionfor the table pool). Defaults are documented inadditional-spring-configuration-metadata.jsonand covered by anApplicationContextRunnertest. The same starter is validated against Spring Boot 4.1.1 / Spring 7.0.9 through thespring-boot4profile.MyBatis
New
mybatis-supportmodule with runtime adapters:IoTDBQueryInterceptorroutes prepared queries through a singleexecuteQuery()because the 2.0.11 driver'sexecute()returnstruewithout populatinggetResultSet();IoTDBLocalDateTypeHandlerandIoTDBBlobTypeHandleruse the driver paths that preserve NULL dates and arbitrary bytes. The generator plugin gainsIoTDBKeyPlugin(nullable TAG components becomeIS NULLpredicates; UPDATE statements are dropped with a warning because IoTDB UPDATE rejectstimein the predicate and FIELD columns in SET) and a bounded, validatingBatchInsertPlugin. Awith-mybatisdistribution profile ships the runtime jar next to the generator plugin.Collector
IoTDBPushSourceconsumes record-format subscription messages through the 2.0.11getRecordTabletIterator()API, validates the topic format and subscribes on the calling thread so that a task that cannot consume fails creation instead of staying registered while silently idle.PushSourceTask/PullSourceTaskpropagate start failures likeSinkTaskalready did.ThingsBoard, connectors, CI
The ThingsBoard module follows the reactor versions and its container ITs default to
apache/iotdb:2.0.11-standalone; each IT now drops the databases it created so the shared container does not run out of region memory.TSFileConfigUtilcovers the newsetEncryptSaltsetter. The CI compile-check matrix is JDK 17/21, adds the ThingsBoard and Grafana connector profiles, a Spring Boot 4 job, and amybatis-integrationjob that runs the generator and MyBatis-Plus mappers against a real IoTDB 2.0.11 service container (bound to all interfaces and awaited with a real client session, not a TCP probe).Validation performed locally
mvn clean verify -Pwith-springboot,with-all-connectors,with-examples,with-thingsboard,with-grafana-connector,with-mybatison JDK 17 and JDK 21: all 40 modules pass (272 tests on JDK 21).mybatis-integrationmatrix entries (iotdb-mybatis-itandiotdb-mybatis-it,spring-boot4) against anapache/iotdb:2.0.11-standalonecontainer started like the CI service, ms precision.-Piotdb-table-itTestcontainers suite: 58/58.execute()leavesgetResultSet()null, update counts are -1,TTL=INFunquoted is rejected, grouped MAX of non-positive floats is fixed, SUM over INT64 returns DOUBLE).Known limitations
This PR has:
for an unfamiliar reader.
for code coverage.
Key changed/added classes (or packages if there are too many classes) in this PR
pom.xml(version baseline, JDK 17 enforcer),.github/workflows/compile-check.yml,Jenkinsfileiotdb-spring-boot-starter:IoTDBSessionPool,IoTDBSessionProperties,IoTDBSessionPoolTestmybatis-support(new):IoTDBQueryInterceptor,IoTDBLocalDateTypeHandler,IoTDBBlobTypeHandlermybatis-generator:BatchInsertPlugin,IoTDBKeyPlugin,LombokPluginexamples/mybatis-generator,examples/mybatisplus-generator(CodeGenerator,IoTDBTableMapper,IoTDBMybatisConfiguration, Velocity templates)iotdb-collector:IoTDBPushSource,PipeRawTabletInsertionEvent,PushSourceTask,PullSourceTaskiotdb-thingsboard-table:IoTDBTableConfiguration,IoTDBTableTimeseriesDao, container ITsdistributions:with-mybatisprofile and assembly🤖 Generated with Claude Code