Carry the writer's permitted serialized IDs into compression schemes

[mhk197]

Summary

The compressor tells schemes which serialized IDs the writer may emit. That is the whole change.

DecimalByteParts is getting a second wire format. The decimal scheme should split wide values into lower parts only when the writer may emit vortex.decimal_byte_parts.v2, and otherwise keep writing single-part arrays under the frozen format. Today a scheme has no way to know what the writer permits. This PR gives it one, and nothing more.

Design

• AllowedSerializedIds is All or Only(HashSet<ArrayId>). The file writer derives it from its enabled editions.
• BtrBlocksCompressorBuilder::retain_allowed_encodings keeps its name, argument, and filter. It still drops schemes whose produced_encodings are not all permitted. It additionally stores the set, intersecting repeated calls, and hands it to the compressor at build.
• CascadingCompressor::with_allowed_serialized_ids only carries the set. It seeds every root CompressorContext with it, and derived contexts inherit it. Filtering stays the builder's job.
• CompressorContext::allows_serialized_id answers for one ID.

One rule for scheme authors, now in the produced_encodings doc: declare the format you always write, and ask the context before writing any other. Check in both the estimate and compress, so one writer configuration always produces one output. If a scheme forgets, the writer's array context rejects the ID at serialization and the write fails loudly.

What this is not

No version selection in the compressor. No predecessor chains (#9770), no replaces relation (#9919), no wrapper for versioned schemes. Each of those chose a scheme version at configuration time and needed a relation between versions to do it. The known cases, wide decimals and per-chunk bitpacking, are one scheme with one branch, so the relation is unnecessary. If a scheme version with a genuinely separate implementation ever appears, a fallback list can be added on top of this without changing it.

Behavior

No scheme on develop reads the context yet, so every existing compressor selects the same schemes and produces the same output. Golden snapshots are unchanged. RLE and OnPair, which build Delta arrays directly when the delta scheme is present, stay correct because a scheme in the compressor still has all its declared IDs permitted.

Follow-ups

• Compress wide decimals into lower parts when the writer permits the v2 format #9759 becomes one if in DecimalScheme: declare vortex.decimal_byte_parts, and split wide values when the context permits the v2 ID. The CUDA preset denies the v2 ID so the GPU never sees lower parts.
• Per-chunk bitpacking in feat(btrblocks): pick per-chunk bit widths when fastlanes.bitpacked_v2 is allowed #9754 takes the same shape.

Tests

• Compressor: unrestricted contexts permit everything, the set reaches descendant contexts, and repeated restrictions intersect.
• Btrblocks: a fake two-format delta scheme built through the builder writes Delta when only the frozen format is permitted and takes its other branch when the newer format is. The declared format is required.
• The builder filters at build, reports All when unrestricted, and intersects repeated calls.
• The writer enables sparse and FSST without registering them and still round-trips the file.

[codspeed]

Merging this PR will regress 1 benchmark

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 3 improved benchmarks
❌ 1 regressed benchmark
✅ 2201 untouched benchmarks
🆕 48 new benchmarks
⏩ 218 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	arrow_checked_add_u32_neon[16384]	12.9 µs	20.4 µs	-36.63%
⚡	Simulation	decode_primitives[f32, (1000, 512)]	64.7 µs	42.6 µs	+52.14%
⚡	WallTime	filtered_sink_i64_avx2[OneNullInEight]	26.2 µs	21.7 µs	+20.58%
⚡	Simulation	allocate_drop_bytes[0]	635.5 ns	527.2 ns	+20.55%
🆕	WallTime	dbp_assemble_kernel_narrow_msp_neon[(I128, 1024)]	N/A	544 ns	N/A
🆕	WallTime	dbp_assemble_kernel_narrow_msp_neon[(I128, 8192)]	N/A	3.8 µs	N/A
🆕	WallTime	dbp_assemble_kernel_narrow_msp_neon[(I256, 1024)]	N/A	914 ns	N/A
🆕	WallTime	dbp_assemble_kernel_narrow_msp_neon[(I256, 8192)]	N/A	6.8 µs	N/A
🆕	WallTime	dbp_assemble_kernel_neon[(I128, 1024)]	N/A	422 ns	N/A
🆕	WallTime	dbp_assemble_kernel_neon[(I128, 8192)]	N/A	3.5 µs	N/A
🆕	WallTime	dbp_assemble_kernel_neon[(I256, 1024)]	N/A	913 ns	N/A
🆕	WallTime	dbp_assemble_kernel_neon[(I256, 8192)]	N/A	6.9 µs	N/A
🆕	WallTime	dbp_split_kernel_all_valid_neon[(I128, 1024)]	N/A	513 ns	N/A
🆕	WallTime	dbp_split_kernel_all_valid_neon[(I128, 8192)]	N/A	3.3 µs	N/A
🆕	WallTime	dbp_split_kernel_all_valid_neon[(I256, 1024)]	N/A	1.3 µs	N/A
🆕	WallTime	dbp_split_kernel_all_valid_neon[(I256, 8192)]	N/A	36.5 µs	N/A
🆕	WallTime	dbp_split_kernel_mixed_null_neon[(I128, 1024)]	N/A	1.2 µs	N/A
🆕	WallTime	dbp_split_kernel_mixed_null_neon[(I128, 8192)]	N/A	8 µs	N/A
🆕	WallTime	dbp_split_kernel_mixed_null_neon[(I256, 1024)]	N/A	1.9 µs	N/A
🆕	WallTime	dbp_split_kernel_mixed_null_neon[(I256, 8192)]	N/A	12.2 µs	N/A
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/edition-aware-schemes (d3526b5) with develop (75b6225)²}

Footnotes

1. 218 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

2. No successful run was found on develop (7eca46d) during the generation of this report, so 75b6225 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩