Blocked BitPacked: decode blocked bitwidths

[mhk197]

Tracking Issue: #10167

Summary

Adds support for decoding BitPacked arrays with blocked bitwidths.

Now each public function that decodes bitpacked arrays with global bitwidths has a version for blocked bitwidths.

Global width	Blocked
unpack_array(array, ctx)	unpack_array_blocked(array, offsets, ctx)
unpack_primitive_array	unpack_primitive_array_blocked
unpack_into_primitive_builder	unpack_into_primitive_builder_blocked
unpack_single(array, index)	unpack_single_blocked(array, offsets, index, ctx)

Callers choose between them by matching on bit_widths().

There is no blocked unpack_map_into_builder: its only caller, the widening CastKernel, declines blocked arrays, so the mapped path would only run in tests.

Design

#10004 only checks block boundary values in debug builds, so the decoder checks them itself:

• Bulk decode casts the offsets child to u64 and executes it once, checks that it holds one boundary per block plus a trailing one, and validates every boundary before writing to the builder. A single offset type keeps one copy of the block loop per value type, for 8 bytes per 1024 values. Each block must be a whole number of 128-byte rows, no wider than the array's type, and the boundaries must span the packed buffer.
• scalar_at reads only the selected block's boundaries, directly for host primitive offsets and through execute_scalar otherwise, and validates just that block.

Both use block_range in bitpacking/array/mod.rs, which validate_primitive_offsets now also uses. Boundaries are relative to the first one, which may be nonzero.

Full blocks unpack straight into the output, like the global decoder's full chunks. A partial first or last block unpacks into one 1024-value scratch block.

[codspeed]

Merging this PR will degrade performance by 11.59%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 2 benchmarks measured no execution time

Nothing ran under measurement, usually because the compiler removed the code under test. These results are not comparable, so they count as unchanged.

Preventing compiler optimizations

❌ 1 regressed benchmark
✅ 2182 untouched benchmarks
🆕 6 new benchmarks
⏩ 409 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌	WallTime	bitpack_blocked_compress_avx2	6.7 µs	7.6 µs	-11.59%
🆕	WallTime	bitpack_blocked_decompress_avx512[u32]	N/A	9.4 µs	N/A
🆕	WallTime	bitpack_blocked_decompress_avx512[u64]	N/A	18.9 µs	N/A
🆕	WallTime	bitpack_blocked_decompress_avx2[u32]	N/A	10.8 µs	N/A
🆕	WallTime	bitpack_blocked_decompress_avx2[u64]	N/A	21.2 µs	N/A
🆕	WallTime	bitpack_blocked_decompress_neon[u32]	N/A	13.8 µs	N/A
🆕	WallTime	bitpack_blocked_decompress_neon[u64]	N/A	29.3 µs	N/A
⚠️	Simulation	bench_compare_primitive[(10000, 2)]	< 1 ns	< 1 ns	N/A
⚠️	Simulation	bench_compare_sliced_dict_primitive[(3333, 10000)]	< 1 ns	< 1 ns	N/A

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing mk/bitpacked-variable-decode (75fe6ef) with develop (c901cf7)}

Footnotes

1. 409 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[joseph-isaacs]

Do we do the same in unblocked?

[mhk197]

yes I believe so

[joseph-isaacs]

you want overlap patches and blocks

[mhk197]

this is what global does as well, can have as a followup pr

[joseph-isaacs]

how is the offset metadata considered here (if you are sliced say 9 values into a block)

[mhk197]

for first block we unpack into scratch buffer but only copy scratch[offset..]