Read RowFn UTF-8 byte lengths from native metadata

[connortsui20]

Summary

Depends on #10262 for the shared validated offset storage.

Length predicates currently prepare UTF-8 strings or materialize a u64 byte-length array before comparison. Explicit offset-length and view-length input elements let a RowFn compare native headers directly into Boolean output without acquiring payload buffers or constructing strings.

Changes

The offset element shares the validated offset domain from the preceding draft. The view element owns only initialized string-view headers. Null lengths are unspecified, and the executor propagates input validity. Both elements reject unsupported physical layouts and report fallible decoding, which preserves the existing valid-only policy for partially valid inputs.

This remains a draft pending investigation of the research prototype's reproducible dense x86 offset regressions and measurements of the exact API port. Matching hot decode instructions support a code-placement hypothesis, but do not establish the cause. No consumer is switched and no query gain is claimed.

Research evidence and the unresolved regression

On develop 9a08b82dc683cd3fff8dcde834c382141f425f48, the corrected private view-length prototype measured 13,074 ns/batch against 22,870 ns for native byte length followed by comparison on 65,536 dense view rows. The median paired ratio was 0.572, with five ratios from 0.569 to 0.580. The native path already avoids UTF-8 validation, so this gain includes eliminating its intermediate 524,288-byte length buffer. Device-buffer controls establish that the metadata binding does not request payload host reads.

Against the first private offset prototype, the corrected version regressed from 46,864 to 60,721 ns/batch under AVX2 release at 65,536 rows, with median paired ratio 1.296. The validation instruction sequences matched after relocation normalization, while linked placement changed. That regression remains unresolved and blocks qualification.

Measurements used c7i.4xlarge, Rust 1.98.0, mimalloc, AVX2, five alternating process pairs, and 30 samples per process. Benchmark used 16 codegen units and release used one, with LTO disabled in both. CPU affinity and an offline SMT sibling controlled execution. These figures measure private research bindings with infallible decode, not this API port with its different nullable policy.

[connortsui20]

Closing along with #10262, which this builds on. Its replacement is #10340.

The measured gain was about 10 µs per 65,536 rows for a length comparison, and it needed two new public unsafe input elements plus layout selection before dispatch. Length predicates already have a good path: byte_length has FSST and OnPair kernels that read stored lengths without decompressing. With #10340, a RowFn that checks value.len() on a VarBin input also gets its lengths straight from the offsets.

---

Generated by Claude Code