Add int8 / byte-vector similarity support - #709
Draft
r-devulap wants to merge 9 commits into
Draft
Conversation
r-devulap
requested review from
MarkWolters,
ashkrisk,
jshook and
tlwillke
as code owners
August 11, 2026 08:26
Contributor
|
Before you submit for review:
If you did not complete any of these, then please explain below. |
r-devulap
force-pushed
the
int8-support
branch
from
August 11, 2026 08:29
7aec240 to
9ee1486
Compare
Contributor
Author
|
To do:
|
jshook
requested changes
Aug 14, 2026
jshook
left a comment
jshook
left a comment
Contributor
There was a problem hiding this comment.
This PR needs to be updated.
- merge conflicts need to be addressed
- the base commit this branch depends on should probably determine the target of the merge, rather than main, since there are direct dependencies.
…ctorValues Introduce a byte-vector parallel to RandomAccessVectorValues so the HNSW pipeline can operate natively on int8 (ByteSequence<?>) vectors without any float32 round-trip. Why new classes instead of overloading ListRandomAccessVectorValues: - RandomAccessVectorValues.getVector() is contractually fixed to return VectorFloat<?>. Adding a constructor that accepts List<ByteSequence<?>> would leave getVector() unable to return the stored bytes without either a ClassCastException at runtime or silent dequantization on every access — both defeating the purpose of a native int8 path. - ByteSequence<?> and VectorFloat<?> are unrelated types with no common parent, so Java's type system offers no return-type covariance that could make a single getVector() work for both. - A separate interface (RandomAccessByteVectorValues) keeps the byte-vector path type-safe end-to-end and ensures that no existing RandomAccessVectorValues consumer can accidentally receive a ByteSequence where it expects a VectorFloat. New files: - RandomAccessByteVectorValues: interface mirroring RandomAccessVectorValues but with getVector() -> ByteSequence<?>; includes the same threadLocalSupplier() default with shared/non-shared logic (shared=false => returns this, shared=true => wraps in ExplicitThreadLocal). - ListRandomAccessByteVectorValues: List<ByteSequence<?>>-backed implementation; isValueShared()==false, copy() returns this.
…ltVectorUtilSupport Add three fundamental signed int8 vector similarity operations to the vectorization support layer so they participate in the same provider dispatch as float similarity. Bytes are treated as signed int8 (Java byte range -128..127). VectorUtilSupport — three new abstract methods: float dotProduct(ByteSequence<?> a, ByteSequence<?> b) float squareDistance(ByteSequence<?> a, ByteSequence<?> b) float cosine(ByteSequence<?> a, ByteSequence<?> b) VectorUtil — three new public static delegates: dotProduct(ByteSequence<?>, ByteSequence<?>) -> impl.dotProduct squareL2Distance(ByteSequence<?>, ByteSequence<?>) -> impl.squareDistance cosine(ByteSequence<?>, ByteSequence<?>) -> impl.cosine DefaultVectorUtilSupport — scalar loop implementations: dotProduct: accumulate (int)a.get(i) * (int)b.get(i), return as float. squareDistance: accumulate (diff * diff) for each signed byte difference. cosine: dot / sqrt(normA * normB) using per-element float promotion. PanamaVectorUtilSupport — scalar stub overrides identical to Default, so jvector-twenty compiles without requiring a SIMD implementation now. SIMD optimisation of byte similarity is a future concern.
New enum in jvector-base/.../vector/ parallel to VectorSimilarityFunction but
operating on ByteSequence<?>, delegating to the VectorUtil byte methods from
Sub-Task 2.
Three variants with return values normalised to [0,1] matching VectorSimilarityFunction
conventions (higher = more similar):
EUCLIDEAN: 1 / (1 + squaredL2 / (n * 255^2))
Normalises by the maximum possible squared distance between two signed int8
vectors (255^2 per dimension) so the result stays in (0,1] regardless of
dimension.
DOT_PRODUCT: (1 + dot / (n * 127^2)) / 2
Normalises by the maximum possible dot product magnitude (127^2 per dimension)
before applying the (1+x)/2 mapping so the result stays in [0,1] regardless
of dimension or whether vectors are unit-norm. For already unit-norm int8
vectors (e.g. Cohere, OpenAI reduced-precision) prefer COSINE.
COSINE: (1 + cosine(v1, v2)) / 2
Cosine is inherently bounded to [-1,1] so no extra normalisation is needed.
Wire RandomAccessByteVectorValues + ByteVectorSimilarityFunction into the existing BuildScoreProvider abstraction so GraphIndexBuilder can build a graph over byte vectors without any change to the builder core. New static factory: byteVectorScoreProvider(RandomAccessByteVectorValues, ByteVectorSimilarityFunction) The returned BuildScoreProvider: - isExact() -> true; all scoring stays byte×byte with no float round-trip. - approximateCentroid(): sums byte elements cast to float then divides by ravv.size() — acceptable one-time cost per the plan. - searchProviderFor(VectorFloat<?>): throws UnsupportedOperationException with a clear message; float queries not supported on the byte-only build path. - searchProviderFor(int node1): captures ravv.getVector(node1) as 'v', builds an ExactScoreFunction lambda node2 -> bvsf.compare(v, ravv.getVector(node2)), returns a DefaultSearchScoreProvider wrapping it. - diversityProviderFor(int node1): delegates to searchProviderFor(node1). - diversityScoreFunctionFor(int node1): same lambda pattern, returned directly as a ScoreFunction.ExactScoreFunction. Uses the same threadLocalSupplier() pattern as randomAccessScoreProvider() so concurrent graph builds are thread-safe: two independent supplier handles (vectors + vectorsCopy) are created so diversity comparisons don't collide.
…) overloads
convenience constructor:
GraphIndexBuilder(RandomAccessByteVectorValues vectorValues,
ByteVectorSimilarityFunction similarityFunction,
int M, int beamWidth, float neighborOverflow, float alpha,
boolean addHierarchy)
Delegates to the BuildScoreProvider constructor via
byteVectorScoreProvider(vectorValues, similarityFunction), then stores
byteVectorValues and byteVectorSimilarityFunction as nullable instance fields
for use by addGraphNode(int, ByteSequence<?>).
build(RandomAccessByteVectorValues) overload:
Parallel build loop that calls scoreProvider.searchProviderFor(node) directly
for each node ordinal, keeping all scoring byte×byte throughout construction.
Does not call getVector() and never touches VectorFloat.
addGraphNode(int, ByteSequence<?>):
Incremental ingest path for byte vectors. Builds an ExactScoreFunction lambda
node2 -> bvsf.compare(vector, byteVectorValues.getVector(node2)) and delegates
to the existing addGraphNode(int, SearchScoreProvider) — no duplication of
graph-construction logic.
Guards against misuse (called on a float-only builder) with a clear
UnsupportedOperationException pointing to the correct constructor.
Two new nullable fields added to GraphIndexBuilder:
RandomAccessByteVectorValues byteVectorValues
ByteVectorSimilarityFunction byteVectorSimilarityFunction
Both are null when the builder is constructed via a float-vector constructor
and non-null only when the byte-vector convenience constructor is used.
…raph Adding GraphIndexBuilder(RandomAccessByteVectorValues, ByteVectorSimilarityFunction, ...) made the existing null,null test call ambiguous since null satisfies both the float and byte-vector overloads. Cast the arguments to (RandomAccessVectorValues) and (VectorSimilarityFunction) to pin the call to the float constructor and restore unambiguous compilation.
Replace scalar loop implementations of dotProduct, squareDistance, and
cosine for ByteSequence with Panama Vector API implementations in
PanamaVectorUtilSupport, and native AVX-512/AVX2 kernels wired through
NativeVectorUtilSupport -> NativeSimdOps JNI bindings.
Java (Panama) path:
- Widen signed bytes to int32 via B2I conversion, accumulate products
in IntVector lanes, then reduce.
- Dispatch on PREFERRED_BIT_SIZE:
512-bit: load 16 bytes (SPECIES_128) -> IntVector.SPECIES_512
256-bit: load 8 bytes (SPECIES_64) -> IntVector.SPECIES_256
128-bit: scalar fallback
- cosine variants accumulate dot/norm products in long after reduction
to avoid int32 overflow on large vectors.
Native path (C++):
- New kernels in jvector_simd_kernels.cpp and
jvector_avx3_dl_kernels.cpp: dot_product_i8, euclidean_i8,
cosine_i8 using Highway SIMD (AVX-512 / AVX2 dispatch).
- Registered in jvector_simd_kernel_list.h and exported via
jvector_simd.cpp.
- Microbenchmarks added in bench_similarity_i8.cpp.
- C++ unit tests added in test_similarity_i8.cpp using a prime-length
(107-element) vector to exercise tail handling.
Java tests:
- TestVectorizationProvider.testSimilarityMetricsByte cross-checks
SIMD results against scalar DefaultVectorUtilSupport baseline.
ScalarQuantizer implements VectorCompressor<ByteSequence<?>> with per-dimension float32→int8 quantization. fit() computes per-dimension min/max from a base RAVV; encode()/encodeTo() map vectors linearly into [-128, 127]; encodeAll() returns an SQVectors in parallel; write()/load() serialize the dimMin/dimMax arrays so the quantizer survives an index reload from disk; dequantize() and quantizeAll() remain as helpers for graph construction and reranking. SQVectors implements CompressedVectors: holds the ByteSequence[] sidecar, provides precomputedScoreFunctionFor/diversityFunctionFor via ByteVectorSimilarityFunction, and has its own write()/load() pair for sidecar persistence. SQExample is a new tutorial that loads siftsmall fvecs, fits a ScalarQuantizer, builds an int8 graph in one parallel shot, saves the graph with original float32 InlineVectors for full-fidelity reranking, and searches every query vector. TutorialRunner gains a "sq" entry point.
Wire type: SQ in YAML into the full Grid benchmark path: CompressorParameters.SQParameters is a sentinel that Grid detects with an instanceof check before the normal float32 compressor path, routing to a dedicated buildInt8InMemory() method. buildInt8InMemory() fits a ScalarQuantizer on the base RAVV, quantizes all base vectors to int8, builds the graph via GraphIndexBuilder(RandomAccessByteVectorValues, ByteVectorSimilarityFunction), embeds the ScalarQuantizer in the index header via SQFeature (so it is recoverable at search time without access to base vectors), and writes original float32 vectors as INLINE_VECTORS for full-fidelity reranking. ConfiguredSystem gains a byteRavv field and an SQ-aware constructor. scoreProviderFor() detects SQ_QUANTIZER in the feature set, recovers the ScalarQuantizer from the OnDiskGraphIndex header, encodes the query on- the-fly, uses byte×byte scoring as the ApproximateScoreFunction for graph traversal, and falls back to the float INLINE_VECTORS reranker for final topK selection. Also adds sift-128-euclidean-int8.yml benchmark config and a corresponding datasets.yml entry.
r-devulap
force-pushed
the
int8-support
branch
from
August 17, 2026 14:22
9ee1486 to
4c789ac
Compare
Contributor
Author
Fixed, it is now rebased with the latest changes in main.
This is no longer a problem since it includes the commits in #708 |
r-devulap
marked this pull request as draft
August 18, 2026 07:51
Contributor
Author
|
Marking this as draft until I resolve the question of leveraging NVQ for INT8. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add int8 / byte-vector similarity support
Extends JVector with end-to-end support for quantized int8 (byte) vectors — from core data types and graph indexing through to SIMD-accelerated similarity kernels and benchmarks.
Note: This branch contains commits from and depends on the
simd-testingbranch (see PR #708). It is recommended to merge that PR first, or target this PR againstsimd-testinguntil it lands.Foundation
b5d3c53: Add
RandomAccessByteVectorValuesinterface andListRandomAccessByteVectorValuesByte-vector parallel to
RandomAccessVectorValues, keeping the HNSW pipeline fully type-safe onByteSequence<?>without any float32 round-trip. A separate interface is used becauseByteSequence<?>andVectorFloat<?>share no common parent and mixing them would cause silent dequantization or runtime casts.0e6c317: Add byte-similarity methods to
VectorUtilSupport/VectorUtil/DefaultVectorUtilSupportAdds
dotProduct,squareDistance, andcosinefor signed int8 vectors to the provider dispatch layer. Scalar implementations land inDefaultVectorUtilSupport;PanamaVectorUtilSupportgets stubs so the module compiles ahead of the SIMD work.a7508ba: Add
ByteVectorSimilarityFunctionenumEUCLIDEAN,DOT_PRODUCT, andCOSINEvariants normalised to[0,1], mirroringVectorSimilarityFunctionconventions for byte vectors.Graph indexing
e93df41 : Add
BuildScoreProvider.byteVectorScoreProviderfactoryWires
RandomAccessByteVectorValues+ByteVectorSimilarityFunctioninto the builder's score-provider abstraction, keeping all scoring byte×byte with no float conversion during graph construction.e1050bd:
GraphIndexBuilderbyte-vector constructor,build(), andaddGraphNode()overloadsConvenience entry points for building and incrementally updating a graph over byte vectors, reusing all existing graph-construction logic without touching
VectorFloat.1d00f35: Fix ambiguous
GraphIndexBuilderconstructor call inTestVectorGraphAdds explicit casts to resolve the overload ambiguity introduced by the new byte-vector constructor.
SIMD acceleration
ByteSequencesimilarity metrics with Panama SIMD and native AVX-512/AVX2 kernelsReplaces scalar stubs with Panama Vector API implementations (B2I widening, 512/256/128-bit dispatch) and native Highway kernels (
dot_product_i8,euclidean_i8,cosine_i8) wired through JNI. Includes C++ and Java unit tests cross-checked against the scalar baseline.Testing & benchmarks
660e806: Add
Int8IndexBuildend-to-end exampleWalks the full pipeline: load
.bvecdata, build a byte-vector graph, query with on-the-fly quantization, and report recall.39ba02d: Add
SiftLoader.readBvecsfor loading.bvecfilesAdds a reader for the binary
bvecformat along withsiftsmalldataset files for tests and examples.9ee1486: Add INT8 benchmark pipeline via
SQcompression type inBenchYAMLAdds
ScalarQuantizer, wires a newSQcompression type into the benchmark grid, and adds asift-128-euclidean-int8.ymlconfig. Queries are quantized on-the-fly; float inline vectors are stored for final reranking.