Repository navigation
Non-exclusive experiment assignments on layer due to bucketization rounding error #94
Description
Is this a support request?
This issue tracker is maintained by LaunchDarkly SDK developers and is intended for feedback on the code in this library. If you're not sure whether the problem you are having is specifically related to this library, or to the LaunchDarkly service overall, it may be more appropriate to contact the LaunchDarkly support team; they can help to investigate the problem and will consult the SDK team if necessary. You can submit a support request by going here and clicking "submit a request", or by emailing support@launchdarkly.com.
Note that issues filed on this issue tracker are publicly accessible. Do not provide any private account information on your issues. If your problem is specific to your account, you should submit a support request as described above.
Describe the bug
We have multiple feature flags and run experiments for each of these flags. All experiments are on the same layer and should thus be mutually exclusive. However, we observe that sometimes, more than one feature flag evaluate to be "in the experiment" for a single context. This happens consistently on a small fraction of contexts.
By logging the conflict cases, I could create an example to reproduce the issue (see below). It turns out that the bucketization in Evaluator.java suffers from floating point precision issues. Concretely, when iterating through the weighted variations, the cumulative sum is computed by converting the weights to float and dividing by 100,000. With a large number of weights, floating point imprecisions accumulate. Since different flags have different weight lists, rounding errors are different across flags and experiment buckets can start to overlap. It would be more accurate and especially consistent between flags to sum up the integer weights and divide and convert to float after.
The weights in the example are taken from a productive flag, so the case is realistic.
To reproduce
Run the "test" (it's more of a demonstration) implemented in https://github.com/atschofen/java-core/tree/atschofen-bucketing-accuracy-test and look at the output.
Expected behavior
Experiments on the same layer should never trigger on the same context.
Logs
none
SDK version
launchdarkly-java-server-sdk: v7.10.2
Language version, developer tools
Java 19
OS/platform
MacOS 15
Additional context
none
Activity
tanderson-ld commentedon Nov 3, 2025
Hi @atschofen , thank you for reporting this and thank you for the specific concrete details, that helps tremendously. We will investigate and let you know what we find and/or fix the issue.
Logged as 1537 internally.
tanderson-ld commentedon Nov 18, 2025
@atschofen , we confirm the issue and are working with our experimentation team to determine if there is a workaround. Applying the fix could affect existing experiments; this is also being evaluated.
atschofen commentedon Jan 28, 2026
Hi @tanderson-ld! Is there an update on the issue? Thank you!
tanderson-ld commentedon Jan 30, 2026
Hi @atschofen, sorry for the delay. We don't have a workaround that you can apply yourself at this moment. The two solutions we've arrived at are the following:
For our knowledge, which of these two solutions would you prefer? It may be some time for us to implement a fix as we are currently focused on improving redundancy of the data distribution systems.
atschofen commentedon Feb 3, 2026
Thank you for the update, @tanderson-ld! We are not too worried about a "jump" in experiment assignments. We run mostly short experiments. So both options would work fine for us. The complexity concern with option 1 is more on your side I guess.
tanderson-ld commentedon Feb 4, 2026
I'll circle back with the rest of the SDK team and will let you know which way we go.
tanderson-ld commentedon Feb 13, 2026
The plot thickens @atschofen . The system wide evaluation specification calls for double precision floating point arithmetic in these related algorithms and currently this Java SDK is using
floatinstead ofdouble. We have to do deeper analysis of customer impact before we can make the fix. I don't have a timeline at this moment. Sorry for the complications.tanderson-ld commentedon Mar 12, 2026
@atschofen, we have not lost sight of this.
github-actions commentedon Jun 11, 2026
This issue is stale because it has been open for 90 days without activity. Remove the stale label or comment, or this will be closed in 14 days.
github-actions commentedon Sep 10, 2026
This issue is stale because it has been open for 90 days without activity. Remove the stale label or comment, or this will be closed in 14 days.