Conversation
…changes ADWIN keeps a window of recent values and, after every sample, looks for a split into an old and a recent part whose means differ by more than a variance sensitive Hoeffding bound. When it finds one the old part is dropped, so the window grows while the stream is stationary and collapses as soon as it moves, and its length becomes an estimate of how long the current regime has lasted rather than a parameter to tune. The window is stored as an exponential histogram, buckets of 1, 2, 4 ... elements with at most five of each size, so a window of n elements needs O(log n) buckets and cuts are tried only at bucket boundaries. The bucket merge carries the sum and the variance exactly, which the tests check against a direct computation over 2000 samples. Signed-off-by: alxkm <19151554+alxkm@users.noreply.github.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #7606 +/- ##
============================================
+ Coverage 81.10% 81.17% +0.06%
- Complexity 7813 7852 +39
============================================
Files 827 828 +1
Lines 24742 24865 +123
Branches 4833 4849 +16
============================================
+ Hits 20068 20184 +116
- Misses 3907 3910 +3
- Partials 767 771 +4 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds ADWIN, adaptive windowing, after Bifet and Gavalda: an average over a window whose length is not a parameter but a result.
Every windowed estimator forces the same bad choice. A long window is accurate while nothing changes and hopelessly slow once something does; a short one reacts immediately and is noisy the rest of the time. ADWIN refuses the choice. It keeps a window of recent values and, after every sample, looks for a way to split it into an old part and a recent part whose means are too far apart to be explained by chance. When it finds one, the old part is dropped. The window therefore grows on its own while the stream is stationary and collapses as soon as the stream moves, and the length it settles at is an estimate of how long the current regime has been running.
"Too far apart" is a variance sensitive Hoeffding bound. For a cut into sub-windows of
n0andn1elements, withvthe variance of the whole window:deltais a confidence level: the probability of cutting a window that never changed is bounded by it, which is what makes the resulting window length trustworthy rather than merely plausible.Keeping every sample would cost O(n) memory, so the window is stored as an exponential histogram: buckets of 1, 2, 4, 8 ... elements, at most five of each size, each holding the sum and the variance of the elements it covers. That is O(log n) buckets for a window of n elements, and cuts are tried only at bucket boundaries, which is what keeps a sample O(log n) instead of O(n) at the cost of a bounded loss of resolution. The merge of two buckets carries the variance exactly, through the usual
n0 * n1 * (u0 - u1)^2 / (n0 + n1)correction.AdwinTestcovers 19 cases. The one that matters most checks the arithmetic rather than the behaviour: over 2000 samples of stationary noise the mean and the variance recovered from the bucket histogram are compared with a direct computation over the raw samples, and they agree to 1e-9. Among the others: a constant stream is never cut and the window holds all of it, a step in the level cuts the window within 50 samples and leaves exactly the samples that followed the step, the window follows a level that changes four times, the bucket count stays inside(MAX_BUCKETS + 1) * (log2(width) + 1)for a window of 5000, the width never exceeds the number of samples seen, and a smaller delta never cuts more eagerly than a larger one.Checklist
clang-format -i --style=file path/to/your/file.java