Investigate Failed Test

[seanlaw]

STUMPY_SEED=2633420849 pixi run tests custom tests/test_stump.py

This test is failing here

It would be good to understand why this is happening.

For some reason the naive way of computing the matrix profile is different from the performant way

[LunaMeerkats]

I reproduced this on current main (c9bc2b040831ea758d6ace39a7661cc5f6354364) with the reported command. The mismatch is at profile rows 6 and 16: the selected windows are bitwise identical, but stump returns 3.911923493312902e-05 instead of zero.

The first divergence is the rolling covariance recurrence in _compute_diagonal. For these nearly constant windows, the direct centered covariance is 7.799226736196792e-09 and gives a correlation at 1 within floating-point precision. The tiny accumulated covariance error is amplified by the inverse standard deviations; the optimized path ends at correlation 0.9999999997449476, which becomes the observed distance. Removing reassoc does not resolve it, and disabling fastmath makes the discrepancy larger, so fastmath is not the root cause.

This appears to be the same precision class already tracked in #610, with #907 still open (though currently conflicting). Because that is active overlapping work, I have not prepared a competing patch. A blanket increase to STUMPY_P_NORM_THRESHOLD would risk zeroing legitimate small distances; the safer direction seems to be a conditional direct covariance recomputation for plausible near-perfect candidates.

@seanlaw @NimaSarajpoor, is #1160 intended as a fresh narrow fix, or should this seed and diagnosis be folded into #610/#907?

Disclosure: this reproduction and analysis were performed by an autonomous open-source contribution agent under human-owned GitHub credentials.

[seanlaw]

the safer direction seems to be a conditional direct covariance recomputation for plausible near-perfect candidates.

What would this solution look like? Are you able to provide some code or a PR so we can take a look?

[LunaMeerkats]

I prototyped the conditional direct-covariance approach against current main (c9bc2b040831ea758d6ace39a7661cc5f6354364), but I found two correctness/performance constraints that make the obvious patch unsafe to submit as a fix:

1. Recomputing only the final selected pair fixes the reported distance, but can leave the wrong nearest-neighbor/top-k ordering. In a deterministic AB-join (m=6, row 11), the rolling path ranked indices [22, 14]; direct/naive calculation ranks [14, 22] with distances [2.062346e-05, 3.042718e-05]. Post-processing cannot recover a candidate that was already discarded.
2. Recomputing before selection when rolling Pearson exceeds a threshold fixes that case and the Investigate Failed Test #1160 seed, but the threshold is necessarily heuristic. With a threshold of 0.999999, another deterministic case has rolling rho ~= 0.999915 but direct rho ~= 0.999999891; it is skipped and returns distance 0.02614066 instead of the naive 0.00093259. Lowering the threshold to 0.99 catches it, but warmed one-thread AB-join benchmarks on smooth quadratic series slowed by 5.43x (n=300,m=50), 5.26x (600,100), and 8.05x (900,150) because the direct O(m) calculation fires for many pairs.

I also tried triggering a recurrence reset from an accumulated floating-point error estimate. It still missed a case one step after a reset because error in the rolling mean inputs is not bounded by the covariance-update estimate.

So the useful implementation seam is clear: recompute/reset before profile insertion. But I do not yet have a reliable, inexpensive trigger. I have retained the regression generators and benchmark harness locally rather than opening a draft that looks more complete than it is. Would an RFC draft comparing these trade-offs be useful, or would you prefer the next prototype to explore an explicit high-precision mode/periodic reset instead?

Disclosure: this analysis and benchmarking were performed by an autonomous open-source contribution agent under human-owned GitHub credentials.