Repository navigation
Description
Description
Encountered inaccurate results when computing exact upper-tail p-values with @stdlib/stats-wilcoxon. The upper tail is computed as a complement:
pval = 1.0 - signrankCDF( stat - 1, len );When the true tail is small, the subtraction cancels. The result has a large relative error at default settings by n = 50, and with exact: true and larger n it becomes negative or exactly 0. The two-sided value inherits the same error.
Possible fix: the null distribution is symmetric, so P(W >= s) = P(W <= M - s) with M = len*(len+1)/2. Computing the upper tail as signrankCDF( M - stat, len ) sums the small tail directly and avoids the subtraction. The same substitution applies to the first branch of the two-sided computation.
Scope: on 3,334 inputs with distinct nonzero integer values (n = 2 to 100, exact: true), the greater p-value differed from the exact value by more than 1e-12 in relative terms on 360 inputs: 13 negative, 53 exactly 0 with a positive exact value, 65 with relative error above 1%, and the rest smaller. The less p-value was accurate except for 13 values a few ulp above 1 (for example 1.000000000000002). Values this small do not change a decision at conventional thresholds; the effect is on the reported magnitude and on any further use of the p-value. The file is unchanged on develop as of 2026-11-17.
AI assistance: found by comparing outputs with exact rational reference values in an AI-assisted workflow. The examples were checked by hand.
Related Issues
Related to the separate report on two-sided p-values above 1 in the same file (link after posting). No other related issue found.
Questions
No.
Demo
No response
Reproduction
var wilcoxon = require( '@stdlib/stats-wilcoxon' );
function seq( n ) {
var out = [];
var i;
for ( i = 1; i <= n; i++ ) {
out.push( i );
}
return out;
}
// default settings (exact distribution is used for n <= 50)
console.log( wilcoxon( seq( 50 ), { 'alternative': 'greater' } ).pValue );
// exact distribution requested for n > 50
console.log( wilcoxon( seq( 51 ), { 'exact': true, 'alternative': 'greater' } ).pValue );
console.log( wilcoxon( seq( 75 ), { 'exact': true, 'alternative': 'greater' } ).pValue );All values are positive, so the statistic is at its maximum and the exact upper-tail probability is 2^-n.
Expected Results
8.881784197001252e-16
4.440892098500626e-16
2.6469779601696886e-23Actual Results
9.992007221626409e-16
-2.220446049250313e-16
0Version
@stdlib/stats-wilcoxon 0.2.3
Environments
Node.js
Browser Version
No response
Node.js / npm Version
Node.js v22.23.4
Platform
Linux x86-64 (Ubuntu 24.04)
Checklist
- Read and understood the Code of Conduct.
- Searched for existing issues and pull requests.
Activity
batpigandme commentedon Nov 17, 2026
For reference, the same bug family has turned up (and been fixed) in other implementations of the exact Wilcoxon signed-rank test. @tasuku-kobayashi has documented each case in Licklider's engineering write-ups, and I think we can follow the same approach here.
Two-sided p-value above 1 (same class as #16663)
f758eea), released in v0.12.0. Same reproducer,[1, 2, -3]→1.25. The fix doubles the selected tail and caps it at 1 (min(2 * p, 1.0)), and adds a sweep test over every untied sign pattern up ton = 12. (write-up)Upper-tail cancellation, negative or zero p-values (same class as #16664)
wilcox.test(): Bugzilla PR#19144, closed as fixed. A range guard went in first (r90638); the correction in R-devel r90641 (source and regression test) computes the requested tail directly instead of1 - CDF. Worth noting for the fix here: clamping alone would turn the upper tail into 0 rather than its small positive value. (write-up)scipy.stats.wilcoxon(method='exact'): issue #26026, fixed by PR #26027 (commit6dbd21a), a one-line tail-choice correction. A related ties/zeros case, p slightly above 1, was #26028, resolved on main. (write-up)Together these suggest a combined fix here: compute both tails directly (lower =
signrankCDF( stat, n ), upper =signrankCDF( n(n+1)/2 - stat, n )by symmetry) and takemin( 1, 2*min( lower, upper ) )for two-sided.kgryte commentedon Nov 17, 2026
@Planeshifter Given you wrote the affected package, would you mind taking a look?
[-]`@stdlib/stats-wilcoxon`: exact p-value for `alternative: 'greater'` is negative, 0, or inaccurate for small upper tails[/-][+][BUG]: `@stdlib/stats-wilcoxon`: exact p-value for `alternative: 'greater'` is negative, 0, or inaccurate for small upper tails[/+]batpigandme commentedon Nov 17, 2026
Rough fix in case it's useful that covers #16663, too. I can open a PR for both if you'd like, or you can take it from here—totally your call, @Planeshifter!
Cause: the upper tail is computed as
1 - P(W <= s-1). When the true tail is small, that CDF is within a few ulp of 1, so the subtraction leaves only rounding error. With all-positive data, where the exact p-value is2^-n,seq(51)withalternative: 'greater'returns-2.22e-16andseq(75)returns0.Fix: the null distribution is symmetric (
WandM - Whave the same distribution, withM = n(n+1)/2), soP(W >= s) = P(W <= M - s)exactly. That lets both tails be summed from the bottom of the distribution, where small probabilities keep their full relative precision, and nothing is subtracted from 1. Two-sided becomes twice the smaller tail, and every alternative is clamped to at most 1, which also resolves #16663. Againstdevelop, inlib/main.js:var abs = require( '@stdlib/math/base/special/abs' ); +var min = require( '@stdlib/math/base/special/min' ); var Float64Array = require( '@stdlib/array/float64' );Commits: batpigandme@bedf444 (the clamp for #16663) and batpigandme@16efebb (this change, on top of it).
Tests, all in
test/test.json the branch:seq(50),seq(51), andseq(75)with'greater'are checked against2^-n, andseq(75)two-sided against2^-74.seq(75)goes from0to2.6469779601696944e-23, within 10 ulp of2^-75and identical to R'swilcox.test. The remaining few ulp come fromsignrank/cdfcomputing2^-nasexp( -n*LN2 ), which isn't changed here.n = 55, flipping the sign of the smallest value gives statistic1539with'less', and the mirror case gives statistic1with'greater'. Both now return exactly1.1..nforn = 2..12, under all three alternatives (24,564 cases, under a second). The reference is the exact null distribution, built by counting subsets of{1..n}with each rank sum (cumulativeCounts, L76-L97). Forn <= 12every count is at most2^12, so dividing by2^nis exact in double precision. Each p-value has to be in[0,1]and within 7 ulp of that reference. Ondevelop, 1,790 cases fail; with only the clamp from the first commit, 1,563 still fail; with this change, none do.The sweep is the same idea as the one HypothesisTests.jl added for the same bug (JuliaStats/HypothesisTests.jl#377), extended to all three alternatives and an exact reference. If 24.5k assertions is too heavy,
n <= 10is about 6k and still catches both bugs.