Skip to content

[BUG]: @stdlib/stats-wilcoxon: exact p-value for alternative: 'greater' is negative, 0, or inaccurate for small upper tails #16664

Open

Description

@tasuku-kobayashi

Description

Encountered inaccurate results when computing exact upper-tail p-values with @stdlib/stats-wilcoxon. The upper tail is computed as a complement:

pval = 1.0 - signrankCDF( stat - 1, len );

When the true tail is small, the subtraction cancels. The result has a large relative error at default settings by n = 50, and with exact: true and larger n it becomes negative or exactly 0. The two-sided value inherits the same error.

Possible fix: the null distribution is symmetric, so P(W >= s) = P(W <= M - s) with M = len*(len+1)/2. Computing the upper tail as signrankCDF( M - stat, len ) sums the small tail directly and avoids the subtraction. The same substitution applies to the first branch of the two-sided computation.

Scope: on 3,334 inputs with distinct nonzero integer values (n = 2 to 100, exact: true), the greater p-value differed from the exact value by more than 1e-12 in relative terms on 360 inputs: 13 negative, 53 exactly 0 with a positive exact value, 65 with relative error above 1%, and the rest smaller. The less p-value was accurate except for 13 values a few ulp above 1 (for example 1.000000000000002). Values this small do not change a decision at conventional thresholds; the effect is on the reported magnitude and on any further use of the p-value. The file is unchanged on develop as of 2026-11-17.

AI assistance: found by comparing outputs with exact rational reference values in an AI-assisted workflow. The examples were checked by hand.

Related Issues

Related to the separate report on two-sided p-values above 1 in the same file (link after posting). No other related issue found.

Questions

No.

Demo

No response

Reproduction

var wilcoxon = require( '@stdlib/stats-wilcoxon' );

function seq( n ) {
	var out = [];
	var i;
	for ( i = 1; i <= n; i++ ) {
		out.push( i );
	}
	return out;
}

// default settings (exact distribution is used for n <= 50)
console.log( wilcoxon( seq( 50 ), { 'alternative': 'greater' } ).pValue );

// exact distribution requested for n > 50
console.log( wilcoxon( seq( 51 ), { 'exact': true, 'alternative': 'greater' } ).pValue );

console.log( wilcoxon( seq( 75 ), { 'exact': true, 'alternative': 'greater' } ).pValue );

All values are positive, so the statistic is at its maximum and the exact upper-tail probability is 2^-n.

Expected Results

8.881784197001252e-16
4.440892098500626e-16
2.6469779601696886e-23

Actual Results

9.992007221626409e-16
-2.220446049250313e-16
0

Version

@stdlib/stats-wilcoxon 0.2.3

Environments

Node.js

Browser Version

No response

Node.js / npm Version

Node.js v22.23.4

Platform

Linux x86-64 (Ubuntu 24.04)

Checklist

  • Read and understood the Code of Conduct.
  • Searched for existing issues and pull requests.

Activity

  1. batpigandme commented on Nov 17, 2026

    @batpigandme
    Member

    For reference, the same bug family has turned up (and been fixed) in other implementations of the exact Wilcoxon signed-rank test. @tasuku-kobayashi has documented each case in Licklider's engineering write-ups, and I think we can follow the same approach here.

    Two-sided p-value above 1 (same class as #16663)

    • Julia, HypothesisTests.jl: issue #373, fixed by PR #377 (merge commit f758eea), released in v0.12.0. Same reproducer, [1, 2, -3] → 1.25. The fix doubles the selected tail and caps it at 1 (min(2 * p, 1.0)), and adds a sweep test over every untied sign pattern up to n = 12. (write-up)

    Upper-tail cancellation, negative or zero p-values (same class as #16664)

    • R, wilcox.test(): Bugzilla PR#19144, closed as fixed. A range guard went in first (r90638); the correction in R-devel r90641 (source and regression test) computes the requested tail directly instead of 1 - CDF. Worth noting for the fix here: clamping alone would turn the upper tail into 0 rather than its small positive value. (write-up)
    • SciPy, scipy.stats.wilcoxon(method='exact'): issue #26026, fixed by PR #26027 (commit 6dbd21a), a one-line tail-choice correction. A related ties/zeros case, p slightly above 1, was #26028, resolved on main. (write-up)

    Together these suggest a combined fix here: compute both tails directly (lower = signrankCDF( stat, n ), upper = signrankCDF( n(n+1)/2 - stat, n ) by symmetry) and take min( 1, 2*min( lower, upper ) ) for two-sided.

  2. kgryte commented on Nov 17, 2026

    @kgryte
    Member

    @Planeshifter Given you wrote the affected package, would you mind taking a look?

  3. added
    BugSomething isn't working.
    StatisticsIssue or pull request related to statistical functionality.
    Numerical AccuracyIssue or pull request concerns numerical accuracy.
    on Nov 17, 2026
  4. changed the title [-]`@stdlib/stats-wilcoxon`: exact p-value for `alternative: 'greater'` is negative, 0, or inaccurate for small upper tails[/-] [+][BUG]: `@stdlib/stats-wilcoxon`: exact p-value for `alternative: 'greater'` is negative, 0, or inaccurate for small upper tails[/+] on Nov 17, 2026
  5. batpigandme commented on Nov 17, 2026

    @batpigandme
    Member

    Rough fix in case it's useful that covers #16663, too. I can open a PR for both if you'd like, or you can take it from here—totally your call, @Planeshifter!

    Cause: the upper tail is computed as 1 - P(W <= s-1). When the true tail is small, that CDF is within a few ulp of 1, so the subtraction leaves only rounding error. With all-positive data, where the exact p-value is 2^-n, seq(51) with alternative: 'greater' returns -2.22e-16 and seq(75) returns 0.

    Fix: the null distribution is symmetric (W and M - W have the same distribution, with M = n(n+1)/2), so P(W >= s) = P(W <= M - s) exactly. That lets both tails be summed from the bottom of the distribution, where small probabilities keep their full relative precision, and nothing is subtracted from 1. Two-sided becomes twice the smaller tail, and every alternative is clamped to at most 1, which also resolves #16663. Against develop, in lib/main.js:

     var abs = require( '@stdlib/math/base/special/abs' );
    +var min = require( '@stdlib/math/base/special/min' );
     var Float64Array = require( '@stdlib/array/float64' );
     	var alpha;
    +	var lower;
    +	var upper;
     	var pval;
     		// Compute test statistic and p-value using exact critical values:
     		stat = T;
    +
    +		// As the null distribution is symmetric, `P(W >= s) = P(W <= M-s)`, where `M = n(n+1)/2`. Computing the upper tail directly avoids the catastrophic cancellation of `1 - P(W <= s-1)` when the tail probability is small.
     		if ( alt === 'two-sided' ) {
    -			if ( stat > ( len * ( len+1 ) / 4 ) ) {
    -				pval = 2.0 * ( 1 - signrankCDF( stat - 1, len ) );
    -			} else {
    -				pval = 2.0 * signrankCDF( stat, len );
    -			}
    +			lower = signrankCDF( stat, len );
    +			upper = signrankCDF( ( len*(len+1)/2 ) - stat, len );
    +			pval = 2.0 * min( lower, upper );
     		} else if ( alt === 'greater' ) {
    -			pval = 1.0 - signrankCDF( stat - 1, len );
    +			pval = signrankCDF( ( len*(len+1)/2 ) - stat, len );
     		} else {
     			pval = signrankCDF( stat, len );
     		}
    +
    +		// Guard against accumulated floating-point error yielding a probability greater than one:
    +		pval = min( 1.0, pval );
     	}

    Commits: batpigandme@bedf444 (the clamp for #16663) and batpigandme@16efebb (this change, on top of it).

    Tests, all in test/test.js on the branch:

    • L369-L404: the repros. seq(50), seq(51), and seq(75) with 'greater' are checked against 2^-n, and seq(75) two-sided against 2^-74. seq(75) goes from 0 to 2.6469779601696944e-23, within 10 ulp of 2^-75 and identical to R's wilcox.test. The remaining few ulp come from signrank/cdf computing 2^-n as exp( -n*LN2 ), which isn't changed here.
    • L406-L434: one-sided values that used to round above 1. For n = 55, flipping the sign of the smallest value gives statistic 1539 with 'less', and the mirror case gives statistic 1 with 'greater'. Both now return exactly 1.
    • L436-L486: an exhaustive sweep over every sign pattern of 1..n for n = 2..12, under all three alternatives (24,564 cases, under a second). The reference is the exact null distribution, built by counting subsets of {1..n} with each rank sum (cumulativeCounts, L76-L97). For n <= 12 every count is at most 2^12, so dividing by 2^n is exact in double precision. Each p-value has to be in [0,1] and within 7 ulp of that reference. On develop, 1,790 cases fail; with only the clamp from the first commit, 1,563 still fail; with this change, none do.

    The sweep is the same idea as the one HypothesisTests.jl added for the same bug (JuliaStats/HypothesisTests.jl#377), extended to all three alternatives and an exact reference. If 24.5k assertions is too heavy, n <= 10 is about 6k and still catches both bugs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

BugSomething isn't working.Numerical AccuracyIssue or pull request concerns numerical accuracy.StatisticsIssue or pull request related to statistical functionality.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions