Skip to content

[BUG]: stats/base/dists/f/cdf loses accuracy for small x and returns 0 for very small x #16655

Open

Description

@clarkf19

Description

The F distribution CDF is inaccurate for small x and returns exactly 0 once d1*x/d2 is below about 1e-16.

For d1*x <= d2, the implementation evaluates betainc( d2/(d2+(d1*x)), d2/2, d1/2, true, true ). The first argument is 1 - d1*x/(d2+d1*x), which loses most of its significant digits when x is small (and rounds to exactly 1 for tiny x), so the upper regularized incomplete beta is computed from a heavily rounded input.

The branches appear to be swapped relative to the numerically stable choice (also used by Boost's fisher_f CDF): when d1*x <= d2, compute betainc( (d1*x)/(d2+(d1*x)), d1/2, d2/2, true, false ) (small argument, no cancellation), and when d1*x > d2, compute betainc( d2/(d2+(d1*x)), d2/2, d1/2, true, true ).

With the branches swapped, the maximum relative error over a grid of d1, d2 in {0.5, 1, 2, 3, 5, 10, 30} and x from 1e-20 to 1e8 drops from 1.0 to about 1.6e-15 (compared with SciPy and mpmath).

The same logic is in lib/main.js, lib/factory.js, and src/main.c.

Related Issues

None

Questions

No.

Demo

N/A

Reproduction

var cdf = require( '@stdlib/stats/base/dists/f/cdf' );

console.log( cdf( 1.0e-20, 2.0, 3.0 ) );
console.log( cdf( 1.0e-12, 2.0, 3.0 ) );
console.log( cdf( 1.0e-12, 10.0, 30.0 ) );

Expected Results

1.0e-20
9.9999999999916667e-13
4.7851851851586008e-59

(reference values computed with mpmath at 40 digits; SciPy's `f.cdf` agrees)

Actual Results

0
1.0000333894309431e-12
4.789970438279987e-59

Version

@stdlib/stats-base-dists-f-cdf 0.2.3 (the same code is on develop)

Environments

Node.js

Browser Version

N/A

Node.js / npm Version

Node.js v24.16.0, npm 11.13.0

Platform

Windows 11

Checklist

  • Read and understood the Code of Conduct.
  • Searched for existing issues and pull requests.

Activity

  1. stdlib-bot commented on Nov 16, 2026

    @stdlib-bot
    Contributor

    👋 Hi there! 👋

    And thank you for opening your first issue! We will get back to you shortly. 🏃 💨

    If you have any further questions while waiting for a response, please join our Zulip community to chat with project maintainers and other community members.

  2. added
    BugSomething isn't working.
    MathIssue or pull request specific to math functionality.
    Numerical AccuracyIssue or pull request concerns numerical accuracy.
    AcceptedRFC feature request which has been accepted.
    on Nov 16, 2026
  3. changed the title [-]stats/base/dists/f/cdf loses accuracy for small x and returns 0 for very small x[/-] [+][BUG]: stats/base/dists/f/cdf loses accuracy for small x and returns 0 for very small x[/+] on Nov 16, 2026
  4. changed the title [-][BUG]: stats/base/dists/f/cdf loses accuracy for small x and returns 0 for very small x[/-] [+][BUG]: `stats/base/dists/f/cdf` loses accuracy for small x and returns 0 for very small x[/+] on Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    AcceptedRFC feature request which has been accepted.BugSomething isn't working.MathIssue or pull request specific to math functionality.Numerical AccuracyIssue or pull request concerns numerical accuracy.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions