Skip to content

VpDivd bugfix#374

Merged
tompng merged 1 commit intoruby:masterfrom
tompng:vpdivd_fix
Jul 15, 2025
Merged

VpDivd bugfix#374
tompng merged 1 commit intoruby:masterfrom
tompng:vpdivd_fix

Conversation

@tompng
Copy link
Copy Markdown
Member

@tompng tompng commented Jul 14, 2025

Fix VpDivd bug which is a blocker of #371.

Fix rounding: VpDivd should always truncate. Remove wrong optimization that uses VpAsgn which rounds with the current rounding mode.

VpDivd(div, rem, a, b)
a=0.9999999999 9999999999 9999999999 999000000E3 (1, 5, 5)
b=0.1E1 (1, 1, 1)

# Before
div=0.1000E4 (1, 1, 3) # Wrong. It shouldn't be rounded with the current rounding mode.
rem=0.0 # Wrong. It shouldn't be zero.

# After
div=0.9999999999 99000000E3 (1, 3, 3)
rem=0.9999999999 99000000E-18 (-2, 2, 6)
rem=0.9999999999 9999999999 9000000E-9

Fix VpDivd loop bug that sometimes skip calculating last DECDIG.

# OK case
VpDivd(div, rem, a, b)
a=0.3000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0001999999 999E73 (9, 9, 10)
b=0.10E2 (1, 1, 1)

div=0.3000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0001999999 99E72 (8, 8, 10)
rem=0.9E1 (1, 1, 10)

# Bug case
VpDivd(div, rem, a, b)
a=0.3000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000999999 999E73 (9, 9, 10)
b=0.10E2 (1, 1, 1)

# Before. a, b, div, rem has same precision as OK case, but the last DECDIG calculation is somehow skipped
div=0.300000000E72 (8, 1, 10)
rem=0.999999999E9 (1, 1, 10)

# After
div=0.3000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000999999 99E72 (8, 8, 10)
rem=0.9E1 (1, 1, 10)

Fix rounding of VpDivd with one: VpDivd can't use VpAsgn which rounds with the current rounding mode.
Fix VpDivd loop bug that sometimes skip calculating last DECDIG.
@tompng
Copy link
Copy Markdown
Member Author

tompng commented Jul 15, 2025

I want to add a test for VpDivd, but the current VpDivd calculation is weird.

VpDivd(c, r, a, b);
c->frac[0] // This value is always zero.

I'll add a test in a separate pull request with VpDivd fix.

@tompng tompng merged commit 4d7bb5b into ruby:master Jul 15, 2025
79 checks passed
@tompng tompng deleted the vpdivd_fix branch July 15, 2025 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant