Skip duplicate try_numba_transform probe in the numpy reproject chunk worker#3123
Open
brendancol wants to merge 2 commits into
Open
Skip duplicate try_numba_transform probe in the numpy reproject chunk worker#3123brendancol wants to merge 2 commits into
brendancol wants to merge 2 commits into
Conversation
_reproject_chunk_numpy already probes the numba fast path before falling back to _transform_coords, which probed it again. Pass no CRS objects to _transform_coords so the inner retry is skipped. Saves one CRS-param parse and four chunk-sized coordinate allocations per output chunk on pyproj-fallback CRS pairs (~11% of the chunk worker). The cupy CPU fallbacks keep the inner probe -- it is their first numba attempt.
brendancol
commented
Jun 9, 2026
brendancol
left a comment
Contributor
Author
There was a problem hiding this comment.
PR Review: Skip duplicate try_numba_transform probe in the numpy reproject chunk worker
Blockers (must fix before merge)
- None.
Suggestions (should fix, not blocking)
- None. The change is two lines plus a comment, and the gating logic in
_transform_coords(xrspatial/reproject/init.py:284) is unchanged for callers that pass CRS objects.
Nits (optional improvements)
- The state CSV rewrite reorders the
interpolate_spline/interpolate-krigingrows. This is just the canonical writer'ssorted()ordering (-sorts before_), not data loss, but it makes the diff look bigger than it is. -
test_fallback_pair_values_match_pyproj_referenceexercises_transform_coordsdirectly rather than the chunk worker, so it overlaps a little withTestExactPrecisionEscapeHatch. Fine to keep since it pins the no-CRS-kwargs call form the worker now uses.
Verification performed
- Reverted
xrspatial/reproject/__init__.pyto the pre-fix state and re-ran the new test class:test_chunk_numpy_probes_fast_path_exactly_oncefails (probe count 2), the other two pass. With the fix it passes. The regression test is genuinely red on the old code. - Full reproject suite (test_reproject.py + coverage + cupy gate): 447 passed.
What looks good
- The fix targets only
_reproject_chunk_numpy; the cupy CPU fallbacks (_reproject_chunk_cupyline ~571,_reproject_dask_cupyline ~1790) still pass CRS objects, and the newtest_transform_coords_still_probes_when_given_crspins that contract so a future cleanup can't silently remove their only numba probe. transform_precision=0semantics are untouched: both the worker's probe and the inner retry were already gated off for the exact path, and the existingTestExactPrecisionEscapeHatchspies still pass.- WKT construction moved into a helper so collection does not touch pyproj when it is not installed.
Checklist
- Algorithm matches reference (identical inputs to both probes; skipping the second cannot change results)
- All implemented backends produce consistent results (numpy/dask+numpy affected; cupy paths unchanged)
- NaN handling is correct (untouched)
- Edge cases covered (probe-count spy, CRS-passing contract, pyproj value parity)
- Dask chunk boundaries handled correctly (per-chunk worker logic unchanged)
- No premature materialization or unnecessary copies (removes allocations)
- Benchmark exists (benchmarks/benchmarks/reproject.py covers reproject)
- README feature matrix: no change needed (no new function, no backend change)
- Docstrings: no public API change
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3106
_reproject_chunk_numpyprobed the numba fast path, then_transform_coordsprobed it again before building the pyproj control grid. The worker now passes no CRS objects to_transform_coords, which skips the inner retry (it is gated on both CRSes being non-None). Saves one CRS-param parse (~10 pyprojto_dict()/to_authority()round-trips) and four chunk-sized float64 allocations per output chunk on pyproj-fallback CRS pairs, measured at ~11% of the chunk worker for a 512x512 chunk (4326 -> Mollweide)._reproject_chunk_cupy,_reproject_dask_cupy) are unchanged: they only try the CUDA path first, so the probe inside_transform_coordsis their first and only numba attempt.Backends: affects numpy and dask+numpy (plus merge's per-block adapter, which routes through the same worker). cupy and dask+cupy behavior unchanged; validated on a CUDA host (numpy/cupy max abs diff 2e-12, on-device transform confirmed).
Test plan:
TestNoDuplicateNumbaFastPathProbe: spy asserts exactly onetry_numba_transformcall per chunk on a no-fast-path pair;_transform_coordskeeps its own probe when handed CRS objects; pyproj-reference value parity for the fallback pair