Skip to content

reproject() output-size guard blocks legitimate lazy dask outputs #3046

@brendancol

Description

@brendancol

Describe the bug

reproject() rejects any computed output grid larger than 1 billion pixels via a hard guard in _compute_output_grid (xrspatial/reproject/_grid.py). The guard runs before backend detection and treats every backend the same. It doesn't distinguish between the paths that materialize the full output array (in-memory numpy, in-memory cupy, the streaming fallback) and the dask path, which builds a lazy map_blocks graph and never holds the whole grid in memory.

So a dask-backed input can't be reprojected when the output grid is large, even though peak memory there is bounded by the chunk size, not the total pixel count.

Example: a CONUS-wide 30 m LANDFIRE EVC raster in EPSG:5070, opened with chunks=2048, reprojected to EPSG:3857. The output grid computes to 186095 x 112377 (about 20.9 billion pixels), which is the correct size for Web Mercator over a continental extent, and the call raises:

```
ValueError: Computed output grid is too large (186095 x 112377 = 20,912,797,815 pixels, limit is 1,000,000,000). Increase the resolution parameter or reduce the output extent.
```

Expected behavior

For dask-backed (lazy) outputs the call should proceed and return a lazy dask DataArray that computes chunk by chunk on demand. The 1 billion pixel cap should apply only to backends that allocate the full output in memory: in-memory numpy, in-memory cupy, and the streaming fallback.

Additional context

The guard's own comment justifies the cap on memory grounds ("1 billion pixels ~= 8 GB for a single float64 array"), which only holds for the materializing backends. The fix is to make the guard backend-aware: bypass it when the output is a lazy dask array, keep it otherwise. No new public parameter is needed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdaskDask backend / chunked arraysoomOut-of-memory risk with large datasets

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions