Skip to content

stream_order_d8: no memory guard on H*W working arrays #1328

@brendancol

Description

@brendancol

stream_order_d8 in xrspatial/hydro/stream_order_d8.py allocates several full-grid working arrays on its eager numpy and cupy backends with no upfront budget check. A 50000x50000 raster asks for about 100 GB of host RAM before anything errors out.

Same pattern as #1318/#1319 for flow_accumulation_d8 and #1323 for hand_d8. Hydro is safety-critical, so the same asymmetric guard applies: eager backends check, dask backends skip since per-tile allocations are already bounded by chunk size.

Allocations

_strahler_cpu (the worst case of the two CPU kernels) allocates:

Array dtype bytes/px
order float64 8
in_degree int32 4
max_in float64 8
cnt_max int32 4
queue_r int64 8
queue_c int64 8
Total 40

The dispatch wrapper also casts fd to float64 and builds an int8 stream_mask before calling the kernel. Conservative budget: 40 B/px CPU.

_stream_order_cupy allocates on the device:

Array dtype bytes/px
flow_dir_f64 float64 8
stream_mask_i8 int8 1
in_degree int32 4
state int32 4
order float64 8
max_in float64 8
cnt_max int32 4
Total 37

Plus the input fa_cp cast (8 B/px) on the device. Conservative budget: 40 B/px GPU.

Worked example

50000x50000 = 2.5e9 pixels. CPU peak working set:

2.5e9 * 40 = 100 GB

Allocated before any sanity check runs.

Fix

Mirror PR #1319: _check_memory and _check_gpu_memory helpers, wired into the numpy and cupy dispatch in stream_order_d8(). Leave dask alone. Add tests for oversize rejection, valid pass-through, dask bypass, dimensions in error message, and cupy oversize gating.

One fix per PR per the security-sweep policy. stream_order_dinf and stream_order_mfd share the shape and need separate issues.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions