Describe the bug
_write_streaming in xrspatial/geotiff/_writer.py creates a ThreadPoolExecutor for per-tile compression on the streaming tiled-write path. The pool is constructed at line 1026 and shut down with tile_pool.shutdown(wait=True) only after the tile-row loop finishes (line 1131).
If anything raises in between (a compression error, a dask compute failure, a disk write failure), the shutdown call is skipped. The outer try/except BaseException one frame up cleans up the temp file but never touches the pool. Worker threads survive the failure path and the executor leaks.
Fix: wrap the executor lifetime in try/finally, or use it as a context manager (with ThreadPoolExecutor(...) as executor:) so shutdown is guaranteed. In-flight futures should be cancelled/awaited cleanly on the error path.
Expected behavior
On any exception raised during the streaming tiled write, the ThreadPoolExecutor is shut down before the exception propagates. No worker thread should outlive the failed call.
Repro sketch
- Call
_write_streaming with a dask-backed input that compresses (e.g. compression='deflate') and a streaming_buffer_bytes small enough to force multiple horizontal segments.
- Inject a failure in the per-tile work (monkey-patch
_compress_block to raise on the second tile, or patch dask compute to raise after the first segment).
- Confirm that no
ThreadPoolExecutor worker threads remain alive after the call returns.
Additional context
test_streaming_write_parallel.py covers the happy path but does not exercise the failure path.
Describe the bug
_write_streaminginxrspatial/geotiff/_writer.pycreates aThreadPoolExecutorfor per-tile compression on the streaming tiled-write path. The pool is constructed at line 1026 and shut down withtile_pool.shutdown(wait=True)only after the tile-row loop finishes (line 1131).If anything raises in between (a compression error, a dask compute failure, a disk write failure), the shutdown call is skipped. The outer
try/except BaseExceptionone frame up cleans up the temp file but never touches the pool. Worker threads survive the failure path and the executor leaks.Fix: wrap the executor lifetime in
try/finally, or use it as a context manager (with ThreadPoolExecutor(...) as executor:) so shutdown is guaranteed. In-flight futures should be cancelled/awaited cleanly on the error path.Expected behavior
On any exception raised during the streaming tiled write, the
ThreadPoolExecutoris shut down before the exception propagates. No worker thread should outlive the failed call.Repro sketch
_write_streamingwith a dask-backed input that compresses (e.g.compression='deflate') and astreaming_buffer_bytessmall enough to force multiple horizontal segments._compress_blockto raise on the second tile, or patch dask compute to raise after the first segment).ThreadPoolExecutorworker threads remain alive after the call returns.Additional context
test_streaming_write_parallel.pycovers the happy path but does not exercise the failure path.