Skip to content

Unexpected behavior for rf_local_add with nodata tiles #229

Description

@emc5ud

Hi, I was playing around with rf_local_add and came across the following results. I expected that with local tile arithmatic, no_data + data = no_data.

This code snippet creates a tile with 0's and 1's, and a tile with all 0's. Then I set 1 as the no data value. This makes the tile with 0's and 1's to now have 0's and "no datas" . Finally, I add the tiles together. The tile with with 0's and 1's does have no data as expected, but the sum does not have any nodata.

tile_size = 100
x = Tile(np.eye(tile_size, dtype='int16'))
y = Tile(np.zeros((tile_size, tile_size), dtype='int16'))

rf = spark.createDataFrame(pd.DataFrame(data = 
        {'eye_tile': [x],
        'zero_tile': [y]}))

mask_ct = CellType('uint16').with_no_data_value(1)
masked_rf = rf.withColumn('eye_tile_nd', 
                           rf_convert_cell_type('eye_tile', mask_ct.cell_type_name)) \
              .withColumn('zero_tile_nd',
                          rf_convert_cell_type('zero_tile', mask_ct.cell_type_name)) \
              .withColumn('masked', rf_local_add('eye_tile_nd', 'zero_tile_nd'))

image

However, when doing the sum directly on the tiles, I do get a result with nodatas.

r = masked_rf.collect()
r[0].eye_tile_nd + r[0].zero_tile_nd

image

The second behavior is what I expected, but regardless I think the behavior should be consistent.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugWhen it really isn't a "feature".

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions