You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
BitShuffle#shuffle for arrays of primitive datatypes does not consider the byte-order (leading to wrong results) #297
The underlying BitShuffle library works internally with an algorithm that was designed for the little-endian format. For this reason, the input data must always be passed in little-endian format.
However, the snappy-java wrapper directly passes an int[] array with the native byte order. Thus the Java implementation does not give the same results on little-endian and big-endian platforms for the shuffle / unshuffle methods.
The shuffle/unshuffle methods still work bijective so this issue is not easily noticeable.
How to fix:
The data array based on primitive types (e.g. int[], short[], long[], etc.) must be converted to little-endian format in the Java implementation if executed on big-endian platforms (before passing the buffer to the external library).
Activity
xerial commentedon May 23, 2023
Related #443