[WIP] Building an optimized and accurate Jpeg Decoder [deprecated]

[antonfirsov]

Update

Anyone seeing this: The information here no longer up-to date, because we discovered PDF.js being inaccurate on progressive Jpegs. Gonna open an issue soon.

Remarks

At the moment of opening this PR, it's way too far ahead of our current master. We need to merge #275, qa-lab, and jpeg-port to wipe-out unrelated changes. Edit: Even the first comments below the PR are old comments on commits.

We should be able to merge a stable state of the new decoder into master before introducing the biggest breaking changes. In order to achieve this, we should branch jpeg-lab into a separate PR (maybe by reusing #274).

If you wish to contribute developing .NET-s first optimized managed-only open source Jpeg decoder, any help is welcome, both from new and old contributors!

Description

Porting PDF.js with #274 did a good job by introducing a cleaner stream parsing logic, capable for unified, and low-memory management of Baseline and Progressive jpeg-s. However, we will not able to speed up the decoder with integer arithmetics and lookup tables. We need to refactor/replace most of it's calculations with floating point SIMD arithmetics.

Design goals

• low-CPU, low-memory, accurate - these trade-offs are natural enemies.
• Make it modular and configurable. If we can't win on all fronts in the previous point, we can provide choices for the user.
• Unit/Integration test as much internals as possible

Non-goals

• Enabling asynchronous scan-by-scan decoding of progressive jpeg-s is not an architectural goal at this moment. We can introduce it in a future version.
• More generally: asynchronicity is not a goal at this point.

Architecture

In the first step, the decoder reads all blocks from all scans, storing them in a compact form in spectral space (FrameComponent.BlockData), even for progressive jpegs. The implementation is basically done by porting PDF.js.

In the second step we should process the spectral blocks MCU-row by MCU-row doing the following steps:

1. Block8x8 -> Block8x8F
2. IDCT + Quantization on Block8x8-s
3. SIMD colorspace conversion on float buffers
4. Pack RBA float buffers into TPixel-s and copy the result into Image<TPixel>
5. Repeat 1.-4. for each MCU row

Implementation Plan [WIP]

General:

• Structs and ref-s are overused (my fault), we can use classes for small objects having only a few instances without performance loss.
• Introduce Block8x8 struct for integer blocks.
• Split JpegDecoderCore into 3 classes:
  • SpectralJpegData: representing Huffmann-decoded, unzigged Block8x8-s for decoded components.
  • JpegStreamDecoder: produces a SpectralJpegData by consuming a stream.
• Introduce integration test for JpegStreamDecoder: we can convert SpectralJpegData into spectral-space Image<Rgba32>, and test it against spectral-space reference images. We should expect exact equality at this point.
• Speed up and the Huffman decoder.

Switch-to-float

• Implement tests for the whole spectral space -> RGB conversion chain. We should define expected results per individual block.
• Find and implement the most accurate floating-point IDCT algorithm that could be implemented with .NET SIMD capabilities. ImageSharp.Formats.Jpeg.GolangPort.Components.DCT seems to be inaccurate.
• Quantization
• SIMD colorspace-conversion.
• Pack Vector4-s into TPixel-s

[JimBobSquarePants]

@antonfirsov I'm closing this one for now so we can create a clean one based on the beta-1 codebase without any additional commits.