klauspost/compress is 21 megabytes

[nhooyr]

Some user's reported an issue with the dependency being large diamondburned/arikawa#11

I confirmed, the dependency is 21 megabytes.

I think the best approach would be to vendor only the code we need from klauspost/compress.

cc @klauspost

[diamondburned]

Seeing how the compress PR used github.com/klauspost/compress/flate, can the stdlib compress/flate be used instead?

[nhooyr]

stdlib compress/flate is significantly slower. See klauspost/compress#216 (comment)

[nhooyr]

And allocates much more memory.

[diamondburned]

I think 2x slower isn't that big of a deal. If anything, the library can also allow an interface to give developers choices when it comes to performance-intensive tasks.

[diamondburned]

Alternatively, I think @klauspost should really split up his repositories. Seeing how the largest file seems to be from compress/zstd, he could probably split 5 of his subpackages into 5 repositories, and we could probably import only the flate one.

Alternatively again, @klauspost should PR his things into the stdlib.

[ravener]

The question is why is he pushing test data to the repo.

Anyway vendoring only needed code sounds good.

[klauspost]

No. If 21 megs is a problem for you, do what you want.

[nhooyr]

The question is why is he pushing test data to the repo.

Everyone pushes tests to repos.

Like @klauspost said, I'm not entirely sure this is a problem worth solving.

[nhooyr]

I think the default should use stdlib and then I can maintain a separate branch if there's enough interest until/if stateless compression ends up in the stdlib. It's always nice to have zero dependencies as well.