Posts made by compman

zstd would be even better if it was possible to use its new --long option, which can be associated with level -19 and multithreading for amazing compression ratio and speed, without the known issue of too large memory usage from --ultra modes (levels 20+). Downside: it requires a fairly recent version of zstd (v1.3.4+).

@george1421 zstd decompression speed is supposed to stay more or less the same whatever the compression level. Only compression becomes slower.

Could this be an IO error beyond 2 GB ?
That used to happen on older 32-bits systems, and is still something to care about with minGW.
I would not expect that from Ubuntu x64, but that conclusion may depend on the compiler used.

@Junkhacker “ultra” compression ratio (>=20) use a lot of memory.
So, either stick to “normal” compression levels (<= 19), or limit the number of threads, to ensure it doesn’t consume too much memory.

@Tom-Elliott this is really unexpected. zstd tends to save a ton of time compared to gzip, on top of a better compression ratio, especially on large files (and anything > 100 MB certainly qualifies).

Except for cases where compression time doesn’t matter, zstd good use range seems to be levels 1 - 8. Default is 3, though I tend to prefer level 5 for my own use. It always compresses better than default gzip 6.

Your test image may contains large uncompressible sections (such as already-compressed files in the image, that no algorithm can compress further). This is typical of a freshly installed Windows OS. But is it representative of fog images ?

Note that, since v1.1.3, there is a multithread mode available with zstd.

It needs to be compiled with specific flags though.
On linux, it means typing make zstdmt
For Windows, there are pre-compiled binaries in the release section : use the zstdmt one.

Since pigz is multi-threaded, it would be more fair to compare to zstdmt, rather than single-threaded zstd.

The number of threads can be selected with command -T#, like xz.