Compression results

1 Comment / Disks, File Systems, FreeBSD, hardware, nvme, Open Source, ZFS, ZFS / By /

Let’s tally up.

For reference, this is one of a series of posts regarding compression testing on zpools:

  1. Experimenting with compression off
  2. Experimenting with compression=lz4
  3. Experimenting with compression=zstd
  4. Compression results – you are here

Samsung-SSD-870 4TB SSD

For the Samsung-SSD-870 4TB SSD no compression we had:

  1. 2:34 = 154s
  2. 2:32 = 152s
  3. 2:30 = 150s
  4. 3:52 = 238s
  5. 4:46 = 286s
  6. 4:16 = 256s
  7. 4:15 = 255s
  8. 4:15 = 255s
  9. 4:49 = 289s

With lz4 we have:

  1. 2:01 = 121s
  2. 2:02 = 122s
  3. 2:04 = 124s
  4. 2:05 = 125s
  5. 3:08 = 188s
  6. 3:07 = 187s
  7. 3:00 = 180s
  8. 3:16 = 196s
  9. 3:11 = 181s

With zstd we got:

  1. 1:51 = 119s
  2. 1:47 = 107s
  3. 1:45 = 105s
  4. 1:45 = 105s
  5. 1:51 = 119s
  6. 2:36 = 156s
  7. 2:36 = 156s
  8. 2:27 = 146s
  9. 2:33 = 153s

The zpool sizes:

Samsung-SSD-870-no-compression    3.62T   324G  3.31T        -         -     0%     8%  1.00x    ONLINE  -
Samsung-SSD-870-compression-lz4   3.62T   191G  3.44T        -         -     0%     5%  1.00x    ONLINE  -
Samsung-SSD-870-compression-zstd  3.62T   148G  3.48T        -         -     0%     3%  1.00x    ONLINE  -

For the base.txz tarball contents, a combination of binary and some text, zstd is the clear winner here. Faster and more compressed.

Samsung-SSD-980-PRO 1TB NVMe

For no compression we had:

  1. 1:42 = 102s
  2. 1:44 = 104s
  3. 1:45 = 105s
  4. 1:44 = 104s
  5. 1:42 = 102s
  6. 1:44 = 104s
  7. 1:43 = 103s
  8. 1:42 = 102s
  9. 1:48 = 108s

For lz4:

  1. 1:42 = 102s
  2. 1:42 = 102s
  3. 1:41 = 101s
  4. 1:43 = 103s
  5. 1:43 = 103s
  6. 1:42 = 102s
  7. 1:42 = 102s
  8. 1:43 = 103s
  9. 1:48 = 108s

For ztd:

  1. 1:42 = 102s
  2. 1:40 = 100s
  3. 1:35 = 95s
  4. 1:32 = 92s
  5. 1:33 = 93s
  6. 1:33 = 93s
  7. 1:30 = 90s
  8. 1:33 = 93s
  9. 1:31 = 91s
NAME                                  SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Samsung-SSD-980-PRO-no-compression    928G   324G   604G        -         -     0%    34%  1.00x    ONLINE  -
Samsung-SSD-980-PRO-compression-lz4   928G   191G   737G        -         -     0%    20%  1.00x    ONLINE  -
Samsung-SSD-980-PRO-compression-zstd  928G   148G   780G        -         -     0%    15%  1.00x    ONLINE  -

Compressed and lz2 are nearly the same. ztd wins again on both speed and compression.

Samsung-SSD-990-EVO 4TB NVMe

With no compression:

  1. 1:46 = 106s
  2. 1:44 = 104s
  3. 1:43 = 103s
  4. 1:43 = 103s
  5. 1:42 = 102s
  6. 1:42 = 102s
  7. 1:43 = 103s
  8. 1:42 = 102s
  9. 1:43 = 103s

With lz4 compression:

  1. 1:46 = 106s
  2. 1:48 = 108s
  3. 1:48 = 108s
  4. 1:42 = 102s
  5. 1:42 = 102s
  6. 1:42 = 102s
  7. 1:43 = 103s
  8. 1:42 = 102s
  9. 1:43 = 103s

With zstd:

  1. 1:30 = 90s
  2. 1:29 = 89s
  3. 1:30 = 90s
  4. 1:30 = 90s
  5. 1:28 = 88s
  6. 1:28 = 88s
  7. 1:28 = 88s
  8. 1:28 = 88s
  9. 1:27 = 87s
NAME                                  SIZE   ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Samsung-SSD-990-EVO-no-compression    3.62T   324G  3.31T        -         -     0%     8%  1.00x    ONLINE  -
Samsung-SSD-990-EVO-compression-lz4   3.62T   191G  3.44T        -         -     0%     5%  1.00x    ONLINE  -
Samsung-SSD-990-EVO-compression-zstd  3.62T   148G  3.48T        -         -     0%     3%  1.00x    ONLINE  -

Again, zstd for the win.

My conclusion: zstd wins. And I should be using the 4TB NVMe with zstd for my FreshPorts development nodes.

Website Pin Facebook Twitter Myspace Friendfeed Technorati del.icio.us Digg Google StumbleUpon Premium Responsive

Related Posts

1 thought on “Compression results”

  1. mirror176

    For testing compression, yes caching can impact the work to read from original disks; consider shutting down ARC for related pools if its impact in the tests is not part of what you want to measure. If you were not reading from cache then you should think about if the read source was a bottleneck. With zstd the speed of decompression should be much faster that compression. Timing a cp command is not adequate to get proper time measurements since the cp command will return before the data is actually written to disk. If you copy enough data in the transfer then you can make that be a mathematically insignificant part of the test. With zstd compression setting increasing you should see write speeds rapidly fall. Decompression speed (assuming compressed records) will usually increase as compression setting increases but only up to a point. Using zstd outside of ZFS (=newer copy) had maximum decompression speed around compression 12-15 on my hardware if memory serves. I think last I ran that the standalone zstd didn’t have multithreaded decompression while ZFS imitates it by passing multiple records to multiple threads for compression and decompression steps. As such, maximum throughput impact of these settings will be dependent on both the performance and count of CPU cores that are available. If memory serves, hyperthreading cores did not benefit these tasks but I didn’t test limiting ZFS compression threads to < or = physical cores only; actually ZFS doesn't stay below total core count on my system for some reason which coupled with any decent zstd compresion setting causes pretty bad lag when it happens.
    I think in my last benchmarking effort I was trying to get my time after a zpool sync. I was writing an iterative loop to memory disk test different GELI ciphers+keylengths + ZFS ashift+compression+recordsize settings. I had intention to integrate ZFS encryption into the testing and run similar with GBDE or whatever the other GEOM encryption is but gave up for now as I continued to see difficulty measuring time properly and GELI detatch + create step was exposing a bug.
    If I get back to it then I need to later follow up with similar measurements on physical hardware (at least of any values near interesting ranges from memory tests) and I too should have shut down ARC for aspects where I am not measuring impacts on ARC.

Leave a Comment

Scroll to Top