Experimenting with compression=zstd

Following on from my previous post where I tested lz4, let’s try compression=zstd.

1. Experimenting with compression off
2. Experimenting with compression=lz4
3. Experimenting with compression=zstd – you are here
4. Compression results

First, we delete the old:

[21:19 r730-01 dvl ~/tmp] % sudo zpool destroy Samsung-SSD-870-compression-lz4                                                                        
[21:19 r730-01 dvl ~/tmp] % sudo zpool destroy Samsung-SSD-980-PRO-compression-lz4
[21:19 r730-01 dvl ~/tmp] % sudo zpool destroy Samsung-SSD-990-EVO-compression-lz4

Then create the new:

[21:19 r730-01 dvl ~/tmp] % sudo zpool create Samsung-SSD-980-PRO-compression-zstd mirror /dev/nda1 /dev/nda2
[21:20 r730-01 dvl ~/tmp] % sudo zpool create Samsung-SSD-990-EVO-compression-zstd mirror /dev/nda0 /dev/nda3
[21:20 r730-01 dvl ~/tmp] % sudo zpool create Samsung-SSD-870-compression-zstd mirror /dev/da12 /dev/da13
[21:20 r730-01 dvl ~/tmp] % sudo zfs set compression=zstd Samsung-SSD-980-PRO-compression-zstd
[21:21 r730-01 dvl ~/tmp] % sudo zfs set compression=zstd Samsung-SSD-990-EVO-compression-zstd
[21:21 r730-01 dvl ~/tmp] % sudo zfs set compression=zstd Samsung-SSD-870-compression-zstd
[21:22 r730-01 dvl ~/tmp] % sudo chown dvl:dvl /Samsung-SSD-*                                   
[21:22 r730-01 dvl ~/tmp] % 

Samsung-SSD-870-compression-zstd – 4TB ssd

Another set:

[21:28 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test1                       
[21:28 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test1
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test1  1.17s user 65.63s system 59% cpu 1:51.82 total

[21:30 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test2                
[21:31 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test2
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test2  1.17s user 65.12s system 61% cpu 1:47.69 total

[21:33 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test3                
[21:33 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test3
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test3  1.51s user 63.81s system 61% cpu 1:45.83 total

[21:35 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test4                
[21:35 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test4
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test4  1.05s user 64.27s system 61% cpu 1:45.91 total

[21:37 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test5                
[21:37 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test5
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test5  1.22s user 64.64s system 59% cpu 1:51.41 total

[21:39 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test6                
[21:39 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test6
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test6  1.24s user 68.56s system 44% cpu 2:36.80 total

[21:42 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test7                
[21:42 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test7
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test7  1.28s user 67.70s system 44% cpu 2:36.72 total

[21:44 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test8                
[21:45 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test8
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test8  1.31s user 67.04s system 46% cpu 2:27.37 total

[21:48 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test9                 
[21:48 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test9 
cp -r DELETE-ME* /Samsung-SSD-870-compression-zstd/test9  1.17s user 68.21s system 45% cpu 2:33.72 total

[21:52 r730-01 dvl ~] % zpool list Samsung-SSD-870-compression-zstd
NAME                               SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Samsung-SSD-870-compression-zstd  3.62T   148G  3.48T        -         -     0%     3%  1.00x    ONLINE  -

Samsung-SSD-980-PRO-compression-zstd – 1TB NVMe

Another set:

[21:51 r730-01 dvl ~/tmp] % ls -l /Samsung-SSD-980-PRO-compression-zstd 
total 0
[21:51 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test1
[21:51 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test1
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test1  1.14s user 68.92s system 68% cpu 1:42.27 total

[21:53 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test2                
[21:54 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test2
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test2  1.06s user 65.03s system 65% cpu 1:40.93 total

[21:55 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test3                
[21:55 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test3
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test3  1.09s user 64.23s system 68% cpu 1:35.16 total

[21:57 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test4                
[21:57 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test4
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test4  1.23s user 64.00s system 70% cpu 1:32.56 total

[21:59 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test5                
[21:59 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test5
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test5  0.99s user 63.88s system 69% cpu 1:33.13 total

[22:01 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test6                
[22:01 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test6
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test6  0.95s user 64.41s system 70% cpu 1:33.18 total

[22:02 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test7                
[22:03 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test7
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test7  1.20s user 63.17s system 70% cpu 1:30.81 total

[22:04 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test8                
[22:05 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test8
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test8  1.09s user 64.28s system 69% cpu 1:33.52 total

[22:06 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test9                
[22:07 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test9
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test9  1.00s user 64.02s system 71% cpu 1:31.39 total

[21:52 r730-01 dvl ~] % zpool list Samsung-SSD-980-PRO-compression-zstd
NAME                                   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Samsung-SSD-980-PRO-compression-zstd   928G   148G   780G        -         -     0%    15%  1.00x    ONLINE  -

Samsung-SSD-990-EVO-compression-zstd – 4TB NVMe

And one more:

[22:08 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test1
[22:09 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test1 
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test1  1.03s user 64.17s system 71% cpu 1:30.83 total

[22:10 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test2                
[22:11 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test2
[cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test2  1.02s user 63.09s system 71% cpu 1:29.47 total

[22:12 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test3               
[22:13 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test3
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test3  0.97s user 64.08s system 71% cpu 1:30.38 total

[22:14 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test4                
[22:16 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test4
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test4  1.12s user 63.86s system 72% cpu 1:30.04 total

[22:18 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test5                
[22:18 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test5
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test5  1.13s user 63.49s system 72% cpu 1:28.66 total

[22:20 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test6                
[22:23 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test6
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test6  1.05s user 63.47s system 73% cpu 1:28.16 total

[22:25 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test7                
[22:25 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test7
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test7  1.03s user 63.42s system 73% cpu 1:28.25 total

[22:27 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test8                
[22:27 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test8
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test8  1.00s user 63.73s system 73% cpu 1:28.19 total

[22:28 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test9                
[22:28 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test9
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test9  1.02s user 63.30s system 73% cpu 1:27.77 total

[22:30 r730-01 dvl ~/tmp] % zpool list Samsung-SSD-990-EVO-compression-zstd
NAME                                   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
Samsung-SSD-990-EVO-compression-zstd  3.62T   148G  3.48T        -         -     0%     3%  1.00x    ONLINE  -

After pizza

That night, it was pizza night, and it was good.

Today, I thought I’d run one two more tests on each of the zstd zpools. Just for fun.

Why two? Caching, perhaps.

Before:

[11:52 r730-01 dvl ~/tmp] % ls /Samsung-SSD-*
/Samsung-SSD-870-compression-zstd:
test1/ test2/ test3/ test4/ test5/ test6/ test7/ test8/ test9/

/Samsung-SSD-980-PRO-compression-zstd:
test1/ test2/ test3/ test4/ test5/ test6/ test7/ test8/ test9/

/Samsung-SSD-990-EVO-compression-zstd:
test1/ test2/ test3/ test4/ test5/ test6/ test7/ test8/ test9/

Starting off:

[11:52 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test10
[11:53 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* //Samsung-SSD-870-compression-zstd/test10
cp -r DELETE-ME* //Samsung-SSD-870-compression-zstd/test10  1.16s user 68.07s system 51% cpu 2:15.47 total

[11:55 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-870-compression-zstd/test11                 
[11:56 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* //Samsung-SSD-870-compression-zstd/test11
cp -r DELETE-ME* //Samsung-SSD-870-compression-zstd/test11  1.17s user 64.65s system 54% cpu 1:59.90 total

I call that a significant difference between the two runs. The original runs were in the 1:45-2:33 range. My theory: the input is now cached.

Next:

[11:58 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test10
[11:59 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test10
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test10  1.12s user 65.26s system 58% cpu 1:54.03 total

[12:01 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-980-PRO-compression-zstd/test11                
[12:02 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test11
cp -r DELETE-ME* /Samsung-SSD-980-PRO-compression-zstd/test11  1.25s user 65.50s system 59% cpu 1:52.88 total

Both slightly slower the same as the original times for this zpool (1:31-1:42).

Finally:

[12:11 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test10                
[12:11 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test10
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test10  1.29s user 64.88s system 56% cpu 1:56.45 total

[12:13 r730-01 dvl ~/tmp] % mkdir /Samsung-SSD-990-EVO-compression-zstd/test11                
[12:14 r730-01 dvl ~/tmp] % time cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test11
cp -r DELETE-ME* /Samsung-SSD-990-EVO-compression-zstd/test11  1.17s user 64.79s system 58% cpu 1:52.55 total

The original tests for this zpool were in the 1:27-1:30 range. These are both 20s slower. No idea why.

Good bye, and thanks for all the fiche

That’s fiche as in short for microfiche. As in storing lots of information in a small space.

[12:21 r730-01 dvl ~/tmp] % sudo zpool destroy Samsung-SSD-870-compression-zstd
[12:21 r730-01 dvl ~/tmp] % sudo zpool destroy Samsung-SSD-980-PRO-compression-zstd
[12:21 r730-01 dvl ~/tmp] % sudo zpool destroy Samsung-SSD-990-EVO-compression-zstd
[12:21 r730-01 dvl ~/tmp] % 

Next, I want to explore the GEOM errors caused by the lingering gpart information on the 1x 1TB NVMe sticks.