User Tools

Site Tools


compression

Compression/Decompression

Using gzip, zip, pigz with 8 threads, and crabz with 8 threads on a 6.5 GB test file, one of the Blast NR tar files. crabz is very much faster for compression than the alternatives.

method compress uncompress 
gzip    335 s      74 s
zip     315 s      70 s
pgzip    41 s      43 s
crabz    21 s      50 s
gzip:  gzip nr.22.tar, gunzip nr.22.tar
zip:   zip -r nr.22.tar.zip nr.22.tar, unzip nr.22.tar.zip
pgzip: /share/apps/pigz/pigz-2.7/pigz -p 8 nr.22.tar, /share/apps/pigz/pigz-2.7/pigz -p 8 -d nr.22.tar.gz
crabz: crabz -p 8 -I nr.22.tar, crabz -d -p 8 -I nr.22.tar.gz
crabz startup: module load python/anaconda-3.8; source /share/apps/bin/conda-3.8.sh; conda activate crabz
compression.txt · Last modified: 2022/02/08 15:15 by root