The benchmark is run inside a Docker container on a temporary AWS i3.large EC2 instance, just to ensure a clean machine with a local SSD volume and nothing else running in the background. We'll test the ZSTD and Deflate algorithms with a low, the default, and a high compression level.Īll the different tests and corresponding creation options that are used in the benchmark are defined in the config.ini file. Compression levels can be lowered to speed up the algorithm at the expense of the compression ratio, or raised to improve the compression ratio at the expense of speed. The ZSTD and Deflate algorithms support custom compression levels throught the ZLEVEL and ZSTD_LEVEL creation options. As the name suggests, the floating point predictor can only be used on floating point data. Predictors work especially well when there is some spatial correlation in the data, and pixels have values which are similar to their neighbours. There are three predictor settings we can test: The Deflate, LZW and ZSTD algorithms support the use of predictors, which is a method of storing only the difference from the previous value instead of the actual value. We are going to test the following compression algorithms: Experiment with your own data to find out what works best for you.
So interpret the results not as the absolute truth, but rather as an indication and a guideline of what the tradeoffs between the different algorithms and configuration options are. It is important to mention here that these are just test files, and that your results will vary depending on the type of data you're working with. Just to give an impression, this is what the Byte, Int16, and float32 images look like zoomed out:
Tiff compressor decompressor download#
See the notes and comments section for a download link and more information. Each file has been cropped to be around 50Mb each in its uncompressed condition. Three test files with commonly used data types have been created for this test: byte.tif, int16.tif, and float32.tif.
We're going to run a benchmark to test compression ratio and read/write speeds for various data types and algorithms with their respective configuration options. With the fast ZSTD compression (GDAL 2.3 and above) and Limited Error Raster Compression (GDAL 2.4) becoming available in our favourite geospatial toolkit, I thought it would be interesting to run some benchmarks and write a guide on compressing and optimizing GeoTIFF files using the latest versions of GDAL.Įspecially if you're working with GDAL's virtual file systems and cloud-optimized GeoTIFFs, deciding on the right compression algorithm and creation options can make a significant difference to indicators such as file size, processing time, and the amount of time and bandwidth consumed when accessing geospatial data over a network. Check it out at or follow on Twitter for news and updates.
It contains lots of maps and charts, all powered by open data. Shameless plug ahead! This article was (probably) written in the course of building Geofolio, which is a project I've been working on that automagically generates factsheets for any place on Earth.
0 kommentar(er)
