Benchmark

The following tools were tested on PNG test corpus by Cédric Louvrier and Guetzli photo dataset (converted to JPEG at highest quality) by Jyrki Alakuijala.

OptimageImageOptimGuetzliJPEGminiTinyPNGSquashGoogle Photos
Lossless optimization
PNG273 711 B296 230 B304 357 B
JPEG16 861 910 B16 862 728 B
Visually lossless optimization
PNG268 084 B297 979 B217 389 B
JPEG3 717 075 B2 083 557 B4 207 455 B3 738 251 B3 579 727 B3 629 611 B2 676 969 B
Score*55/5531/5525/3118/3124/559/314/31

*Score shows how many images are visually indistinguishable from originals under normal viewing conditions. Additional tests revealed other issues.

Notes

  • ImageOptim scored 24/24 in lossy PNG compression but did not perform well on gradients and flat graphics in extended testing.
  • Tools like TinyPNG and Crunch use the original PNGQuant for color quantization, and generally perform worse than ImageOptim in terms of quality and lossless compression.
  • TinyPNG images could be losslessly compressed by 6.4% more using Optimage.
  • Guetzli achieved best quality/size ratio in some cases but failed to provide consistent quality especially at lower quality settings.
  • Guetzli is extremely slow requiring approximately 1 minute of CPU and 100 MB of memory per 1 MPix.

Test samples

Excessive ringing artifacts in JPEGmini and other tools except Guetzli
JPEGmini – 84 408 B, Optimage – 70 127 B
Excessive ringing artifacts in chroma in JPEGmini and other tools except Guetzli
JPEGmini – 54 943 B, Optimage – 74 338 B
Destructive chroma subsampling in ImageOptim, TinyPNG and Squash
ImageOptim – 76 680 B, Optimage – 140 976 B
Visible block boundary artifacts on soft gradients in Guetzli, JPEGmini and ImageOptim
Guetzli – 155 512 B, Optimage – 101 425 B
Blurred fine details in all other tools
Guetzli – 159 855 B, Optimage – 78 811 B
Blurred fine details in all other tools
JPEGmini – 26 759 B, Optimage – 26 983 B
Strong dithering artifacts on gradients in TinyPNG
TinyPNG – 16 344 B, Optimage – 536 B
Gradient banding in TinyPNG
TinyPNG – 11 968 B, Optimage – 12 262 B

Additional tests

Broken gradient in ImageOptim and TinyPNG
ImageOptim – 5 127 B, Optimage – 14 276 B
Noticeable color shift when stripping metadata in ImageOptim
ImageOptim – 72 313 B, Optimage – 176 935 B
Rotated photo when metadata containing Exif Orientation tag stripped in ImageOptim, also ImageOptim could not strip sensitive metadata in a highly compressed image, none of the tools could fix orientation losslessly, JPEGmini added a hidden metadata with version information and image hash
ImageOptim – 749 364 B, Optimage – 1 341 604 B
High local error in all other tools except Guetzli
JPEGmini – 4 382 B, Optimage – 1 448 B
Significant quality degradation caused by overcompression in JPEGmini and all other tools
JPEGmini – 18 122 B, Optimage – 25 791 B