AVIF conversion fails for JPG files with embedded colour profiles when not resizing

support
1

I recently added AVIF image conversion to my Hugo theme, along the same lines as the pre-existing WebP support. However, when I build my site, I will occasionally get errors like this:

ERROR error building site: [REDACTED] execute of template failed: template: _partials/images/get-image.html:57:29: executing "_partials/images/get-image.html" at <$img.Process>: error calling Process: failed to  image "[REDACTED]/images/mystery-usb-cable.jpg": encodeNRGBA: Failed to add image to encoder: Encoding of color planes failed

I’m pretty sure this is a bug, because I convert the base image into several sizes of both AVIF and WebP and most of them are generated without issue. Here is what I end up with in my resources dir for this post:

image
image860×712 114 KB

The resized AVIF files (for widths 1,200×, 800× and 500×) are fine, it’s just the full-size one that fails. For almost all of my other posts, all of the AVIF files generate without issue.

The resulting HTML is:

<picture class="picture " itemprop="image">
      <source srcset="/blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_807f0f5714f6237a.avif, /blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_44068a193038f586.avif 1200w, /blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_3c41b1cb6dc3f18d.avif 800w, /blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_346f57c16d4bec31.avif 500w" sizes="(min-width: 1248px) 60vw, 100vw" />
      <source srcset="/blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_f55502b34742e3bc.webp, /blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_a282ff089a7466f1.webp 1200w, /blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_52002bf2aa24ccf1.webp 800w, /blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_56afb9a5d9f12e6a.webp 500w" sizes="(min-width: 1248px) 60vw, 100vw" />
      <img
        class="u-photo picture__image"
          src="/blog/posts/mapping-out-my-infrastructure/images/mystery-usb-cable_hu_ba79a6b55e76a7ff.jpg"
          width="4624" height="3472" alt="A cable reading {{&lt; q &gt;}}Super High Speed {{&lt; abbr &#34;usb&#34; &gt;}}4{{&lt; /q &gt;}}" loading="lazy"/>
</picture>

All JPEGs in this post were taken on the same device (a Google Pixel 8a running GrapheneOS, using the OpenCamera app).

It seems like the issue affects all of the JPEG files within this post, but the one PNG file (router-diagram, the last set of images in the second pic) has no issue:

image
image918×719 123 KB

image
image934×693 108 KB

If I delete the files in resources and re-run the build, the same issue recurs. So it’s not a one-off error. If I run the build a second time (without deleting resources), I get no errors but the 0B files are still present.

Here is one of the raw image files that triggers the issue:

mystery-usb-cable
mystery-usb-cable1920×1442 447 KB

But other JPEG images in other posts are fine, so it’s not specifically the file format; here’s one that works for comparison:

PIA21209_hires
PIA21209_hires1432×1024 228 KB

If I open the working image in GIMP, I get no popup. If I open one of the faulty JPEGs in GIMP, I get this prompt:

image
image574×380 23.9 KB

So I imagine this is what causes the issue, but I have no idea what any of it means. But I assume that Hugo is supposed to be able to handle whatever it is, and currently does so in the .Resize path but not the .Process one.

Temporary Workaround

Here is a bash script to run in your output directory that will automatically replace broken 0B AVIF files with the largest working version of the same image:[1]

#!/usr/bin/env bash
#
# For every 0-byte AVIF file in a target directory, find the largest file
# that shares the same base filename (differing only by a numeric suffix)
# and copy that largest file over the 0-byte file.
#

set -euo pipefail

# CHANGE THIS IF YOU USE A DIFFERENT DIRECTORY
public_dir=./public

# Find every 0-byte AVIF file in the directory.
#   -print0 / while read -r -d ''  → safely handles filenames with spaces/newlines
find $public_dir -type f -name '*.avif' -size 0 -print0 |
while IFS= read -r -d '' zerofile; do

    # Strip the .avif extension to get the base name (e.g. "photo" from "photo.avif").
    basename_full=$(basename "$zerofile")
    basename_noext="${basename_full%.avif}"

    # Extract the non-numeric prefix of the base name.
    # Example: "photo_03"  →  prefix "photo_",  numeric part "03"
    #          "photo"     →  prefix "photo",   numeric part ""
    prefix="${basename_noext%%_hu_[0-9a-z]*}"

    dir=$(dirname "$zerofile")

    # Find the largest file whose base name starts with the same prefix
    # (within the same directory), excluding the 0-byte file itself.
    # We pick the single largest by byte size; ties broken by first found.
    largest=$(find "$dir" -maxdepth 1 -type f -name "${prefix}*.avif" -size +0c \
                   ! -path "$zerofile" \
                   -printf '%s\t%p\n' 2>/dev/null \
              | sort -t $'\t' -k1,1nr \
              | head -n 1 \
              | cut -f2)

    if [[ -n "$largest" && -f "$largest" ]]; then
        cp -f "$largest" "$zerofile"
        echo "Replaced 0-byte file: $zerofile"
        echo "    with largest match: $largest ($(stat -c %s "$largest") bytes)"
    else
        echo "No non-zero match found for: $zerofile (prefix: '$prefix')" >&2
    fi

done

echo "See https://discourse.gohugo.io/t/avif-conversion-fails-for-jpg-files-with-embedded-colour-profiles-when-not-resizing/57392 for discussion about this bug"

  1. First draft made with Kagi Assistant, then tweaked ↩︎

2

I tested that failing JPEG and … it didn’t fail for me, which I assume means that the JPEG you uploaded to this forum got processed by Discourse …

Could you point me to an original failing JPEG, or send it via mail to bjorn.erik.pedersen@gmail.com and I will have a look.

3

Sent via email.

4

I have checked, and this is not connected to color profile vs not:

  • The error message isn’t great (I will fix that), but the image size 4624×3472 blows the AVIF wasm sanddox (currently 384 MiB memory limit).
  • In my head, 4624×3472 photos isn’t a great user experience on the web, so for your bigger JPEGs I suggest you either scale them down before adding them to Hugo or doing a Resize in Hugo before converting to AVIF.
5

I’ve updated my template to .Resize any image with height or width >2,000px to 2,000px max. before passing the value to any further .Process or .Resize (with format conversion) calls and that has fixed it :raising_hands: (although I did have to delete my resources dir to get it to replace all of the 0B images)