Build timeout Jpeg vs Webp processing

Polarhardboiled · May 11, 2021, 6:47pm

In order to try the new and much expected Webp image processing, I have modified the way to process images in a working projet.

I only add “Webp” to all codes such as Resize, etc for image processing.

That’s the only change made in the project.

With usual jpeg processing the build time was 595 ms
With the new Webp the build time was 91672 ms (it was necessary to add a timeout config to have the site successfully deployed by Netlify)
Same difference on my local machine in development environment.

You can see both screen snapshots to make the comparaison.

Is it normal or is there a way to better use this feature ? And is it worth using webp image processing ?

Thanks for advice

alexandros · May 11, 2021, 7:12pm

The current WEBP implementation relies on an external library hence the requirement for Hugo Extended.

If there were plenty of images upon the first conversion it is normal to have a longer build time as these files need to be processed all at the same time.

However on subsequent builds you should see the time required return to what you had before as the generated files will be cached under the resourceDir.

I suggest that you search the web for this question.
There are a lot of articles regarding WEBP vs JPEG etc.

chrillek · May 11, 2021, 9:17pm

That depends on how you define “worth”. In any case, it makes Google’s lighthouse happy, which in turn might influence your page’s rank in search results. But if that’s “worth it” is up to you to decide.

andreseduardop · May 11, 2021, 9:34pm

@Polarhardboiled Can I suggest an external processor to generate the Webp images?
Infranview’s batch conversion function has worked well for me for this job.

jmooring · May 12, 2021, 12:45am

Someone who casually reads this topic in the future might reach the erroneous conclusion that Hugo’s WebP conversion is slower than other internal image conversions. It is not.

I created a minimal site with 100 JPEG photographs. Each image is approximately 6.0 MB, 4032x3024. Then I ranged through all of the images (sequential processing).

Before running each test, I cleared the cache with:

rm -rf resources/_gen/images

ID	Description	Code	Time (seconds)
1	Resize	`.Resize "600x"`	55
2	Resize to PNG	`.Resize "600x png"`	63
3	Resize to GIF	`.Resize "600x gif"`	76
4	Resize to WebP	`.Resize "600x webp"`	55

I ran each of these tests 3 times, presenting the average in the table above.

Comparing tests 1 and 4, there is not a discernible difference between resizing a JPEG vs. resizing plus conversion to WebP.

As @alexandros describes, once you have processed the images they are cached, and subsequent builds do not need to process the images again. For example, after the last test in the table above, I ran hugo without clearing the cache. The total build time was 0.023 seconds.

I understand that these four tests do not produce identical images. There are always trade-offs between format, size, quality, and speed, but in my view Hugo’s defaults are sane, and this comparison should allay any performance concerns.

Polarhardboiled · May 12, 2021, 7:13am

Thanks a lot.
Relying on your comments, my conclusion is:

Hugo is still efficient and fast whatever the image processing used
choosing between Webp vs jpeg is not directly linked to Hugo process but more to general Web site implementation
I have to investigate why my build time is so long compared with the @jmooring test (by the way, I didn’t know that processing to png, gif was possible)

davidsneighbour · May 12, 2021, 10:04am

We would need to see sample code to know more, but my guess in the dark is that “all codes” is overkill. You probably convert multiple times when you only need to convert once and then work on the image (or work on the image and then convert).

You could start the whole layout by loading an image resource, then resize it to the maximum size required with setting it to webp and then ignoring webp for the rest of the procedures and using copies of that first resource.

Polarhardboiled · May 12, 2021, 10:22am

I have a partial image-header for each post as follows:

{{ with $.Resources.GetMatch "*header*" }}
      {{ $image100 := .Resize "20x webp q75 Center" }}
      {{ $image600 := .Resize "600x webp q85 Center" }}
      {{ $image900 := .Resize "900x webp q85 Center" }}
      {{ $image1200 := .Resize "1200x webp q85 Center" }}
{{ $image1600 := .Resize "1600x webp q85 Center" }}
        <link itemprop="contentUrl" href="{{ .RelPermalink }}">
        <meta itemprop="encodingFormat" content="{{ .MediaType }}">
        <meta itemprop="name" content="{{ .Name | plainify }}">
        <img
          loading="lazy"
          itemprop="thumbnailUrl"
          alt="{{ .Name | plainify }}"
          class="lazyload blur-up"
          src="{{ $image100.RelPermalink }}"
          data-srcset="{{ $image600.RelPermalink }} 600w, {{ $image900.RelPermalink }} 900w, {{ $image1200.RelPermalink }} 1200w, {{ $image1600.RelPermalink }} 1600w"
          data-sizes="auto"
        >{{ end }}

I have also a gallery shortcode as:

{{ with $.Get"title" }}<h3>{{ . }}</h3>{{ end }}
{{ with $.Get "content"}}<h4>{{ . | markdownify }}</h4>{{ end }}
{{ $seed := .Get "src" }}
{{ $unique := delimit (shuffle (split (md5 $seed) "" )) "" }}
<div class="gallery">
  {{ range $image := (.Page.Resources.ByType "image").Match (printf "%s*" (.Get "src")) }}
  {{ $src := $image }}
  {{ $thumb := .Resize "500x webp" }}
  <a data-fslightbox="{{ $unique | safeHTMLAttr }}" href="{{ $src.RelPermalink }}">
      <img
      loading="lazy"
      class="lazyload blur-up"
      data-src="{{ $thumb.RelPermalink }}"
      alt="{{ .Name | markdownify }}"
      >
  </a>
  {{ end }}
</div>

And other small pieces of code dealing with image thumbs

You probably right when suggesting to convert only once.

davidsneighbour · May 12, 2021, 10:28am

Quick and untested:

{{ with $.Resources.GetMatch "*header*" }}
      {{ $imageLarge := .Resize "1600x webp q85 Center" }}
      {{ $image100 := $imageLarge.Resize "20x q75 Center" }}
      {{ $image600 := $imageLarge.Resize "600x Center" }}
      {{ $image900 := $imageLarge.Resize "900x Center" }}
      {{ $image1200 := $imageLarge.Resize "1200x Center" }}

The first resize takes the largest required format, the others work with the already transformed and just resize them. Also if your quality does not change then no need to tell Hugo again. I am not sure about the Center, so let’s leave it in. If I remember it properly then that parameter would have been a point of focus if the resizing is absolute and cuts off something. In your case the width is dynamic depending on the resized height and Center should not be needed in all cases.

jmooring · May 12, 2021, 4:49pm

This is not true. Tools such as ImageMagick will, by default, use the q value of the source file. Hugo does not do this. The default value is 75, regardless of the q value of the source file.

The anchor option (Center, TopLeft, etc.) is irrelevant when resizing. This option is only applicable to the .Fill method.

As @davidsneighbour explains, there is a significant reduction in processing time using a previous transformation as the source.

I created a minimal site with 25 JPEG photographs. Each image is approximately 6.0 MB, 4032x3024. Then I ranged through all of the images (sequential processing).

Before running each test, I cleared the cache with:

rm -rf resources/_gen/images

The time shown for each test is the average of 3 runs.

Test 1 - Original code

{{ range resources.Match "*.jpg" }}
  {{ $image1600 := .Resize "1600x webp q85" }}
  {{ $image1200 := .Resize "1200x webp q85" }}
  {{ $image900 :=  .Resize "900x webp q85" }}
  {{ $image600 :=  .Resize "600x webp q85" }}
  {{ $image100 :=  .Resize "100x webp" }}
{{ end }}

68 seconds

Test 2 - Use result of initial transformation for subsequent transformations

{{ range resources.Match "*.jpg" }}
  {{ $image1600 := .Resize "1600x webp q85" }}
  {{ $image1200 := $image1600.Resize "1200x webp q85" }}
  {{ $image900 :=  $image1600.Resize "900x webp q85" }}
  {{ $image600 :=  $image1600.Resize "600x webp q85" }}
  {{ $image100 :=  $image1600.Resize "100x webp" }}
{{ end }}

44 seconds (35% reduction compared to Test 1)

Test 3 - Same as Test 2 with cleaner code

In additional to cleaner code, this test proves that specifying the same format for subsequent transformations has no effect on performance.

{{ range resources.Match "*.jpg" }}
  {{ $image1600 := .Resize "1600x webp q85" }}
  {{ $image1200 := $image1600.Resize "1200x q85" }}
  {{ $image900 :=  $image1600.Resize "900x q85" }}
  {{ $image600 :=  $image1600.Resize "600x q85" }}
  {{ $image100 :=  $image1600.Resize "100x" }}
{{ end }}

44 seconds (35% reduction compared to Test 1)

Test 4 - Use result of previous transformation for current transformation

{{ range resources.Match "*.jpg" }}
  {{ $image1600 := .Resize "1600x webp q85" }}
  {{ $image1200 := $image1600.Resize "1200x q85" }}
  {{ $image900 :=  $image1200.Resize "900x q85" }}
  {{ $image600 :=  $image900.Resize "600x q85" }}
  {{ $image100 :=  $image600.Resize "100x" }}
{{ end }}

36 seconds (47% reduction compared to Test 1)

Test 5 - Same as Test 4 with q=75 (default) for all images

{{ range resources.Match "*.jpg" }}
  {{ $image1600 := .Resize "1600x webp" }}
  {{ $image1200 := $image1600.Resize "1200x" }}
  {{ $image900 :=  $image1200.Resize "900x" }}
  {{ $image600 :=  $image900.Resize "600x" }}
  {{ $image100 :=  $image600.Resize "100x" }}
{{ end }}

34 seconds (50% reduction compared to Test 1)

Test 6 - Same as Test 4 with q=70 for all images

{{ range resources.Match "*.jpg" }}
  {{ $image1600 := .Resize "1600x webp q70" }}
  {{ $image1200 := $image1600.Resize "1200x q70" }}
  {{ $image900 :=  $image1200.Resize "900x q70" }}
  {{ $image600 :=  $image900.Resize "600x q70" I 
  {{ $image100 :=  $image600.Resize "100x q70" }}
{{ end }}

34 seconds (50% reduction compared to Test 1)

Test 7 - Same as Test 4 with q=50 for all images

I wouldn’t do this in most cases. I think the risk of poor image quality is too high.

{{ range resources.Match "*.jpg" }}
  {{ $image1600 := .Resize "1600x webp q50" }}
  {{ $image1200 := $image1600.Resize "1200x q50" }}
  {{ $image900 :=  $image1200.Resize "900x q50" }}
  {{ $image600 :=  $image900.Resize "600x q50" }}
  {{ $image100 :=  $image600.Resize "100x q50" }}
{{ end }}

30 seconds (56% reduction compared to Test 1)

system · May 14, 2021, 4:49pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Performance, hugo 3.2k products, 8k images	17	1306	October 10, 2020
Question about Hugo's image processing and build time vs Gatsby Image component support	4	379	April 2, 2022
I need help with diagnosing slow builds support	12	476	December 10, 2022
Webp Image Processing support	3	4547	December 15, 2019
Huge building time when deploying site support	6	330	April 13, 2023