How fast is site/images generation supposed to be? Which image size I'm supposed to use?

I’m not even sure whether it’s an issue or not, but assuming Hugo claims to be super-fast, I’m now a bit curious because… my blog draft generation takes at least 2 minutes and at least few times more than that with responsive images generation (Responsive and optimized images with Hugo | BryceWray.com)

But even leaving responsive images aside (wrap around with {{ if not hugo.IsDevelopment }}):

  1. Generate site without the post. Total in 24404 ms → seems a bit slow, but might be fine for one-time job
                   | RU  | EN
-------------------+-----+-----
  Pages            | 122 | 41
  Paginator pages  |  11 |  0
  Non-page files   |  52 | 23
  Static files     |  17 | 17
  Processed images | 185 |  0
  Aliases          |  41 | 16
  Cleaned          |   0 |  0
  1. Add the post without any images in it, run hugo build again (on top of previously created dir) → Total in 1051 ms, seems ok
  2. Add photos. I have 91 image file for the post, totaling 412 Mb, so average 4Mb per photo. → and now watch the magic…
INFO  build:  step render substep pages site ru outputFormat html duration 1m44.5801053s
INFO  build:  step render substep pages site ru outputFormat json duration 155.1245ms
INFO  build:  step render substep pages site ru outputFormat rss duration 25.0697ms
INFO  build:  step render substep pages site en outputFormat html duration 99.3612ms
INFO  build:  step render substep pages site en outputFormat json duration 138.8415ms
INFO  build:  step render substep pages site en outputFormat rss duration 2.9989ms
INFO  build:  step render pages 173 content 120 duration 1m45.0075453s
INFO  build:  step render deferred count 0 duration 0s
INFO  build:  step postProcess duration 0s
INFO  build:  duration 1m45.2375945s

                   | RU  | EN
-------------------+-----+-----
  Pages            | 123 | 41
  Paginator pages  |  11 |  0
  Non-page files   | 147 | 23
  Static files     |  17 | 17
  Processed images | 457 |  0
  Aliases          |  41 | 16
  Cleaned          |   0 |  0

Total in 105280 ms

Almost 2 minutes. For just one post. For the sake of statistics, if I remove both public and resources/_gen and run the full generation again, it now takes… 125 seconds. Which is in line with what I saw earlier - basically single the post adds 1.5 minutes of pure generation time.

And for the curious reader - if I now enable responsive images generation and run the build again, then (even without clearing build directories) hugo build first timed out with 10 minutes and then finished on the second run with 16 seconds. So, 10 minutes basically for almost a single post.

My img shortcode (which I use for every image and what I believe takes the most time) does a few things:

  • Finds image using $.Page.Resources.GetMatch
  • Renames it to some human-readable name using resources.Copy $newName
  • Resizes once using $resource.Resize "600x"
  • Gets md5 hash of the path.
  • Extracts EXIF tags using with $resource.Exif
  • Resizes again using $imgRsc.Process (print "resize 640x jpg box") (yes-yes this is probably redundant, some artifact left after merging my initial img with responsive generation code from the linked page above, but if I enable responsive image generation, I’ll have 6 more resizes anyway, so it shouldn’t significantly matter)

That’s all of potentially resourse-consuming code I see in my img.

I think in the first place it’s myself doing something wrong. Am I not supposed to use original images in my page resources? Is there some kind of recommendation which size should the images be to be processed faster by Hugo?
But in the second place, I honestly don’t think 100 images is a lot. I’m running it on my Windows full tower PC, on SSD (can provide full specs if needed), why is single post (I’m going to have at least a few dozens of similar sized posts) generation takes minutes on full PC? I have my mini-blog with 20 posts, each is some long trip/travel report. This particular post was composed after 4-days cycling trip in Europe, and I haven’t even migrated to Hugo my posts about 2-3 weeks vacations (which would easily have 10-15+ photos per day → 200 photos at least).

The images I put to Hugo are (mostly) JPGs generated from RAW via Lightroom and resized during export to 1620 width and ~1Mb, but for some posts (like the one in subj) also JPGs from my phone camera (Pixel 8, 4000*3000 res jpg as is, 3-11 Mb)

In case you haven’t already done so, this is a good place to start:
https://gohugo.io/troubleshooting/performance/

Total in 123300 ms:

Template Metrics:

       cumulative       average       maximum      cache  percent  cached  total
         duration      duration      duration  potential   cached   count  count  template
       ----------      --------      --------  ---------  -------  ------  -----  --------
    4m24.6586274s  4.200930593s  1m27.4578163s          0        0       0     63  _default/list.html
    4m12.7627076s  2.605801109s  1m22.8932426s          7        0       0     97  partials/block/desc.html
    2m22.7020822s  445.944006ms    1.7559871s          0        0       0    320  shortcodes/img.html
     2m5.7024049s  7.394259111s   55.7983278s          0        0       0     17  post/single.html
     2m0.3325811s  1.191411694s   55.7313273s         22        0       0    101  partials/head.html
      57.6991482s  571.278695ms   33.1792248s         93        0       0    101  partials/head-imgs-css.html
      16.4748844s  140.810977ms    4.5076221s         59        0       0    117  partials/block/image.html
       7.5125036s   751.25036ms    1.9350558s          0        0       0     10  search/search.json

If I read this correcty, then each single img takes half second in average => so ~50 seconds for my ~100-images post. and the longest post generation is 1.5 minutes (partials/block/desc.html is basically content of the post)

Not sure it helps me much though, I kind of already knew that that’s images what takes longest time :slight_smile: It just confirms it.

Tell me about virus scanning.

Seems off?


image

Something’s doing something. Please follow these directions and test again.

27th is few days ago… Anyway, I found the exclusions list (in a different place than indicated, that’s why I was confused), added both hugo.exe as process, as file, and the whole folder with my content, restarted the clean build - no significant changes, Total in 136362 ms

When running generation, if I check Task manager, hugo.exe takes 30-40% of CPU, ~150 MB of RAM and is the most CPU consuming process there (Intel Core i5-6500)

At this point, without access to your site, I’m not able to provide any guidance.

While I don’t mind sharing, I’m not sure it’s the right use of your time to ask you to debug my actual site =) I can try to prepare some simpler one-pager reproducer if you think this would help.

BTW I’ve also tried to do the same on my remote Linux VPS (where I’m planning to deploy the result eventually). But it’s quite the same there if not worse (just run the full generation without responsive images from scratch):

First executed not-extended version by accident, crashed on lack of webp resize, but still took Total in 143708 ms


Template Metrics:

       cumulative       average       maximum      cache  percent  cached  total
         duration      duration      duration  potential   cached   count  count  template
       ----------      --------      --------  ---------  -------  ------  -----  --------
   4m9.035668919s  16.602377927s  2m20.164740732s          0        0       0     15  post/single.html
   4m9.005136624s  3.411029268s  2m20.164229787s         20        0       0     73  partials/head.html
    4m3.35295258s  1.776298923s  3.481745758s          0        0       0    137  shortcodes/img.html
    24.591207169s  571.888538ms  3.079603681s          0        0       0     43  _default/list.html
    24.070267544s  359.257724ms  3.076168917s         98        0       0     67  partials/block/image.html
     5.362495453s   73.458841ms  1.572931371s        100        0       0     73  partials/head-imgs-css.html

Then I’ve replaced to extended version and run it on top of what was there already (so I assume only webp generation was missing), Total in 73894 ms:


Template Metrics:

       cumulative       average       maximum      cache  percent  cached  total
         duration      duration      duration  potential   cached   count  count  template
       ----------      --------      --------  ---------  -------  ------  -----  --------
  1m33.690706236s  5.511218013s  56.937182159s          0        0       0     17  post/single.html
   1m31.95802153s   910.47546ms  56.923224203s         39        0       0    101  partials/head.html
  1m31.409382743s  905.043393ms  56.911893451s         93        0       0    101  partials/head-imgs-css.html
    33.347018121s  529.317747ms  10.792197286s          0        0       0     63  _default/list.html
    24.975569897s  213.466409ms  5.786411997s         59        0       0    117  partials/block/image.html
    13.780962694s  1.378096269s  3.508618135s          0        0       0     10  search/search.json
     9.046601238s    93.26393ms  2.875632479s          7        0       0     97  partials/block/desc.html
     6.787362817s   21.210508ms  735.159249ms          0        0       0    320  shortcodes/img.html

But what about the initial questions? How fast is images resize is supposed to be (how fast is it for you, if you create a page with 100 4Mb JPG images and do resource.Resize a couple of times each)? What is the common/recommended practice there? Am I really not doing anything suspicious?

Not to put too sharp an edge on it, but I’m not going to spend any time on this without access to your site.

1 Like
  1. Image processing is relatively slow.
  2. Reading from file cache is fast, so try to make sure you do that whenever possible.
  3. Having large 4MB originals as a starting point doesn’t (to me anyhow) make much sense if your target is the web. Create a preset in LIghtRoom that exports sensible sized JPEGS.

Also, you should work on your communication skills. Your tone does not invite people to help you.

3 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.