Huge building time when deploying site

Hello,

This is the build time on my computer:

                   | EN
-------------------+------
  Pages            |  20
  Paginator pages  |   0
  Non-page files   |  24
  Static files     |   6
  Processed images | 276
  Aliases          |   0
  Sitemaps         |   1
  Cleaned          |   0

Total in 407 ms

And this is the build time on netlify:

9:18:40 AM:                    | EN
9:18:40 AM: -------------------+------
9:18:40 AM:   Pages            |  20
9:18:40 AM:   Paginator pages  |   0
9:18:40 AM:   Non-page files   |  24
9:18:40 AM:   Static files     |   6
9:18:40 AM:   Processed images | 276
9:18:40 AM:   Aliases          |   0
9:18:40 AM:   Sitemaps         |   1
9:18:40 AM:   Cleaned          |   0
9:18:40 AM: Total in 119155 ms
9:18:40 AM: ​
9:18:40 AM: (build.command completed in 1m 59.2s)

I imagine that this is because of the images. But even so, the difference is huge, and I will have really more images at the end …

Accessory question, each time I do a small modification, netlify rebuild everything and reprocess every image. Is there a way to avoid this ?

Thank you very much for you help.

Hi,

I’m using the following command in netlify.toml to get more debug info:

[build]
publish = "public"
command = "hugo --cleanDestinationDir --gc  --templateMetrics --templateMetricsHints --printMemoryUsage"

Maybe you can get more insight this way?

Accessory question, each time I do a small modification, netlify rebuild everything and reprocess every image. Is there a way to avoid this ?

I’m not familiar with “cache handling”, but it’s a keyword you may want to search for

Thank you very much for your help, it’s much clearer now:

10:10:10 AM:        duration      duration      duration  potential   cached   count  count  template
10:10:10 AM:      ----------      --------      --------  ---------  -------  ------  -----  --------
10:10:10 AM:   14m13.642281298s  1m17.603843754s  2m1.951258888s         18        0       0     11  partials/menu.html
10:10:10 AM:   8m7.957885765s  1m21.326314294s  2m1.986966782s          0        0       0      6  _default/single.html
10:10:10 AM:   6m6.00923374s  1m1.001538956s  2m1.985234725s          0        0       0      6  _default/list.html
10:10:10 AM:   2m1.918460851s  5.300802645s  9.919021404s          0        0       0     23  _default/_markup/render-image.html
10:10:10 AM:   2m1.916552295s  5.300719665s  9.918894324s         44        0       0     23  partials/lazyimg.html
10:10:10 AM:   2m1.828543249s  2.648446592s  7.548087747s        100        0       0     46  partials/resizer/auto.html
10:10:10 AM:   45.363559353s  986.164333ms  2.304727241s        100        0       0     46  partials/resizer/simple.html
10:10:10 AM:    340.509829ms   26.193063ms   35.754715ms         66        0       0     13  partials/meta.html
10:10:10 AM:     79.993559ms     3.47798ms   27.461028ms         44        0       0     23  partials/renderer/lqip-webp.html
10:10:10 AM:     53.495498ms    2.325891ms   26.475001ms         36        0       0     23  partials/renderer/lqip.html
10:10:10 AM:     46.124739ms    2.005423ms   26.118792ms         46        0       0     23  partials/renderer/native.html
10:10:10 AM:     38.339011ms     555.637µs    1.436138ms         50        0       0     69  partials/renderer/_srcset.html
10:10:10 AM:     30.547292ms   30.547292ms   30.547292ms          0        0       0      1  index.html
10:10:10 AM:      4.180786ms     597.255µs     934.852µs          0        0       0      7  _internal/_default/rss.xml
10:10:10 AM:      4.070939ms     313.149µs    1.908594ms         46        0       0     13  partials/breadcrumbs.html
10:10:10 AM:      4.068722ms      46.235µs     708.098µs          4        0       0     88  partials/doc-titles.html
10:10:10 AM:      1.853151ms      142.55µs     336.228µs        100        0       0     13  partials/lazyimg-setup-nojs.html
10:10:10 AM:      1.497342ms      115.18µs     287.995µs        100        0       0     13  partials/lazyimg-setup.html
10:10:10 AM:      1.284443ms    1.284443ms    1.284443ms          0        0       0      1  _default/_markup/render-codeblock-goat.html
10:10:10 AM:        794.71µs      794.71µs      794.71µs          0        0       0      1  shortcodes/festivals_spots.html
10:10:10 AM:       398.945µs     398.945µs     398.945µs          0        0       0      1  _internal/_default/sitemap.xml
10:10:10 AM:       345.147µs      26.549µs     142.107µs        100        0       0     13  partials/footer.html
10:10:10 AM:        15.596µs      15.596µs      15.596µs          0        0       0      1  robots.txt

If I read it correctly, it’s the rendering time of my menu which is very long. I have a recursive fonction that build the menu depending of the folder structure. I’ve borrowed it from the hugo book theme with some light modifications.

It works fine on my computer most of the times - I had once or two a build failed, with no apparent reasons.

This is the code of partials/menu.html:

<!--
https://github.com/alex-shpak/hugo-book/blob/master/layouts/partials/docs/menu-filetree.html
-->
{{ $section := .Section }}
<h3>{{ with .Site.GetPage "section" $section }}{{ .Title }}{{ end }}</h3>

{{ with .Site.GetPage $section }}
  {{ template "section-children" (dict "Section" . "CurrentPage" $) }}
{{ end }}

{{ define "section-children" }}{{/* (dict "Section" .Section "CurrentPage" .CurrentPage) */}}
  <ul>
    {{ range (where .Section.Pages "Params.pagehidden" "ne" true) }}
      {{ if .IsSection }}
        <li {{- if .Params.BookFlatSection }} class="book-section-flat" {{ end -}}>
          {{ template "page-link" (dict "Page" . "CurrentPage" $.CurrentPage) }}
          {{ template "section-children" (dict "Section" . "CurrentPage" $.CurrentPage) }}
        </li>
      {{ else if and .IsPage .Content }}
        <li>
          {{ template "page-link" (dict "Page" . "CurrentPage" $.CurrentPage) }}
        </li>
      {{ end }}
    {{ end }}
  </ul>
{{ end }}

{{ define "page-link" }}{{/* (dict "Page" .Page "CurrentPage" .CurrentPage) */}}
  {{ $current := eq .CurrentPage .Page }}
  {{ $ancestor := .Page.IsAncestor .CurrentPage }}

  {{ if .Page.Params.menuOff }}
    <span>
      {{- partial "doc-titles" .Page -}}
    </span>
  {{ else  }}
    {{ if $current }}
    <span class="active">{{ partial "doc-titles" .Page }}</span>
    {{ else }}
    <a href="{{ .Page.RelPermalink }}" class="{{ if $current }}active{{ end }}">
      {{- partial "doc-titles" .Page -}}
    </a>
    {{ end }}

  {{ end }}
{{ end }}

Perhaps a cached partial helps in this case, given that the menu doesn’t change on a per-page basis. Or does it?

I suspect that the metrics for rendering the section menu are misleading, and that the underlying cause is related to image processing.

Do this (locally) and post the build time:

rm -rf resources/ public/
hugo

Now do this (locally) and post the build time:

hugo

Additional information needed to diagnose this:

  • Hugo version (local)
  • Hugo version (remote)
  • Link to your project repository
1 Like

Yes.

This is the Hugo sites build on Netlify:

Note the number of images.

This is all about cache setup. This is how the Hugo docs config looks like:

[caches]
  [caches.getjson]
    dir    = ":cacheDir/:project"
    maxAge = -1
  [caches.getcsv]
    dir    = ":cacheDir/:project"
    maxAge = -1
  [caches.images]
    dir    = ":cacheDir/images"
    maxAge = "1440h"
  [caches.assets]
    dir    = ":resourceDir/_gen"
    maxAge = -1

When you put your image cache below :cacheDir it will work out of the box on Netlify. If the HUGO_CACHEDIR env var is not set, Hugo will detect that it’s running on Netlify and set it to a directory that will be restored on next build.

Thank you very much for the rapidity and clarity of your help!

jmooring, when I remove the resources folder, in fact the build time is 1 minute on my computer.

Thank you bep, I added the caches directives in my config.toml and indeed now it takes 375ms to build, yeahh !

About cache, in the hugo doc, it says to include the resource directory in source control. Is it an equivalent method ? Or the one you propose is better ?

Many thanks !