Can't build website with 16GB of RAM (and it crashes my system)

[Jibec]

I did a thread on the same website with different focus: Hint on how to make website generation faster?
Since this thread, I updated to Hugo 0.153.0 and removed RSS and it reduced the RAM consumption. I chose to mark the subject as resolved, since it is were “how to reduce ram consumption”.

Here is the full website content (I removed static files to save 3GB of space): https://jibecfed.fedorapeople.org/partage/2026-01-20-website.7z

I’m using Linux to generate this, with the hugo release from github.

hugo v0.153.0-b4128babb839feaa962c00e777f0d2fe8d811f7d linux/amd64 BuildDate=2025-12-19T10:14:01Z VendorInfo=gohugoio

When I try to build the website, the RAM goes beyond the RAM limit hugo is supposed to respect and after a while, my system get stuck because reaching 100% of ram usage.

There is 83000 files, most of these contains big tables generated as markdown files.
Most pages looks like that: fr-French (Français) - translation progress for f43 - Fedora localization statistics

What would be the way for me to reduce RAM consumption by using better template structure?
I tried to remove the territories pages that contains lists of pages related to this territory, but it had no significant impact: FR France - Fedora localization statistics

[Jibec]

this might answer my question: Hugo sites over 100,000+ pages (issues, remedies)? - #2 by jmooring (using segments)

[Jibec]

I have try with segments, but I may have misunderstood how it works

If I add this:

[segments]
  [segments.f00]
    [[segments.f00.includes]]
      kind = '{home,term,taxonomy}'
    [[segments.f00.includes]]
      path = '{/f7,/f7/**,/f8,/f8/**,/f9,/f9/**}'
  [segments.f10]
    [[segments.f10.includes]]
      kind = '{home,term,taxonomy}'
    [[segments.f10.includes]]
      path = '{/f1*,/f1*/**}'
  [segments.f20]
    [[segments.f20.includes]]
      kind = '{home,term,taxonomy}'
    [[segments.f20.includes]]
      path = '{/f2*,/f2*/**}'
  [segments.f30]
    [[segments.f30.includes]]
      kind = '{home,term,taxonomy}'
    [[segments.f30.includes]]
      path = '{/f3*,/f3*/**}'
  [segments.f40]
    [[segments.f40.includes]]
      kind = '{home,term,taxonomy}'
    [[segments.f40.includes]]
      path = '{/f4*,/f4*/**}'
  [segments.territories]
    [[segments.territories.includes]]
      kind = '{home,term,taxonomy}'
    [[segments.territories.includes]]
      path = '{/territories,/territories/**}'
  [segments.other]
    [[segments.other.includes]]
      kind = '{home,term,taxonomy}'
    [[segments.other.excludes]]
      path = '{/f*,/f*/**,/territories,/territories/**}'

I’m able to now laucnh ../hugo --logLevel info --renderSegments f10 then same with f10 etc.
Then the other catch other pages I may have forgotten.
I assume a loop in a bash file will do the work.

Still, the whole is still really slow and generate every time all the static files even if not in path.
For example, other generates one page and takes about 2 minutes and 95% of my RAM:

$ hugo --logLevel info --renderSegments other
Start building sites … 
hugo v0.153.0-b4128babb839feaa962c00e777f0d2fe8d811f7d linux/amd64 BuildDate=2025-12-19T10:14:01Z VendorInfo=gohugoio

INFO  build:  step process substep collect files 83082 files_total 83082 pagesources_total 83082 resourcesources_total 0 duration 685.382157ms
INFO  build:  step process duration 685.438596ms
INFO  dynacache: adjusted partitions' max size evicted 3661 numGC 27 limit 3.87 GB alloc 3.93 GB totalAlloc 8.61 GB
INFO  dynacache: adjusted partitions' max size evicted 3660 numGC 27 limit 3.87 GB alloc 4.84 GB totalAlloc 9.52 GB
INFO  dynacache: adjusted partitions' max size evicted 3661 numGC 28 limit 3.87 GB alloc 4.23 GB totalAlloc 10.67 GB
INFO  dynacache: adjusted partitions' max size evicted 1372 numGC 28 limit 3.87 GB alloc 4.58 GB totalAlloc 11.02 GB
INFO  build:  step assemble duration 40.977521744s
INFO  build:  step render substep pages site en outputFormat html duration 142.588719ms
INFO  build:  step render pages 1 content 1 duration 180.440897ms
INFO  build:  step render deferred count 0 duration 464ns
INFO  build:  step postProcess duration 4.975µs
INFO  build:  duration 41.843593454s
INFO  static: syncing static files to / duration 1m51.194642629s

                  │  EN   
──────────────────┼───────
 Pages            │     1 
 Paginator pages  │     0 
 Non-page files   │     0 
 Static files     │ 12313 
 Processed images │     0 
 Aliases          │     0 
 Cleaned          │     0 

Total in 111212 ms

any hint on how to better use Hugo for my usecase?

[irkode]

I would would move the static images aside and see the performance without.

guess you already checked out performance stuff in the docs and played with template metrics

• I had analyzed a big project with lots large and looping templates. just using {{- instead of {{ for the templates resulted in much less memory usage and a 10% speed time.

use the template metrics to find your time stoppers:

• Merging of maps is quite expensive check for places where you loop and merge a lot
• using partialCached for some of the calculations may be good.

regarding segments: a segment has access to the whole site!, If you have a page, that reads some data from all the other pages … well I guess segments won’t help that much

anything you can share to play around?

[Jibec]

The whole website is published there: https://jibecfed.fedorapeople.org/partage/2026-01-20-website.7z
Is this what you mean by sharing something to play around?

I created even smaller segments that the example, I process f7, then f8, etc (see content folder), and it works, but still is quiet slow, like 150 seconds for 1500 pages, but at least I can publish all I want.

The templates are quite raw, there is almost no go code inside it (I did not learn how to do that yet)

Yes, I did play with template metrics if you think about caching, the results there still apply: Hint on how to make website generation faster?

[jmooring]

When I build your site as-is, the build runs for over 30 minutes then dies. The memory (16 GB) and swap space (8 GB) are full.

Looking at your site, you are calling the ref shortcode over 5 million times.

When I replace those calls with foo, so that the markdown links looks like [am](foo), the build completes in less than 2 minutes. Only 60% of the memory is used, and none of the swap is used.

$ hugo --templateMetrics --templateMetricsHints

Template Metrics:

       cumulative       average       maximum      cache  percent  cached  total  
         duration      duration      duration  potential   cached   count  count  template
       ----------      --------      --------  ---------  -------  ------  -----  --------
  7m56.109887758s    5.756167ms  811.715847ms          0        0       0  82713  single.html
  3m11.568872623s    1.108628ms  387.742949ms          0        0       0  172798  _markup/render-table.html.html
    34.014866673s     409.393µs    64.59626ms         64        0       0  83086  _partials/head.html
    10.277516008s       123.7µs   16.358618ms         52        0       0  83084  _partials/breadcrumb.html
     3.141710922s   84.911106ms  161.417771ms          0        0       0     37  list_packages.html
     2.999061345s   11.579387ms  182.465235ms          0        0       0    259  territories/term.html
     2.333754955s  2.333754955s  2.333754955s          0        0       0      1  sitemap.xml
     1.914039378s      23.036µs   30.594304ms        100      100   83085  83086  _partials/footer.html
     1.537852455s      18.509µs   82.162132ms        100      100   83085  83086  _partials/nav.html
     1.391085035s      16.742µs    32.67831ms        100      100   83085  83086  _partials/head_custom.html
     1.069137993s      12.868µs   10.231549ms         58        0       0  83085  _partials/header.html
     851.267769ms  851.267769ms  851.267769ms          0        0       0      1  territories/list.html
     588.632298ms   15.908981ms   86.787748ms          0        0       0     37  list_languages.html
     353.553363ms      28.963µs    24.40429ms          0        0       0  12207  _shortcodes/resource.html
     146.251169ms    3.952734ms   82.499874ms          0        0       0     37  release.html
     103.072595ms  103.072595ms  103.072595ms          0        0       0      1  index.html
      81.697551ms   81.697551ms   81.697551ms          0        0       0      1  404.html
            660ns         660ns         660ns        100        0       0      1  _partials/footer_custom.html


                  │  EN   
──────────────────┼───────
 Pages            │ 83087 
 Paginator pages  │     0 
 Non-page files   │     0 
 Static files     │ 11278 
 Processed images │     0 
 Aliases          │     0 
 Cleaned          │     0 

Total in 97446 ms

Of the templates above, the only one I looked at was _partials/breadcrumb.html. This partial is inefficient due to recursion. Use the Ancestors method instead, though I doubt it will make a material difference.

You have 172 K tables; not sure what can be done about that.

---

Also note that I closed my IDE (VS Code) before building the site from a terminal; I didn’t want its file watcher(s) to affect metrics.

[irkode]

not as impressive as the ref finding

Ryzen 16core 64GB RAM on Windows with Virsu Scanner Running
Total in 453371 ms
max around 25GB taken

with all {{ }} replaced with {{- -}} may break final layout NL/space

max around 20GB taken
Total in 367294 ms

[jmooring]

If you really, really want all those links to be absolute instead of relative:

1. Replace all the ref shortcode calls with their argument. For example, change this:

   [ca]({{< ref "/f8/language/ca.md" >}})

   To this:

   [ca](/f8/language/ca)

   Using this command ^[1]:

   find ./content -type f -name "*.md" -print0 | xargs -0 sed -i -E 's/\{\{<\s+ref\s+"(.*?)\.md"\s+>\}\}/\1/g'

2. Create a Markdown render hook based on our embedded code, maybe adding a console warning in the case of broken links.

   layouts/_markup/render-link.html

   {{- $u := urls.Parse .Destination -}}
   {{- $href := $u.String -}}
   {{- if strings.HasPrefix $u.String "#" -}}
     {{- $href = printf "%s#%s" .PageInner.RelPermalink $u.Fragment -}}
   {{- else if and $href (not $u.IsAbs) -}}
     {{- $path := strings.TrimPrefix "./" $u.Path -}}
     {{- with or
       ($.PageInner.GetPage $path)
       ($.PageInner.Resources.Get $path)
       (resources.Get $path)
     -}}
       {{- $href = .Permalink -}}
       {{- with $u.RawQuery -}}
         {{- $href = printf "%s?%s" $href . -}}
       {{- end -}}
       {{- with $u.Fragment -}}
         {{- $href = printf "%s#%s" $href . -}}
       {{- end -}}
     {{- else -}}
       {{- warnf "The link render hook was unable to resolve the destination %q in %q" $u.String .Page.String -}}
     {{- end -}}
   {{- end -}}
   <a href="{{ $href }}" {{- with .Title }} title="{{ . }}" {{- end }}>{{ .Text }}</a>
   {{- /**/ -}}
   

With that change, and optimizing the breadcrumbs partial (which made no difference), the build looks like this:

build metrics

$ hugo --templateMetrics --templateMetricsHints

Template Metrics:

       cumulative       average       maximum      cache  percent  cached  total  
         duration      duration      duration  potential   cached   count  count  template
       ----------      --------      --------  ---------  -------  ------  -----  --------
  15m30.303356201s   11.247365ms  943.183506ms          0        0       0  82713  single.html
  6m35.910824472s      54.224µs  222.157728ms          0        0       0  7301292  _markup/render-link.html
  3m55.527401082s    1.363021ms  199.158045ms          0        0       0  172798  _markup/render-table.html.html
    31.002220155s     373.134µs   95.864662ms         64        0       0  83086  _partials/head.html
     8.684977475s     104.532µs   42.148201ms         46        0       0  83084  _partials/breadcrumb.html
     5.275340499s   88.522716ms  206.922177ms          0        0       0     37  list_packages.html
     3.166462524s  3.166462524s  3.166462524s          0        0       0      1  sitemap.xml
     3.206397809s      26.555µs    95.34692ms        100      100   83085  83086  _partials/head_custom.html
      1.85293435s      22.301µs   87.734053ms        100      100   83085  83086  _partials/nav.html
     1.824623445s       21.96µs   55.054365ms         58        0       0  83085  _partials/header.html
     1.817006076s      21.868µs   40.553146ms        100      100   83085  83086  _partials/footer.html
     1.614301945s    6.232826ms   80.193272ms          0        0       0    259  territories/term.html
     664.509034ms      54.436µs   55.937378ms          0        0       0  12207  _shortcodes/resource.html
     619.802434ms   16.751417ms   91.602804ms          0        0       0     37  list_languages.html
      473.29053ms   473.29053ms   473.29053ms          0        0       0      1  territories/list.html
     178.507025ms    4.824514ms   87.376752ms          0        0       0     37  release.html
     107.693455ms  107.693455ms  107.693455ms          0        0       0      1  index.html
       85.86656ms    85.86656ms    85.86656ms          0        0       0      1  404.html
          1.039µs       1.039µs       1.039µs        100        0       0      1  _partials/footer_custom.html


                  │  EN   
──────────────────┼───────
 Pages            │ 83087 
 Paginator pages  │     0 
 Non-page files   │     1 
 Static files     │ 11278 
 Processed images │     0 
 Aliases          │     0 
 Cleaned          │     0 

Total in 161987 ms

Using the link render hook isn’t free (there are 7.3 million calls), but the site still builds in less than three minutes. Memory usage was about 60% of 16GB with zero swap.

Building the site without the template metric flags doesn’t make much difference.

Here’s the modified site:

git clone --single-branch -b hugo-forum-topic-56591 https://github.com/jmooring/hugo-testing hugo-forum-topic-56591
cd hugo-forum-topic-56591
hugo server

It includes an intentionally broken link to demonstrate the warning message emitted in the case of broken links.

---

1. Or in your case I guess that means changing your content generation scripts. ↩︎

[jmooring]

Finally, from the documentation for the ref shortcode:

https://gohugo.io/shortcodes/ref/

[Jibec]

Thanks for your help, yes I were changing my code: Fied version. A great.

I’m afraid I don’t understand why to create a hook since it works as is (I’m not familiar with hugo templates, this hook contains lot of complexity I don’t understand)

thanks a lot for your help

[jmooring]

@Jibec

I’m afraid I don’t understand why to create a hook

I prefaced my remarks with:

@jmooring

If you really, really want all those links to be absolute instead of relative:

In your case, if you don’t need absolute links, don’t use a hook.