I’m in the process of upgrading from 0.119.x to 0.142.0 (finally). When running hugo I get build times similar to what I’ve been getting on 0.119, however when running hugo server, it’s taking around 16mins to build before the live server starts.
I unfortunately can’t share the repo as it’s private. I would attempt to reproduce in a demo, but I don’t really know where to start.
Are there areas to which I should pay particular attention that could be the culprit in this massive build time difference?
Run the server with --templateMetrics and --templateMetricsHints and identify the layout files that slow things down. In my experience it’s image processing (do not delete/ignore resources folder after one initial slow run) and pager for list pages (use higher limits for the displayed items per page).
Sorry for the off-topic question, but how many pages does your site have? 16 minutes seems too long to me, especially when Hugo is claimed to be the fastest SSG on the market.
Thanks, @davidsneighbour! I don’t think it’s a general layout performance issue since running a build without using the dev server (i.e. hugo) is completing in a reasonable, expected amount of time. It’s only when running the dev server that I get the unusually long build times. I’m also not getting the same disparate build times in 0.119.x.
You need to provide much better information about your site.
I’d be glad to, but I’m not really certain where to start. Potentially relevant parts of the puzzle that come to mind:
Stylesheets use Sass and toCSS (configured for Dart Sass) for transformation.
I’m using Tailwind 3.x via postCSS
My primary stylesheet loading looks like the below:
{{- $css := resources.Get "scss/main.scss" | toCSS (merge $toCSSOptions (dict "targetPath" "static-assets/css/main.css")) | postCSS $postCSSOptions -}}
{{- if not hugo.IsDevelopment -}}
{{ $css = $css | minify | fingerprint }}
{{- end -}}
{{ with $css }}
{{ if not hugo.IsDevelopment }}
<link href="{{ .RelPermalink }}" rel="stylesheet" data-integrity="{{ .Data.Integrity }}" />
{{ else }}
<link href="{{ .RelPermalink }}" rel="stylesheet" />
{{ end }}
{{ end }}
{{ with .CurrentSection.Resources.GetMatch "styles.section.{scss,css}" }}
{{- $sectionCSS := . | toCSS $toCSSOptions | postCSS $postCSSOptions -}}
{{- if not hugo.IsDevelopment -}}
{{ $sectionCSS = $sectionCSS | minify | fingerprint }}
{{- end -}}
{{ with $sectionCSS }}
{{ if not hugo.IsDevelopment }}
<link href="{{ .RelPermalink }}" rel="stylesheet" data-integrity="{{ .Data.Integrity }}" />
{{ else }}
<link href="{{ .RelPermalink }}" rel="stylesheet" />
{{ end }}
{{ end }}
{{ end }}
{{ with .Resources.GetMatch "styles.{scss,css}" }}
{{- $pageCSS := . | toCSS $toCSSOptions | postCSS $postCSSOptions -}}
{{- if not hugo.IsDevelopment -}}
{{ $pageCSS = $pageCSS | minify | fingerprint }}
{{- end -}}
{{ with $pageCSS }}
{{ if not hugo.IsDevelopment }}
<link href="{{ .RelPermalink }}" rel="stylesheet" data-integrity="{{ .Data.Integrity }}" />
{{ else }}
<link href="{{ .RelPermalink }}" rel="stylesheet" />
{{ end }}
{{ end }}
{{ end }}
Javascript is transpiled using js.Build (with minification for production builds). fingerprint is used for production builds as well (using the same conditional logic as the CSS above).
Below are the stats when completing a build (without the dev server):
Tried renaming the static folder, but the initial build for starting the dev server still took ~15mins.
The static folder is about 144.5MB. The largest file is about 27MB, second largest is 10.25MB, 23 files are between 1MB and 8MB, and the remaining ~5,000 files are under 1MB.
INFO dynacache: adjusted partitions' max size evicted 748 numGC 94 limit 4.00 GB alloc 4.15 GB totalAlloc 25.80 GB
Can you:
Check if you see simllar log lines when running hugo --logLevel info?
Can you try to set Hugo’s memory limit to something bigger and try again., e.g.export HUGO_MEMORYLIMIT=8 and try again (8 GB).
I don’t have a good explanation as to why this happens in the server and not when building, but there’s something that “eats” memory somewhere, and that makes the cache much less useful.
INFO build: step render substep pages site ro outputFormat html duration 3m6.198061375s
with 3.5 min for 83 pages seems at least unusual.
with template metrics you have
for example >100.000 calls to GetLink which sum up to a minute…
you have much more pages and files in EN site do you merge missing pages…
all that said:
I think that all (and things we do not know) play together to get such a result
imho that won’t work out here with some extracts from your site and log files… one who is willing to analyze and pin that down needs a full repo to do.
I’ve continued to do more testing. Have yet to determine anything conclusive, but three details stand out as being noteworthy:
I currently have 23 languages configured in my config/_default/languages.toml file. When reducing this to two, but not deleting any of the localized content files themselves, the build times for hugo server go back to relatively normal.
Using hvm, I tested with various versions to pinpoint when the behavior begins. I get pretty normal build times with hugo server for 0.140.1, but the really long builds for hugo server start with 0.140.2. I’ve tried clearing caches and switching back and forth several times to confirm, and it’s consistently reproducible that 0.140.1 is normal whereas 0.140.2 is not.
When comparing the output of running hugo server --templateMetrics --templateMetricsHints --logLevel debug using 0.142.0 against the output from 0.119.0, I’m seeing a massive difference in the number of times postcss: use config file ... appears. For 0.119.0 (and in fact 0.140.1) it’s <50 (here are the logs from 0.119.0); for 0.142.0 it’s 859 instances.
reminds me, there where more points in my post - machine stats, mem, …
I’m running an Apple M1 Pro with 16GB of RAM.
what about comparing the results of using hugo server and hugo with any config
I haven’t thoroughly tested this yet, but did a bit of testing here with no discernible differences.
I had, by accident, this set in my OS env on my MacBook for 3 months without thinking about it:
HUGO_MEMORYLIMIT=1
And I build some pretty big Hugo sites.
Which means max 1 GB. If Hugo uses more than that it will try to free memory by evicting things from cache. This allows e.g. building sites pulling in thousands of big JSON files via resources.Get "foo.json" | transform.Unmarshal without needing a massive amount of ram.
By default we set aside 1/4 of the system ram. In my case that is 32/4 => 8GB. In your case that is 16/4 => 4GB, which should be plenty.
But there are some very unusual stuff going on that is impossible for us to debug further without access to the full source code.
That said, I would strongly recommend you start by upgrading to TailwindCSS v4 and get rid of PostCSS in your setup.
None of those links makes much sense to me, but you’re probly right in that this is a rebuild-loop that comes from hugo_stats.json file changing.
Again, I don’t know what the original project is configured, but it’s worth testing if remove any watch/mount of the hugo_stats.json file makes the server start normally.