Hugo build takes too long

We are running a website using Hugo. It has taken more than 2 hours to complete the build process. Please check the status of our build process. Is it a reasonable time or are we something missing in the build process? how can we improve this process?
Pages | 61675

Paginator pages | 13427

Non-page files | 7819

Static files | 6530

Processed images | 0

Aliases | 26973

Sitemaps | 1

Cleaned | 0

Total in 7092827 ms

Please read Request Help for tips how to format your code and what information to add.

According to your output it took 7 minutes, not 2 hours. You might have a hardware issue that is blocking file system access while running Hugo.

With that many pages to render you should run the following command, to receive more information about the “cacheability” of your partials and layouts:

hugo server --i18n-warnings --templateMetrics --templateMetricsHints

This will help to find a solution to your problem. But it could be that the amount of time (if it is 7 minutes) is normal with 60k+ pages. The 2 hours point to another problem not within the reach of Hugo.

/pko

1 Like

7092827 ms means 118.21 Mins

We have 61K pages and which took around 2 hours time

Please check my file specific log details.
0 1h3m54.744369495s 180.060307ms 483.301532ms 21297 _default/single.html
0 59m14.958716718s 1.816534857s 2.458608566s 1957 taxonomy/collections.html
0 27m58.599412005s 779.65602ms 1.696144656s 2153 _default/brief_popular.html
0 27m58.307573206s 779.52047ms 1.673215399s 2153 _default/brief_latest.html
0 22m50.761190367s 1.639666495s 2.23897575s 836 taxonomy/topics.html
0 7m19.385390416s 145.588267ms 656.350739ms 3018 section/recent-update.html
0 5m58.275244457s 527.651317ms 810.354323ms 679 _default/regulated-single.html
0 5m26.402692121s 45.151845ms 431.753066ms 7229 _default/list.html
0 2m51.199669481s 300.350297ms 738.388269ms 570 taxonomy/geographyzipcodes.html
0 2m23.464397712s 746.238µs 274.981373ms 192250 _default/li.html
0 2m19.948976715s 90.289662ms 306.431117ms 1550 events/single.html
0 1m21.230996158s 41.317902ms 243.599272ms 1966 news/single.html
6 1m15.047343589s 3.523845ms 44.492283ms 21297 partials/offering-cadd-questions.html
0 1m4.599510132s 244.695114ms 686.160219ms 264 communities/medigy-chronic-care-management-ccm/single.html

Is it anything wrong in our HTML pages? Still I couldn’t figure out the reason for this issue.

Please read Requesting Help .

1 Like

If my math is correct, you’re creating about 103,000 pages at around 68ms per page. That’s not terrible, but I have a feeling that can be reduced. However, without seeing the site source, it’s difficult to provide concrete answers.

My main recommendation would be to dig into your layouts and move cacheable complexity into cached partials. Note that you can have cached variants, too. If it’s possible, reduce the number of taxonomies.

Attack these templates first since they appear to be where most of the cumulative time is spent:

If you want any more help than that, we’d need to see the site source or at least the theme.

1 Like

I’ve just run my company’s internal Hugo build on commodity hardware (4 cores), and I get about 1000 pages per second, i.e. a 3966 page site took “4841ms” - so 60,000 pages assuming linear scaling, would still only take a little over a minute.

Two things that can make your build slow: firstly templates that aren’t cached, and secondly, templates that cannot be cached because they ‘walk the file system’ to find parameters that are perhaps defined in the front matter of the page, or in a section/_index.md page.

We do the latter, i.e. walk the filesystem for parameters in _index.md files, and removing that makes our build even faster, taking about a quarter of the time I mentioned above (i.e. 1s vs 4s).

2 Likes

My sincere apology for the late reply. After several error and trial methods as per the instructions that are mentioned by @moorereason @barryl, finally we optimized the build process and reduced the build time to 4.62 minutes for process around 90K pages. Thanks for the extreme help and support @moorereason @barryl @gethugothemes . We are still optimizing the layouts.

3 Likes

Which is about 3ms per page–a 95% reduction! Nice work.

If you ever have time to explain the specific changes you made, I’m sure other users would be interested in learning from your experience.

3 Likes

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.