Resources for hugo to build big sites

support
1

Hi
I need some consultation with building a big site.
Hugo v0.74.2
Server - debian 8, 12 X Intel® Xeon® CPU E5-2650 v4 @ 2.20GHz, 20124 MiB RAM

I want to build a site with about a million posts.
I made 1 millon test posts with only title, 4 tags, 4 categories and 3 paragraphs of text.
I taken hugo-lamp theme without any changes.
But Hugo did not build content.
I saw a message: Building sites … Child process is finished, exiting… and hugo stops. No generated content.

But if I reduse quantity of posts to 150000, hugo works slowly but successfully.

I want to know can Hugo work with so many pages in general.
Or Hugo needs more resources then I have on my server.
How much resources need Hugo to build a million pages?

Thank you very much.

2 Likes
2

You may want to read this.

2 Likes
3

thank you very much, but unfortunately it is not for me
This script only generate content and do not help to build site.
I already have my generated content. Hugo can’t build html site.
I need something, to build site without overloading memory.
Render to disk option do not help, PartialCached too.
If my server is out of memory I want to know how much does it need to build 1 milloin pages.

4

I think we can do a bit of math here. 20Gig of memory on your server can successfully do 150k pages so I would assume a 1million would require 130.3G of memory. Once again this was just my guess and I don’t know how GoLang or Hugo works.

5

You should watch out for https://github.com/gohugoio/hugo/pull/7426 – I will follow up with the .Content part after that. The problem is, I get distracted by … life and work.

I will not promise “millions”, but the main issue I see today with lots of data is how Hugo handles memory. With the above we should get to a place where “stuff” can be shifted out of memory when needed.