This really isn’t a “tips & tricks” topic, but rather some notes about the content adapter feature introduced in v0.126.0.
Here’s an example site to explore capabilities:
git clone --single-branch -b hugo-github-issue-12440 https://github.com/jmooring/hugo-testing hugo-github-issue-12440
cd hugo-github-issue-12440
hugo server
This site uses content adapters to create pages from remote data sources.
On my very average laptop, with a terrible network connection, the initial build is 20 seconds, or about 0.001 seconds/page. Of that, approximately 9 seconds is spent downloading the 49MB remote data file. Subsequent builds are 11 seconds (the data file is cached), or about 0.0006 seconds/page.
Note that the “posts” section is paginated with 10 pages per pager, creating 2000 pagers in addition to the 20,000 posts. So the number is closer to 0.0005 seconds per page.
While this example site creates pages from remote data, you can also create pages from local data files and global resources (e.g., create a page for each image in the assets directory).
One benefit of local files (e.g. in assets) is that it works better for edits when running the server (we need to improve this for remote files, but that needs some thinking).
So, I suspect my 4 year old laptop is not performant. While the build was running, I saw random spikes of 100% on CPU and Disk in the task manager. RAM usage by GIT was peaking around 1200MB. (Also on a ‘terrible’ 10MBPs connection)
Summary
$ git clone --single-branch -b hugo-github-issue-12440 https://github.com/jmooring/hugo-testing hugo-github-issue-12440
cd hugo-github-issue-12440
hugo server
Cloning into 'hugo-github-issue-12440'...
remote: Enumerating objects: 65, done.
remote: Counting objects: 100% (52/52), done.
remote: Compressing objects: 100% (45/45), done.
remote: Total 65 (delta 9), reused 33 (delta 3), pack-reused 13
Receiving objects: 100% (65/65), 15.72 KiB | 58.00 KiB/s, done.
Resolving deltas: 100% (10/10), done.
port 1313 already in use, attempting to use an available port
Watching for changes in C:\Users\arif\sites\hugo-github-issue-12440\{archetypes,assets,content,layouts,static}
Watching for config changes in C:\Users\arif\sites\hugo-github-issue-12440\hugo.toml
Start building sites …
hugo v0.126.0-32c967551be308fbd14e5f0dfba0ff50a60e7f5e+extended windows/amd64 BuildDate=2024-05-14T13:24:11Z VendorInfo=gohugoio
| EN
-------------------+--------
Pages | 20021
Paginator pages | 1999
Non-page files | 4
Static files | 1
Processed images | 4
Aliases | 7
Cleaned | 0
Built in 309229 ms
Environment: "development"
Serving pages from disk
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:53957/ (bind address 127.0.0.1)
Press Ctrl+C to stop
Summary
$ hugo env
hugo v0.126.0-32c967551be308fbd14e5f0dfba0ff50a60e7f5e+extended windows/amd64 BuildDate=2024-05-14T13:24:11Z VendorInfo=gohugoio
GOOS="windows"
GOARCH="amd64"
GOVERSION="go1.22.2"
github.com/sass/libsass="3.6.5"
github.com/webmproject/libwebp="v1.3.2"
Summary
Device name HP-Elitebook-820-G2
Processor Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz 2.30 GHz
Installed RAM 8.00 GB (7.88 GB usable)
System type 64-bit operating system, x64-based processor
(HDD 500GB)
Windows Defender is running…and now that you mention it, I also see some spikes in the Task Manager when hugo server is running…I am beginning to suspect if it has been behind my slow builds in recent months.
We changed the default in Hugo 0.123.0 from writing to memory to writing to disk when running the server. This is almost always a good thing (esp. if you’re low on memory), but obviously not if you’re having a virus scanner checking all files written to disk.
You could try running hugo server --renderToMemory and see if that makes a difference.
@jmooring I am assuming the whole hugo/bin folder is to be excluded?
@bephugo --renderToMemory first build is 39674ms. Second is 12337 ms and the third is 11290ms. Impressive! I had already cloned the project, so I just run the server command after deleting the public and resources folder before running the first command.
So, the antivirus is indeed to blame! But hugo server --renderToMemory is also faster! (And maybe hugo server -M should be an alias for the latter)
Start > Settings > Privacy & security > Windows Security > Open Windows Security > Virus & threat protection > Manage settings > Add or remove exclusions > Add an exclusion > Process
Then type hugo.exe add press the Add button.
I tested before and after.
Adding hugo.exe as a process exclusion reduced build times by 75%, and the build times are about the same as I reported earlier (16 seconds initial build, 9 seconds subsequent builds).
Took me 10s to build the Content Adapters example on an M1 Macbook Pro (that is heavily loaded with hundreds of opened tabs in Safari as well as other layout and graphics software).
Pretty good performance.
Also the implementation is much more elegant and straightforward than what was being proposed in the past.
Thanks! I will be using this in production, as soon as time permits.
In my defense, I’m testing in a VM that (obviously) shares resources.