# Content adapters: examples and performance

**URL:** <https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830>\
**Category:** tips & tricks\
**Tags:** content-adapters\
**Created:** [May 15, 2024, 3:09pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830 "2024-05-15T15:09:09Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmooring](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/jmooring/32/4214_2.png) [@jmooring](https://discourse.gohugo.io/u/jmooring)\
**Post date:** [May 15, 2024, 3:09pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/1 "2024-05-15T15:09:09Z")

</div>

This really isn’t a “tips & tricks” topic, but rather some notes about the [content adapter](https://gohugo.io/content-management/content-adapters/) feature introduced in v0.126.0.

Here’s an example site to explore capabilities:

```plaintext
git clone --single-branch -b hugo-github-issue-12440 https://github.com/jmooring/hugo-testing hugo-github-issue-12440
cd hugo-github-issue-12440
hugo server

```

This site uses content adapters to create pages from remote data sources.

- The “books” section is a detailed example containing 4 pages created from:  
[https://gohugo.io/shared/examples/data/books.json](https://gohugo.io/shared/examples/data/books.json) (5 KB)

- The “posts” section contains 20,000 pages created from:  
[https://www.veriphor.com/shared/data/posts.json](https://www.veriphor.com/shared/data/posts.json) (49 MB)

On my very average laptop, with a terrible network connection, the initial build is 20 seconds, or about 0.001 seconds/page. Of that, approximately 9 seconds is spent downloading the 49MB remote data file. Subsequent builds are 11 seconds (the data file is cached), or about 0.0006 seconds/page.

Note that the “posts” section is paginated with 10 pages per pager, creating 2000 pagers in addition to the 20,000 posts. So the number is closer to 0.0005 seconds per page.

While this example site creates pages from remote data, you can also create pages from local data files and global resources (e.g., create a page for each image in the assets directory).

---

<div class="post-metadata">

**Author:** ![bep](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/bep/32/3332_2.png) [@bep](https://discourse.gohugo.io/u/bep)\
**Post date:** [May 15, 2024, 3:29pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/2 "2024-05-15T15:29:47Z")

</div>

> [@jmooring](#):
>
> While this example site creates pages from remote data, you can also create pages from local data files and global resources (e.g., create a page for each image in the assets directory).

One benefit of local files (e.g. in `assets`) is that it works better for edits when running the server (we need to improve this for remote files, but that needs some thinking).

---

<div class="post-metadata">

**Author:** ![bep](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/bep/32/3332_2.png) [@bep](https://discourse.gohugo.io/u/bep)\
**Post date:** [May 15, 2024, 3:32pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/3 "2024-05-15T15:32:37Z")

</div>

> [@jmooring](#):
>
> Subsequent builds are 11 seconds (the data file is cached),

It builds in 5,5 seconds on my MacBook Pro M1 32 GB.

---

<div class="post-metadata">

**Author:** ![jmooring](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/jmooring/32/4214_2.png) [@jmooring](https://discourse.gohugo.io/u/jmooring)\
**Post date:** [May 15, 2024, 3:37pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/4 "2024-05-15T15:37:26Z")

</div>

It also scales in a linear fashion. If I duplicate the “posts” section 4 times to generate 100,000 pages, the build time per page remains the same.

---

<div class="post-metadata">

**Author:** ![bep](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/bep/32/3332_2.png) [@bep](https://discourse.gohugo.io/u/bep)\
**Post date:** [May 15, 2024, 4:10pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/5 "2024-05-15T16:10:38Z")

</div>



---

<div class="post-metadata">

**Author:** ![bwintx](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/bwintx/32/10450_2.png) [@bwintx](https://discourse.gohugo.io/u/bwintx)\
**Post date:** [May 15, 2024, 4:32pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/6 "2024-05-15T16:32:59Z")

</div>

On a Mac Studio M2 64 GB with a 2.5 Gbit connection, built the first time in 9.753 seconds and the second time in 5.070 seconds. Remarkable.

---

<div class="post-metadata">

**Author:** ![jmooring](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/jmooring/32/4214_2.png) [@jmooring](https://discourse.gohugo.io/u/jmooring)\
**Post date:** [May 15, 2024, 4:38pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/7 "2024-05-15T16:38:14Z")

</div>

I feel I’m being tech-shamed. 😄 In my defense, I’m testing in a VM that (obviously) shares resources.

---

<div class="post-metadata">

**Author:** ![bwintx](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/bwintx/32/10450_2.png) [@bwintx](https://discourse.gohugo.io/u/bwintx)\
**Post date:** [May 15, 2024, 4:39pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/8 "2024-05-15T16:39:30Z")

</div>

Your data is perhaps more applicable to other users and more real-world business environments, however.

---

<div class="post-metadata">

**Author:** ![frjo](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/frjo/32/18626_2.png) [@frjo](https://discourse.gohugo.io/u/frjo)\
**Post date:** [May 15, 2024, 4:56pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/9 "2024-05-15T16:56:53Z")

</div>

Thanks for posting this example! This is a brilliant new feature in Hugo and the performance is impressive indeed.

Now I more than ever want to try this out on a real project.

---

<div class="post-metadata">

**Author:** ![Arif](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/arif/32/20931_2.png) [@Arif](https://discourse.gohugo.io/u/Arif)\
**Post date:** [May 15, 2024, 6:44pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/10 "2024-05-15T18:44:33Z")

</div>

So, I suspect my 4 year old laptop is not performant. While the build was running, I saw random spikes of 100% on CPU and Disk in the task manager. RAM usage by GIT was peaking around 1200MB. (Also on a ‘terrible’ 10MBPs connection)

> **Summary**
>
> ```auto
> $ git clone --single-branch -b hugo-github-issue-12440 https://github.com/jmooring/hugo-testing hugo-github-issue-12440
> cd hugo-github-issue-12440
> hugo server
> Cloning into 'hugo-github-issue-12440'...
> remote: Enumerating objects: 65, done.
> remote: Counting objects: 100% (52/52), done.
> remote: Compressing objects: 100% (45/45), done.
> remote: Total 65 (delta 9), reused 33 (delta 3), pack-reused 13
> Receiving objects: 100% (65/65), 15.72 KiB | 58.00 KiB/s, done.
> Resolving deltas: 100% (10/10), done.
> port 1313 already in use, attempting to use an available port
> Watching for changes in C:\Users\arif\sites\hugo-github-issue-12440\{archetypes,assets,content,layouts,static}
> Watching for config changes in C:\Users\arif\sites\hugo-github-issue-12440\hugo.toml
> Start building sites …
> hugo v0.126.0-32c967551be308fbd14e5f0dfba0ff50a60e7f5e+extended windows/amd64 BuildDate=2024-05-14T13:24:11Z VendorInfo=gohugoio
> 
> | EN
> -------------------+--------
> Pages | 20021
> Paginator pages | 1999
> Non-page files | 4
> Static files | 1
> Processed images | 4
> Aliases | 7
> Cleaned | 0
> 
> Built in 309229 ms
> Environment: "development"
> Serving pages from disk
> Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
> Web Server is available at http://localhost:53957/ (bind address 127.0.0.1)
> Press Ctrl+C to stop
> 
> ```

> **Summary**
>
> ```auto
> $ hugo env
> hugo v0.126.0-32c967551be308fbd14e5f0dfba0ff50a60e7f5e+extended windows/amd64 BuildDate=2024-05-14T13:24:11Z VendorInfo=gohugoio
> GOOS="windows"
> GOARCH="amd64"
> GOVERSION="go1.22.2"
> github.com/sass/libsass="3.6.5"
> github.com/webmproject/libwebp="v1.3.2"
> 
> ```

> **Summary**
>
> ```auto
> Device name	HP-Elitebook-820-G2
> Processor	Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz 2.30 GHz
> Installed RAM	8.00 GB (7.88 GB usable)
> System type	64-bit operating system, x64-based processor
> (HDD 500GB)
> 
> ```

---

<div class="post-metadata">

**Author:** ![jmooring](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/jmooring/32/4214_2.png) [@jmooring](https://discourse.gohugo.io/u/jmooring)\
**Post date:** [May 15, 2024, 6:49pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/11 "2024-05-15T18:49:07Z")

</div>

I noticed that you are running windows. From a recent topic on this forum, is there any chance that you have a virus scanner running?

---

<div class="post-metadata">

**Author:** ![Arif](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/arif/32/20931_2.png) [@Arif](https://discourse.gohugo.io/u/Arif)\
**Post date:** [May 15, 2024, 6:52pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/12 "2024-05-15T18:52:05Z")

</div>

Second run `Built in 172424 ms`.

> [@jmooring](#):
>
> is there any chance that you have a virus scanner running?

Windows Defender is running…and now that you mention it, I also see some spikes in the Task Manager when hugo server is running…I am beginning to suspect if it has been behind my slow builds in recent months.

---

<div class="post-metadata">

**Author:** ![jmooring](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/jmooring/32/4214_2.png) [@jmooring](https://discourse.gohugo.io/u/jmooring)\
**Post date:** [May 15, 2024, 6:52pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/13 "2024-05-15T18:52:48Z")

</div>

Add an exclusion and see what happens.

---

<div class="post-metadata">

**Author:** ![bep](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/bep/32/3332_2.png) [@bep](https://discourse.gohugo.io/u/bep)\
**Post date:** [May 15, 2024, 7:02pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/14 "2024-05-15T19:02:04Z")

</div>

> [@Arif](#):
>
> I am beginning to suspect if it has been behind my slow builds in recent months.

We changed the default in Hugo 0.123.0 from writing to memory to writing to disk when running the server. This is almost always a good thing (esp. if you’re low on memory), but obviously not if you’re having a virus scanner checking all files written to disk.

You could try running `hugo server --renderToMemory` and see if that makes a difference.

---

<div class="post-metadata">

**Author:** ![Arif](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/arif/32/20931_2.png) [@Arif](https://discourse.gohugo.io/u/Arif)\
**Post date:** [May 15, 2024, 7:22pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/15 "2024-05-15T19:22:24Z")

</div>

@jmooring I am assuming the whole `hugo/bin` folder is to be excluded?

@bep `hugo --renderToMemory` first build is `39674ms`. Second is `12337 ms` and the third is `11290ms`. Impressive! I had already cloned the project, so I just run the server command after deleting the public and resources folder before running the first command.

So, the antivirus is indeed to blame! But `hugo server --renderToMemory` is also faster! (And maybe `hugo server -M` should be an alias for the latter)

---

<div class="post-metadata">

**Author:** ![bep](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/bep/32/3332_2.png) [@bep](https://discourse.gohugo.io/u/bep)\
**Post date:** [May 15, 2024, 8:09pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/16 "2024-05-15T20:09:16Z")

</div>

> [@Arif](#):
>
> So, the antivirus is indeed to blame! But `hugo server --renderToMemory` is also faster!

Yea, this particular example may be that. But once you throw some big files into the mix (lots of images etc.) you start getting memory constrained.

---

<div class="post-metadata">

**Author:** ![jmooring](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/jmooring/32/4214_2.png) [@jmooring](https://discourse.gohugo.io/u/jmooring)\
**Post date:** [May 15, 2024, 8:57pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/17 "2024-05-15T20:57:19Z")

</div>

> [@Arif](#):
>
> I am assuming the whole `hugo/bin` folder is to be excluded?

With Microsoft Defender Antivirus:

**Start** \> **Settings** \> **Privacy & security** \> **Windows Security** \> **Open Windows Security** \> **Virus & threat protection** \> **Manage settings** \> **Add or remove exclusions** \> **Add an exclusion** \> **Process**

Then type `hugo.exe` add press the **Add** button.

I tested before and after.

Adding hugo.exe as a process exclusion reduced build times by 75%, and the build times are about the same as I reported earlier (16 seconds initial build, 9 seconds subsequent builds).

---

<div class="post-metadata">

**Author:** ![Arif](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/arif/32/20931_2.png) [@Arif](https://discourse.gohugo.io/u/Arif)\
**Post date:** [May 15, 2024, 10:30pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/18 "2024-05-15T22:30:41Z")

</div>

Thanks. I have just gotten one of my data sources working with the new method (and it was a learning experience). I am impressed! Congrats to you all.

---

<div class="post-metadata">

**Author:** ![jmooring](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/jmooring/32/4214_2.png) [@jmooring](https://discourse.gohugo.io/u/jmooring)\
**Post date:** [May 16, 2024, 2:24pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/19 "2024-05-16T14:24:04Z")

</div>

Virus scanning and performance:  
[https://gohugo.io/troubleshooting/performance/#virus-scanning](https://gohugo.io/troubleshooting/performance/#virus-scanning)

---

<div class="post-metadata">

**Author:** ![alexandros](https://avatars.discourse-cdn.com/v4/letter/a/ecc23a/32.png) [@alexandros](https://discourse.gohugo.io/u/alexandros)\
**Post date:** [May 16, 2024, 4:31pm UTC](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830/20 "2024-05-16T16:31:38Z")

</div>

@bep @jmooring & co

Great work! 👍

Took me 10s to build the Content Adapters example on an M1 Macbook Pro (that is heavily loaded with hundreds of opened tabs in Safari as well as other layout and graphics software).

Pretty good performance.

Also the implementation is much more elegant and straightforward than what was being proposed in the past.

Thanks! I will be using this in production, as soon as time permits.

[Next page](https://discourse.gohugo.io/t/content-adapters-examples-and-performance/49830.md?page=2)