Testing (timing actually)

I would like to somehow test how various strategies impact Hugo’s build times. Is there a nice way to “hook” somehow into Hugo to get lists of times that certain things need to run? I can read :wink: and I am eager to learn, so linkdrops are ok.

Sample test:

I wonder which one of the following strategies saves/costs more time:

  • a configuration value is set in data/path/to/file.toml
  • method a) loading every time via {{ site.Data.path/to/file/variable | default "value" }}
  • method b) loads once and return via partialCached
  • method c) loads once and save in a scratch variable that is reused
  • i run hugo with each method and 100 calls to get that value
  • i get some form of output with a time per method

Does Hugo have some form of hooks that a different go script could somehow use to play around? Worst case is of course to let a shell script run a test site multiple times and collect an average.

I am only aware of the CLI --templateMetrics flag, I suppose that you would have to use external tools to run a benchmark.

Data Files in great numbers are known to slow down build time see: Data Files eat memory · Issue #1065

1 Like

That tag weirdly is in my “I don’t like you”-list :wink: It tells me the following layout file is not cachable (0), but I am not sure if it’s worth pestering the team with a bug report:

<div class="progress" id="readingprogress">
    <div class="progress-bar"
         id="readingprogress-bg"
         role="progressbar"
         aria-label="Readingprogress"
         aria-valuenow="0"
         aria-valuemin="0"
         aria-valuemax="100"></div>
</div>

Of course the timing output on this param might help or even be what I want… I will play around with it. Putting a “method” into a layout and checking on it’s average timing value might be what I want.

Yeah :laughing: I doubled my site creation from about 1 minute to 2 minutes by fragmenting everything into modules (up to 3 levels deep) with their own namespaced data configuration. It feeds my OCD to have everything in the right place, but it costs dearly on the speed.

Please explain.

That’s all there is. The code above has not one single variable and Hugo says it’s not something to cache. I would expect a 100 but hugo --templateMetrics says 0.

I sometimes use a Go benchmark to build the docs site. I suppose you could do the same thing to test changes to a site. Here’s how:

Create a Go test file in the root of the hugo codebase as seen below. Set the flags as needed, but mainly set the --source flag to the path of your site:

package main

import (
        "testing"

        "github.com/gohugoio/hugo/commands"
)

func BenchmarkDocsSite(b *testing.B) {
        flags := []string{
                "--quiet",
                "--source=docs",
                "--renderToMemory",
        }

        for i := 0; i < b.N; i++ {
                r := commands.Execute(flags)
                if r.Err != nil {
                        b.Fatal(r.Err)
                }
        }
}

Run the benchmark and save the output to a file. I use tee for this:

go test -bench=. -benchmem -count=5 . | tee baseline.txt

Make changes to your project and rerun the benchmark, saving to another file:

go test -bench=. -benchmem -count=5 . | tee variant-1.txt

Install benchstat if you don’t have it already (don’t do this while within the Hugo project or it may update your go.mod file):

go get golang.org/x/perf/cmd/benchstat

Use benchstat to compare the results:

benchstat baseline.txt variant-1.txt

Curious if this shows you anything useful.

I can’t reproduce this. Can you provide a site that reproduces this?

I added you to my repo. It’s a bit large and moduled.

1 Like

Bug filed for the cache hint issue:

1 Like