Cachebuster death loop

carlmjohnson · June 29, 2023, 6:56pm

I’m trying to use the Hugo 112 cachebuster feature, and basically anything I do results in a death loop in which the server builds the site, writes out a hugo_stats.json, builds the site, writes out a hugo_stats.json, etc.

I’ve tried a lot of experiments to make a configuration that doesn’t result in a death loop, but I haven’t found it yet.

carlmjohnson · June 29, 2023, 7:02pm

Here is a typical debug log:

INFO build: running step process duration 51.574323ms INFO build: running step assemble duration 471ns INFO postcss: use config file "/Users/adhoc/src/spl/poor-richard/postcss.config.js" INFO build: running step render duration 4.538412624s INFO build: running step postProcess duration 69.429737ms Total in 4660 ms INFO Received System Events: [CHMOD "/Users/adhoc/src/spl/poor-richard/hugo_stats.json" WRITE "/Users/adhoc/src/spl/poor-richard/hugo_stats.json"]

Change detected, rebuilding site.
2023-06-29 15:00:41.326 -0400
DEBUG Rebuild for events [“WRITE "/Users/adhoc/src/spl/poor-richard/hugo_stats.json"”]
DEBUG cachebuster: Matching “assets/watching/hugo_stats.json” with source “assets/watching/hugo_stats\.json”: match!
DEBUG cachebuster: Matching “assets/watching/hugo_stats.json” with source “(postcss|tailwind)\.config\.js”: no match
DEBUG cachebuster: Matching “assets/watching/hugo_stats.json” with source “assets/.\.(js|ts|jsx|tsx)$": no match
DEBUG cachebuster: Matching “assets/watching/hugo_stats.json” with source "assets/.\.(.*)$”: match!
DEBUG Matching “js/js/esbuild/embed-frame.js” with target “tw”: no match
DEBUG Matching “js/js/esbuild/embed-frame.js” with target “json”: no match
DEBUG Matching “svg/img/icons/close-quotations.svg” with target “tw”: no match
DEBUG Matching “svg/img/icons/close-quotations.svg” with target “json”: no match
DEBUG Matching “svg/img/icons/open-quotations.svg” with target “tw”: no match
DEBUG Matching “svg/img/icons/open-quotations.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/circle-arrow-left.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/circle-arrow-left.svg” with target “json”: no match
DEBUG Matching “js/js/esbuild/embed-host.js” with target “tw”: no match
DEBUG Matching “js/js/esbuild/embed-host.js” with target “json”: no match
DEBUG Matching “js/js/esbuild/enhancements.js” with target “tw”: no match
DEBUG Matching “js/js/esbuild/enhancements.js” with target “json”: no match
DEBUG Matching “svg/img/logos/favicon.svg_8848f55d07695b7ff7188138f23d69e3” with target “tw”: no match
DEBUG Matching “svg/img/logos/favicon.svg_8848f55d07695b7ff7188138f23d69e3” with target “json”: no match
DEBUG Matching “svg/@fontawesome/brands/github.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/brands/github.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/brands/twitter.svg” with target “tw”: match!
DEBUG Matching “svg/@fontawesome/solid/share.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/share.svg” with target “json”: no match
DEBUG Matching “scss/css/mailchimp.scss_70f987bf3566a9cd5365f2e6a43facf7” with target “tw”: no match
DEBUG Matching “scss/css/mailchimp.scss_70f987bf3566a9cd5365f2e6a43facf7” with target “json”: no match
DEBUG Matching “svg/@fontawesome/brands/instagram.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/brands/instagram.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/magnifying-glass.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/magnifying-glass.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/user.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/user.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/regular/calendar-check.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/regular/calendar-check.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/chevron-down.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/chevron-down.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/spinner.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/spinner.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/xmark.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/xmark.svg” with target “json”: no match
DEBUG Matching “svg/img/logos/circle-default-on-trans.svg” with target “tw”: no match
DEBUG Matching “svg/img/logos/circle-default-on-trans.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/brands/apple.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/brands/apple.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/brands/flipboard.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/brands/flipboard.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/heart.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/heart.svg” with target “json”: no match
DEBUG Matching “svg/img/logos/favicon.svg” with target “tw”: no match
DEBUG Matching “svg/img/logos/favicon.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/circle-xmark.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/circle-xmark.svg” with target “json”: no match
DEBUG Matching “js/js/esbuild/smoothscroll.js_e437d13810da44639218af5b3b1ef3ef” with target “tw”: no match
DEBUG Matching “js/js/esbuild/smoothscroll.js_e437d13810da44639218af5b3b1ef3ef” with target “json”: no match
DEBUG Matching “js/js/esbuild/smoothscroll.js” with target “tw”: no match
DEBUG Matching “js/js/esbuild/smoothscroll.js” with target “json”: no match
DEBUG Matching “svg/@fontawesome/regular/envelope.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/regular/envelope.svg” with target “json”: no match
DEBUG Matching “js/js/esbuild/embed-frame.js_e437d13810da44639218af5b3b1ef3ef” with target “tw”: no match
DEBUG Matching “js/js/esbuild/embed-frame.js_e437d13810da44639218af5b3b1ef3ef” with target “json”: no match
DEBUG Matching “js/js/esbuild/embed-host.js_93c053c5d6b9dd60fc0431302292e672” with target “tw”: no match
DEBUG Matching “js/js/esbuild/embed-host.js_93c053c5d6b9dd60fc0431302292e672” with target “json”: no match
DEBUG Matching “svg/img/logos/banner-default-on-trans.svg” with target “tw”: no match
DEBUG Matching “svg/img/logos/banner-default-on-trans.svg” with target “json”: no match
DEBUG Matching “minjs/js/static/nomodules.frozen.minjs” with target “tw”: no match
DEBUG Matching “minjs/js/static/nomodules.frozen.minjs” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/check.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/check.svg” with target “json”: no match
DEBUG Matching “other/css/tw-merri.css” with target “tw”: match!
DEBUG Matching “png/img/logos/circle-black-on-trans-920.png_8848f55d07695b7ff7188138f23d69e3” with target “tw”: no match
DEBUG Matching “png/img/logos/circle-black-on-trans-920.png_8848f55d07695b7ff7188138f23d69e3” with target “json”: no match
DEBUG Matching “css/css/tw-merri.css_ad9a12809e06de71aca16e9138fbecf4” with target “tw”: match!
DEBUG Matching “svg/@fontawesome/solid/circle-plus.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/circle-plus.svg” with target “json”: no match
DEBUG Matching “png/img/logos/circle-black-on-trans-920.png” with target “tw”: no match
DEBUG Matching “png/img/logos/circle-black-on-trans-920.png” with target “json”: no match
DEBUG Matching “svg/@fontawesome/regular/copy.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/regular/copy.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/brands/google.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/brands/google.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/comment.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/comment.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/phone.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/phone.svg” with target “json”: no match
DEBUG Matching “svg/@fontawesome/brands/facebook-f.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/brands/facebook-f.svg” with target “json”: no match
DEBUG Matching “other/embed.js” with target “tw”: no match
DEBUG Matching “other/embed.js” with target “json”: no match
DEBUG Matching “png/img/logos/square-white-on-black-920.png” with target “tw”: no match
DEBUG Matching “png/img/logos/square-white-on-black-920.png” with target “json”: no match
DEBUG Matching “css/css/font-rale.css” with target “tw”: no match
DEBUG Matching “css/css/font-rale.css” with target “json”: no match
DEBUG Matching “svg/@fontawesome/solid/circle-arrow-right.svg” with target “tw”: no match
DEBUG Matching “svg/@fontawesome/solid/circle-arrow-right.svg” with target “json”: no match
DEBUG Matching “scss/css/mailchimp.scss” with target “tw”: no match
DEBUG Matching “scss/css/mailchimp.scss” with target “json”: no match
DEBUG Matching “svg/img/logos/banner-state-college.svg” with target “tw”: no match
DEBUG Matching “svg/img/logos/banner-state-college.svg” with target “json”: no match
DEBUG Matching “css/css/font-merri.css” with target “tw”: no match
DEBUG Matching “css/css/font-merri.css” with target “json”: no match
DEBUG Matching “css/css/tw.css” with target “tw”: match!
DEBUG Matching “js/js/esbuild/enhancements.js_1b9430a292039bc94f23e7237aef1f3c” with target “tw”: no match
DEBUG Matching “js/js/esbuild/enhancements.js_1b9430a292039bc94f23e7237aef1f3c” with target “json”: no match
INFO build: running step process duration 32.109046ms
INFO build: running step assemble duration 969ns
INFO postcss: use config file “/Users/adhoc/src/spl/poor-richard/postcss.config.js”
^C⏎

And some relevant config:

[[module.mounts]]
source = "assets"
target = "assets"

[[module.mounts]]
source = "hugo_stats.json"
target = "assets/watching/hugo_stats.json"

[[module.mounts]]
source = "src"
target = "assets/js"

[[module.mounts]]
source = "node_modules/@fortawesome/fontawesome-free/svgs"
target = "assets/@fontawesome"

[[module.mounts]]
source = "static"
target = "static"

[build]
writeStats = true
noJSConfigInAssets = true

[[build.cachebusters]]
source = "assets/watching/hugo_stats\\.json"
target = "tw"

[[build.cachebusters]]
source = "(postcss|tailwind)\\.config\\.js"
target = "css"

[[build.cachebusters]]
source = "assets/.*\\.(js|ts|jsx|tsx)$"
target = "js"

[[build.cachebusters]]
source = "assets/.*\\.(.*)$"
target = "$1"

carlmjohnson · June 29, 2023, 7:09pm

The debug logs also seem to refer to some cashebuster rules even if they’re not in my config, so I think there’s something wrong with the overwriting of defaults.

carlmjohnson · June 29, 2023, 7:46pm

I’m trying to delete as much of my site as possible to minimize whatever difference exist between my site and the demo, but I still have a death loop:

INFO  build: running step process duration 16.25778ms
INFO  build: running step assemble duration 374ns
INFO  postcss: use config file "/Users/adhoc/src/spl/poor-richard/postcss.config.js"
INFO  build: running step render duration 2.306304418s
INFO  build: running step postProcess duration 53.710184ms
Total in 2376 ms
INFO  Received System Events: [CHMOD         "/Users/adhoc/src/spl/poor-richard/hugo_stats.json" WRITE         "/Users/adhoc/src/spl/poor-richard/hugo_stats.json"]

Change detected, rebuilding site.
2023-06-29 15:39:43.582 -0400
DEBUG Rebuild for events ["WRITE         \"/Users/adhoc/src/spl/poor-richard/hugo_stats.json\""]
DEBUG cachebuster: Matching "assets/~/hugo_stats" with source "hugo_stats": match!
DEBUG cachebuster: Matching "assets/~/hugo_stats" with source "(postcss|tailwind)\\.config\\.js": no match
DEBUG cachebuster: Matching "assets/~/hugo_stats" with source "assets/.*\\.(js|ts|jsx|tsx)$": no match
DEBUG cachebuster: Matching "assets/~/hugo_stats" with source "assets/.*\\.(.*)$": no match
DEBUG Matching "css/css/tw.css_8c59c3265176257da1c2c4bd8c6b6494" with target "css": match!
DEBUG Matching "minjs/js/static/nomodules.frozen.minjs" with target "css": no match
DEBUG Matching "js/js/esbuild/embed-host.js" with target "css": no match
DEBUG Matching "js/js/esbuild/embed-host.js_93c053c5d6b9dd60fc0431302292e672" with target "css": no match
DEBUG Matching "other/embed.js" with target "css": no match
DEBUG Matching "svg/@fontawesome/regular/copy.svg" with target "css": no match
DEBUG Matching "svg/img/logos/favicon.svg_8848f55d07695b7ff7188138f23d69e3" with target "css": no match
DEBUG Matching "js/js/esbuild/embed-frame.js_e437d13810da44639218af5b3b1ef3ef" with target "css": no match
DEBUG Matching "svg/@fontawesome/solid/circle-plus.svg" with target "css": no match
DEBUG Matching "css/css/tw.css" with target "css": match!
DEBUG Matching "png/img/logos/square-white-on-black-920.png" with target "css": no match
DEBUG Matching "js/js/esbuild/embed-frame.js" with target "css": no match
DEBUG Matching "js/js/esbuild/smoothscroll.js" with target "css": no match
DEBUG Matching "js/js/esbuild/enhancements.js" with target "css": no match
DEBUG Matching "js/js/esbuild/enhancements.js_1b9430a292039bc94f23e7237aef1f3c" with target "css": no match
DEBUG Matching "svg/img/logos/favicon.svg" with target "css": no match
DEBUG Matching "png/img/logos/circle-black-on-trans-920.png" with target "css": no match
DEBUG Matching "png/img/logos/circle-black-on-trans-920.png_8848f55d07695b7ff7188138f23d69e3" with target "css": no match
DEBUG Matching "js/js/esbuild/smoothscroll.js_e437d13810da44639218af5b3b1ef3ef" with target "css": no match
INFO  build: running step process duration 15.851813ms
INFO  build: running step assemble duration 513ns
INFO  postcss: use config file "/Users/adhoc/src/spl/poor-richard/postcss.config.js"
^C⏎

bep · June 29, 2023, 7:56pm

We (Hugo) compares the hugo_stats.json on disk with that’s in memory before be write anything. This has worked very well in the tests I’ve been doing (on MacOs and Windows).

In your case you seem to end up with different hugo_stats.json on every rebuild.

What you could do is to

Run hugo
Store away the stats file
Rerun hugo
Compare the two files and see … what.

I suspect some odd whitespace issue, though, which may be harder to figure out.

carlmjohnson · June 29, 2023, 9:18pm

It seems like it’s getting confused about exactly which IDs are on the site because they’re part of a page.Store that increments as the page goes along.

carlmjohnson · June 29, 2023, 9:20pm

See Featured request: Auto-ID helper function · Issue #7734 · gohugoio/hugo · GitHub

carlmjohnson · June 29, 2023, 9:27pm

Okay, stripping out the auto-ID partial fixes the doomloop, but causes new problems. Thanks for the pointer.

bep · June 30, 2023, 7:31am

I reopened this (for discussion): Featured request: Auto-ID helper function · Issue #7734 · gohugoio/hugo · GitHub

carlmjohnson · June 30, 2023, 11:36am

I can’t comment on the issue because it is still closed.

Hashing the relpermalink doesn’t work because the idea is to produce unique IDs on one page, not a link from one page to another. Eg if you have a form, each field in the form requires a unique id to link a label to its field. And you don’t know in advance how many times the form will be included on a page. Similar issues exist for aria-labelled-by etc.

The real fault is with W3C for making us use IDs for things that aren’t supposed to be linked to or even unique, but I can’t make them change.

bep · June 30, 2023, 12:13pm

D
unlocked.

Sure, but then you need to increment a local counter that starts at 0 or something. My main take is that having a form that changes its ID between Hugo runs is a bad idea, and I’m pretty sure it can be avoided using the tools available in Hugo today.

bep · July 1, 2023, 8:23am

I think in most cases, you don’t care about the IDs, so we should provide a way to ignore them:

Topic		Replies	Views
Hugo building before Tailwind classes are created support	11	727	November 30, 2023
Do I need to configure cache busters if I am not using Tailwind? support	5	46	December 17, 2024
Using the new `cachebusters` feature with a theme support	7	879	June 7, 2023
Slow Tailwind / Hugo builds. How to troubleshoot NPM / hugo pipes' performance? support	2	374	April 19, 2023
Tailwind JIT and Hugo Purge issues support	10	2760	December 21, 2021

Cachebuster death loop

Related topics