I wrote partial to aggregate views from a json file in the Scratch of each page, sensitive to aliases declared in the page to capture views from prior locations of each page. Because it aggregates views, this partial should only be executed one time.
To ensure it was only executed once, I made it a cached partial, but that doesn’t work. Instead, the partial is executed as many times as there are threads, creating some nasty race conditions.
So I had to set HUGO_NUMWORKERMULTIPLIER=1 in addition to using a cached partial. That’s an ugly workaround because it limits the whole site build to just one worker thread.
I’ve considered refactoring the code to not use page Scratch, but that turns out not to be trivial. Moreover, it would still waste cycles and memory on duplicate information. So that most likely isn’t the right solution.
Is there a better way to do this?
Ideally there would be a way to:
in a partial itself, to declare that it can only be executed once.
cache results of partials declared this way.
ensure in the hugo executable code that it is only executed once.
block return from the partial on subsequent calls till cached results are available.
I know multithreaded code can be difficult, but this seems to be a well-defined use case, and focused enough that it shouldn’t be too difficult to implement. Though, I don’t know enough of go to implement it for myself.
I should also add that many of my pages have aliases. In our migration to hugo most pages changed URLs, but we keep redirects to the old URLs using aliases. And that’s why aggregation is needed, and just a simple lookup won’t do.
The partial makes use of hugo’s URL resolution machinery (site.GetPage) to aggregate views across all URLs associated with a page.
That doesn’t work because there is only one scratch for the whole site, not one for each page. It’s not clear to me how to efficiently make a mapping between pages and scratches. Perhaps a map would work, which uses the RelLink of each page as a key? But what would be an efficient way of creating that map?
Perhaps a scratch that maps to scratches is a better way? Is that even allowed?
A related tip: There was a discussion earlier where one wanted to run something once after the build, which can be accomplished by defining an output format for the home page and put it last. You can do something similar by putting it first.
I understand this. But that doesn’t take away the need to have one scratch per page, not one for the whole site.
This is interesting.
But won’t single page outputs be compiled concurrently with the home page? It seems that this only works in ensuring the ordering for a single page type (e.g. the index), but not ordering across the whole site. If so, that won’t work.