Handle Partial-in-Partial caching / Block Inheritance

support
1

I’m the maintainer of the Bilberry Hugo Theme and at the moment im struggeling with the following topic (maybe this is useful for other users / developers and someone has an idea how to handle the issue):

Link to the issue: https://github.com/Lednerb/bilberry-hugo-theme/issues/208

This theme uses Content Types in form of code, gallery, video, audio, etc.
They all have the same overall structure in a partial that is called default-content.html.

For each content type there are specific header, for example the options to include a youtube-video or a spotify-song to the video or audio type.

For using the different content types I’ve also created the specific partials and there the problem arises:

Example with the “My trip to Scotland” example post:

SINGLE_PAGE-View:
Inherits from the _default/baseof.html and includes the correct partial:

Then the partials/content-type/gallery.html partial will include the partials/default-content.html and contains the features for the gallery-slider and also the correct fontawesome icon is set:

Then the default-content.html is included (for many other content types also):

In an earlier version I’ve included the default-content as a normal partial via the partial command.

However, it seems that partials get cached and when using the same structure above for the index / list view, the contents of default-content will not be rendered correctly and in an unpredictable way.

What I’ve tried so far

In the last days I changed the code to use the partialCached instruction while setting also a $PAGE_TYPE variable in the hope to tell hugo to render the default-content partial twice - once for the list view with the summary output and the continue reading link; once for the single page content.

However the rate of race-conditions regarding the cache are dropped from almost 10% to only 1% the theme is nevertheless in an unstable status at the moment.

The next approach today has the idea of using blocks and inherit from the _default/single.html file, refactoring the partials with the respective templates (for example: gallery/single.html, audio/single.html etc.) but it seems that inheritance is not possible in an flexible way to prevent breaking the DRY-principle.

Thank you for reading this topic!
I appreciate any tips regarding the issue and how it might can be fixed without producing a lot of duplicated code.

1 Like
2

I’ve noticed that sometimes you call partials as described in the documentation:

{{ partial "directory/file.html" . }}

but at other times you omit the extension:

{{ partial "directory/file" . }}

Is there a particular reason for doing that?

3

Thanks for that catch, although I don’t think that it’s related to this specific problem.

However, this is not intended.

It’s interesting, if there is a particular reason for having the .html version in the docs - maybe for a more clear syntax to differentiate between the classic partials and the newer inline partials?

4

No. The documentation has included the extension since March 5, 2017. Here’s a somewhat related discussion:

https://discourse.gohugo.io/t/fixed-calling-partial-without-specifying-html-suffix/6028

I suspect you are correct, but it would be wise to test.

5

I’ve refactored the code and checked that always the .html suffix is present.

However, the issue still occurs.

I’ve also reset the theme to a partly working state.

Now having issues in about 6 out of 1000 builds but the test I run does not cover every page.

I will post the test script and adjustments needed to test it later.

6

Instead of posting code, can you create a minimal site to reproduce the problem, and post the link?

7

Already online in on the Demo page:

If you open the My trip to Scotland article you can see that only the summary is displayed with the Continue reading link instead of the full article:

8

Sorry, I wasn’t clear. To make it easier for others to reproduce the issue, please post instructions such as:

git clone foo/bar
cd bar
hugo server
9

General SetUp

hugo new site bilberry-test
cd ./bilberry-test/themes
git clone https://github.com/Lednerb/bilberry-hugo-theme.git
cp -r bilberry-hugo-theme/exampleSite/* ../

cd ../
hugo

Some times it works like expected, sometimes the issue occurs.

Check successful build

In order to test if the error occurs, the following python script can be used within the bilberry-test directory:

checkbuild.py

import shutil
import os

numberOfBuilds = 100

for x in xrange(0,numberOfBuilds):
    # delete old builds
    shutil.rmtree('./' + str(x), True)

    # build to specif dir
    print "Building... #" + str(x)
    os.system("hugo --quiet -d " + str(x))

for x in xrange(0,numberOfBuilds):
    # count errors
    with open('./' + str(x) + '/index.html') as f:
        if 'I could write a whole book about this fantastic journey' in f.read():
            print('Missing "Continue reading" in build #' + str(x))

    with open('./' + str(x) + '/gallery/my-trip-to-scotland/index.html') as f:
        if 'Continue reading' in f.read():
            print('False "Continue reading" in build #' + str(x))
10

For anyone else who’s testing, here’s the same script for py3.

#!/usr/bin/env python

import shutil
import os

numberOfBuilds = 1000

for x in range(0,numberOfBuilds):
    # delete old builds
    shutil.rmtree('./' + str(x), True)

    # build to specif dir
    print("Building... #" + str(x))
    os.system("hugo --quiet -d " + str(x))

for x in range(0,numberOfBuilds):
    # count errors
    with open('./' + str(x) + '/index.html') as f:
        if 'I could write a whole book about this fantastic journey' in f.read():
            print('Missing "Continue reading" in build #' + str(x))

    with open('./' + str(x) + '/gallery/my-trip-to-scotland/index.html') as f:
        if 'Continue reading' in f.read():
            print('False "Continue reading" in build #' + str(x))
11

I ran the test script 3 times; no errors.
Then I changed numberOfBuilds to 1000 and ran it again; no errors.
This is a 4 core CPU.

12

Which OS and version of Hugo do you use?

I use the following one on an 8 core CPU:

Hugo Static Site Generator v0.74.2-48565DE6 linux/amd64 BuildDate: 2020-07-17T17:25:56Z
13

Ubuntu 20.04.
Hugo built from master yesterday.

14

I tried testing again today with various system loads; still unable to reproduce.

There are a few more places where the “.html” extension is not present when calling partials:

https://github.com/Lednerb/bilberry-hugo-theme/blob/master/layouts/_default/list.html#L7
https://github.com/Lednerb/bilberry-hugo-theme/blob/master/layouts/_default/single.html#L5
https://github.com/Lednerb/bilberry-hugo-theme/blob/master/layouts/partials/article-wrapper.html#L5

Again, probably irrelevant, but wouldn’t hurt to fix.

As I read and re-read your post and the issue on GitHub, it seems like the problem boils down to this block of code in layouts/partials/default-content.html:

{{ if or (.Scratch.Get "singlePage") (.Params.noSummary) }}
  {{ .Content }}
{{ else }}
  {{ .Summary }}
  ...
  ...
{{ end }}

Meaning, sometimes .Scratch.Get "singlePage" returns false when you expect it to return true, and vice versa. Is that correct?

15

Thanks for the review. I’ve fixed it, but the error still occurs.

Yes, this is correct.

16

First, .Scratch.Get "singlePage" will return false even if the scratch wasn’t set, which shouldn’t happen, but the test should be more robust. For example, set a scratch named “template_type” with a value of “single” or “list”, then test for either of those values and throw an error if it’s not set. But don’t do that yet…

Second, this is an interesting read:

https://discourse.gohugo.io/t/notabug-race-condition-around-scratch-variables/17647/3

I was using $.Scratch in a partial and then passing that scratch on to a second partial. This is racy; don’t do it.

I have no idea if the second sentence is true, but it sort of matches what you describe. Additionally, the OP was using $.Scratch while you’re using .Scratch, so they may have had a scope problem instead.

Here’s what I would do as a test: eliminate the scratch, and pass the template type when calling the partial. For example:

layouts/_default/single.html:


{{ partial (print "content-type/" .Type ".html") (dict "ctx" . "template_type" "single") }}

layouts/partials/content-type/gallery.html:


{{ partial "default-content.html" (dict "ctx" .ctx "template_type" .template_type) }}

layouts/partials/default-content.html:


{{ if or (eq .template_type "single") (.ctx.Params.noSummary) }}

Unfortunately, it’s a lot more work than this, because you’ll have to prepend .ctx to every page variable referenced in the partials.

Or, you could try doing what the OP did:

Making a new scratch variable, setting “href”, “color”, and “title” on that, and then passing that scratch variable on to a second partial fixed it.

17

Thanks @jmooring for the research and your help with this.

I’ve decided to refactor the partial inclusions because it seems as a more clean way of fixing the issue instead of having yet another .Scratch there.

It seems to me that .Scratch is a good way to go if there are no partial-in-partial calls.
Otherwise it’s easier to explicitly call the partial with the template_type as you have shown above.

18

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.