Need help with deep recursive custom shortcode

support
1

Hello, I implemented a shortcode called glossary to provide modals that contain definitions for terms when user clicks on said term on a page. These definitions can contain calls to other glossary shortcodes of related terms.

Flow:

  • The terms with definitions are present in frontmatter of content/glossary.md.
  • The shortcode is called from a content page with the arguments term and displayTerm (optional outside glossary.md frontmatter).
  • ShortCode retrieves glossary.md, finds the description, calls markdownify on it (may contain other glossary shortcode) and renders the modal.

Initially it worked well, I had even handled cyclic recursion. But now that the number of terms have increased to around 100 often with deep linking between the terms, the build time takes forever.

The issue is that there is repeated work being done for the same term. I tried to cache the result of markdownify of term descriptions through Scratch but looks like it does not support persistence/scope over cross shortcode calls.

Is there any way to implement the caching to improve build time? or is there a better method of implementing the feature? I feel its better to avoid all hassle by just using javascript to produce modals dynamically only when a user clicks them instead of statically rendering all modals during build.

Here is the glossary.html shortcode (skipping styles and script):

{{- $glossary := site.GetPage "glossary.md" -}}

{{- $term := .Get "term" | lower -}} {{/* The term to be defined */}}
{{- $displayTerm := .Get "displayTerm" | default (.Get "term") -}} {{/* The term to be displayed */}}
{{- $ancestorTerms := .Get "ancestorTerms" | default "" -}} {{/* The recursive terms history */}}

{{- $definition := "" -}} {{/* The definition of the term */}}
{{- $renderedDefinition := "" -}} {{/* The rendered definition of the term */}}
{{- $cachedDefinition := "" -}} {{/* The cached definition of the term */}}

{{- warnf $ancestorTerms -}}

{{- /*
    /* If the term is already in ancestorTerms, then prevent cyclic recursion by not rendering the term again.
    /* If the rendered definition is already cached, then use the cached definition.
    /* Else find the definition of the term in the glossary, render it and cache it.
    /* If no definition is found, do not set the rendered definition.
    /*
*/ -}}
{{- if not (strings.Contains $ancestorTerms (printf "{%s}" $term)) -}} {{/* Prevents cyclic recursion */}}
    {{- $cachedDefinition = .Scratch.Get $term -}}

    {{- if $cachedDefinition -}}
        {{- $renderedDefinition = $cachedDefinition -}}
    {{- else -}}
        {{- range $key, $value := $glossary.Params.glossary -}}
            {{- if eq (lower $key) $term -}}
                {{- $definition = $value -}}
            {{- end -}}
        {{- end -}}

        {{- if $definition -}}
            {{- /*
                * The glossary shortcode is of the pattern {{< glossary term="term" displayTerm="displayTerm" ancestorTerms="ancestorTerms" >}}
                * ancestorTerms is used to prevent infinite recursion in the glossary shortcode where terms are delimited by {}
                * Below matches the value of the ancestorTerms attribute of the glossary shortcode and appends the current term to the ancestorTerms.
            */ -}}
            {{- $shortcodePattern := `\{\{[ ]*<[ ]*glossary[ ]+term="(?<d_term>[^"]+)"(?:[ ]+displayTerm="(?<d_displayTerm>[^"]+)")?(?:[ ]+ancestorTerms="(?<d_ancestorTerms>[^"]+)")?[ ]*>[ ]*\}\}` -}}
            {{- $renderedDefinition = replaceRE $shortcodePattern (printf `{{< glossary term="$d_term" displayTerm="$d_displayTerm" ancestorTerms="%s{%s}" >}}` $ancestorTerms $term) $definition -}}

            {{- .Scratch.Set $term ($renderedDefinition | markdownify) -}}
        {{- end -}}
    {{- end -}}
{{- end -}}


{{- /*
    /* If the rendered definition is not empty, then render the glossary modal.
    /* Else, display the term as is.
    /*
*/ -}}
{{- if $renderedDefinition -}}
    {{- /* Prevents multiple glossary terms on the same page from sharing the same modal ID */ -}}
    {{- $unique_id := delimit (shuffle (seq 1 15)) "" -}}

    {{- /* Render the glossary modal with cached or newly processed definition */ -}}
    <span class="glossary-container" onclick="openModal('{{- $term -}}', '{{- $unique_id -}}')">
        <span class="glossary-term"> {{- $displayTerm -}} </span>

        <span id="modal-{{- $term -}}-{{- $unique_id -}}" class="glossary-modal">
            <span class="modal-content">
                <span> {{- $renderedDefinition -}} </span>
                <span class="modal-close" onclick="closeModal('{{- $term }}', '{{- $unique_id }}')">&times;</span>
            </span>
        </span>
    </span>
{{- else -}}
  {{- $displayTerm -}}
{{- end -}}

my repo: GitHub - sandeshShahapur/ADeveloperHasNoName at glossary

use hugo server command without any flags as some of them like -D will give errors. errors are related to the shortcode and i don’t know how to debug it (slice bounds out of range).

my env:

$ hugo env
hugo v0.135.0+extended windows/386 BuildDate=unknown
GOOS="windows"
GOARCH="386"
GOVERSION="go1.23.2"
github.com/sass/libsass="3.6.6"
github.com/webmproject/libwebp="v1.3.2"

any help in appreciated

2

Do you know about partials.IncludeCached | Hugo yet?

If not, give it a read. I think what you want is to put the retrieval of the glossary and the parsing into markdown in a partial that is cached and then load that cached partial in your shortcode.

On the other side, I could be wrong, but I think if you put your glossary into a toml file in data/some/path/glossary.toml you can load it via site.Data.some.path.glossary and Hugo will already cache that for you… but I am not 100% sure about that. Even if not, you can work better with TOML/YAML/JSON on managing your data. Then load it, parse it, and cache that call.

In 95% of cases you don’t need to use a scratch anymore. Check your sources, if the forum post you are relying on is older than 2 years then ignore the advise if it contains scratch.

3

just cloned your repo and it bails out with hugoand hugo server so no chance to build…

why are you duplicating the terms in frontmatter and the markdown within glossary.md

You could use a scratch at the page level .Page.Scratch.

But guess a wrapped cahchedPartial as @davidsneighbour said might be a choice

4

Hey, I attempted to implement the suggestions you made but I did not succeed. While the build processing time issue remained, there were also other issues or errors which were hard to debug. I don’t know Hugo enough to make it work.

Found it too much hassle so instead I implemented the feature through javascipt which works great. The build times are sub 1s and I can increase the number of terms in my glossary without it having observable future performance impacts.

5

bad code haha. i now implemented a template to render the glossary page.

I ended up switching to implementation through javascript.

you can try it out now - GitHub - sandeshShahapur/ADeveloperHasNoName at main

1 Like