Collecting values over a site (using .Site.Store?)

What I want to do: have a shortcode that adds itself to a list/map - and have that map displayed on a page (with a list of all the “things” on one page).

Is there a good way to do it?

I almost know one way: I could use .Site.Store.SetInMap to put things into a map. And I can read stuff out of the map.

But… the thing I don’t know how to do:
Force the page that I want the “result” to be on to be the last one built (so all the other pages have put their stuff into the map first).

As far as I know, there is no way to control the order in which pages are built, so some lucky page is last, and would have the complete list. Other pages will only have the information from the pages that were rendered before it.

Is there either (1) some way to designate a page to be built last (so my scheme of using .Site.Store would work) or (2) some better way to do this?

right from the bat I would go with:

  • a dedicated template for that last page
  • within that template use templates.Defer

@gleicher

The optimal approach may depend on what you are collecting and how you are presenting it.

For example, are you collecting one or more snippets of content? For example:

{{% thing %}}
This is **thing** one on Post 1.
{{% /thing %}}

{{% thing %}}
This is **thing** two on Post 1.
{{% /thing %}}

And when listing these, do you want to include a link to the locations from which they were collected?

@irkode

Thank you!

That’s magic! I was able to get a (miniature) test with defer to work.

Just curious… why did you suggest step 1?
In my test, I did it by hacking the existing “single page” template - replacing the existing {{ .Content }} with:

        {{- if .Params.defer }}
                {{- $ctx := dict "page" . }}
                {{- with (templates.Defer (dict "data" $ctx)) }}
                    {{ .page.Content }}
                    {{warnf "defer: %s" .page.Title}}
                {{- end }}
            {{- else}}
			{{ .Content }}
            {{ warnf "no defer: %s" .Title }}
            {{- end}}

Is there some unintended consequence of this that I am going to discover?

(I did this before the later reply)

Thank you.

Yes, that code is very similar to what I want to do. The first use case is very short snippets of text (so keeping them as strings in a map seems OK).

I was still thinking about the basic version (creating a checklist of all the things over the pages). Links back might be a nice thing in the future.

For now, I think “optimal” is “easiest” - which is pretty short term thinking. (yes, there is a lot of technical debt).

Actually, optimal is easiest. Give me a few minutes to dig up a similar, perhaps identical, use case from the past.

1 Like

Try this:

git clone --single-branch -b hugo-forum-topic-54816 https://github.com/jmooring/hugo-testing hugo-forum-topic-54816
cd hugo-forum-topic-54816
hugo server

Files of interest:

  • layouts/_shortcodes/capture.html
  • layouts/snippets/section.html

The “capture” shortcode uses the page store to hold an array of all snippets captured on the page. Each element in the array is a map which holds an anchor id and the shortcode context.

The “snippets” template ranges over all pages on this site, forces rendering of each one so that the page stores are updated, then lists the snippets with a link to where they were captured. I like this approach because you can easily filter and sort the snippet listing.

2 Likes

Thank you!
I really appreciate your generousity with your time and knowledge.

There is a lot for me to learn from this! I get the basic idea (loop over the pages and look into their individual stores). I will need to read it carefully and think about how this applies. (and a ton of other lessons I am taking from it).

Some quick questions… (good answers lead to better questions)

  1. why the underscore shortcodes “layout/_shortcodes” and not “layout/shortcodes”?
  2. how does the section snippet know not to force itself to render (and cause a loop)?
  3. do the pages have their front matter before they are rendered (so, for example, could I wrap the inside of the loop with something like `{{ if .Params.collectstuff }}`` (so I could manually enable which pages this happens on).

And a bunch of why this and not that…

  1. If I understand correctly, the “layout/snippets/section.html” page is a specialized layout that applies to the “_index.md” page of the snippets section. could this have been a shortcode that was put on some random page in the middle?
  2. Is there some best practice to make sure that this specialized section.html page tracks changes to the “generic” used by default? (or in this case just in layouts)
  3. How would this compare with the defer strategy? It seems less magical (although, the force rendering trick is pretty magical too).

Again, thanks for your generosity. This is really amazingly helpful.

New template system in v0.146.0 and later. See notes.

Yes, but perform a debug.Timer A/B test. I think you’ll find that looking for the existence of a page store is plenty fast, and might be faster than another conditional. I tested HasShortcode and was surprised at the performance difference.

I don’t understand the question.

Yes.

I don’t understand the question.

It’s easier for me to reason about.

always a hundred ways to die :wink: I like @jmooring approach

to answer the question:

I consider that page something special , I would also have expected it’s rather a list page than a normal page in the middle of all the others.
using a dedicated template

  • shows clearly that this is special
  • avoids deadlocks if you don’t set the params correctly when adding a new page
  • makes each template clearer
    no condition to evaluate 100 times for one hit
1 Like

All good reasons! Thank you!

Thank you!

I need to go off and do some learning…

One more note about performance. For the test site I added 2000 pages that don’t call the “capture” shortcode. There’s wasn’t a material difference in build time, meaning that looping over every page in the site didn’t make a lot of difference when compared to just looping over those with page stores.

For a site with a few hundred pages I wouldn’t bother with the A/B testing, unless you have some really nasty shortcodes that do stupid questionable things.

1 Like

Disregard my performance notes above. My test was unrealistic. It is significantly faster to test for HasShortcode before you force content rendering, for example:

  {{ range site.Pages }} {{/* filter and sort as needed */}}
    {{ if .HasShortcode "capture" }}
      {{ $noop := .WordCount }} {{/* force content rendering */}}
      {{ with .Store.Get "things" }}
        <ul>
          {{ range . }}
            <li>
              {{ .ctx.Inner | .ctx.Page.RenderString  }}
              <a href="{{ .ctx.Page.RelPermalink }}#{{ .anchor }}">(go to source)</a>
            </li>
          {{ end }}
        </ul>
      {{ end }}
    {{ end }}
  {{ end }}

I’ve updated the test repository.