.Scratch in partial called from shortcode

support
1

trying to collate random shortcode related js from all site pages. Plan is to use page .Scratch to store js file content referenced by an arbitrary set of partials. These are then collated together across site and dumped into a single file from a partial in the _default/baseof.html.

Problem is that on site compilation, the shortcodes are processed after the partial that collates all the site.AllPages.Scratch. Then the pages are processed but the scratch at this point is useless.

Is there a way to force the partial called in the template to be processed after all the pages are processed? Can it be added to a future update? perhaps defer processing of a partial if it reads .Scratch till after all partials that write to .Scratch are processed?

2

If I correctly understand what you are trying to do, you need to force the page to render before accessing the scratch.

content/post/post-1.md

{{< my-shortcode >}}

layouts/shortcodes/my-shortcode.html

{{ partial "my-partial.html" . }}

layouts/partials/my-partial.html

{{ .Page.Scratch.Set "foo" "bar" }}

layouts/_default/baseof.html

{{ $noop := .Content }} {{/* IMPORTANT */}}
{{ .Scratch.Get "foo" }}

Note that this will produce consistent results when building the site with hugo, but may produce inconsistent results when running hugo server. See:
https://github.com/gohugoio/hugo/issues/8863

2 Likes
3

thanks for the prompt reply. I’ll try to wrap my head around this and get back to you

4

.Scratch compilation issue

Here’s a detailed overview of the problem I’m trying to wrap up: a partial called from a page template is compiling before all the content pages are processed. This means that any js added to individual page scratch will not make it into the js file.

Repeating the conclusion in lieu of a summary:

As can be seen from the warnf output in the compile log, the gallery.js is put in the .page.Scratch after the partials/utils/collate-code.gtml is processed.

Possible solution: defer any code that reads .Scratch from outside the page context until all site-wide .Scratch writes are done. Writing from site context to a .page.Scratch should cause an error, thus reducing the chance of a race condition.

Usefulness: It allows ‘dynamic’ bubbling of content related features to bubble up to the site templates.

Feasible? I don’t know, but it would be great to have.

_default/baseof.html

The default page calls a partial that compiles .scratch from all content pages and dumps it into a js file.

<!DOCTYPE html>
{{ $opts := ( dict ) }}
<html lang="{{ $.Site.Language.Lang }}">
{{ $opts := ( dict "path" "boot/" "js_opts" $opts )}}
{{ partial "utils/set-code.gtml" $opts }}
<head>{{ partial "page/site/head/c.gtml" . }}</head>
<body class='noScript' >
{{ block "header" . }}{{ partial "page/site/header/c.gtml" . }}{{end}}
{{ block "main" . }}{{end}}
{{ block "footer" . }}{{ partial "page/site/footer/c.gtml" . }}{{end}}

{{ /* ################# */}}
{{ partial "utils/collate-code.gtml" ( dict "path" "js/lib.js" ) }}
{{ /* ################# */}}


  </body>
</html>

collate-code.gtml

This code is called from baseof.html and goes through the whole site looking for pages with .Scratch that contains javascript code. It is collated and saved with a link calling it back.

Problem: all the page .Scratch are empty as this code is called before the pages are processed.

{{ $options := dict }}
{{ $lib := dict }}

{{ warnf "## collate-code: %s" site.Title }}
{{ warnf "## collate-code: %+v" site.Params.js }}

{{ range site.AllPages }}
{{ $code := printf "console.log('testing embedded js in page %s');" .File.Path }}
{{ $code = dict .URL $code }}
{{ warnf "## collate-code: %s" $code }}
{{ $lib = merge $lib $code }}
{{ with .Scratch.Get "js" }}
  {{ $lib = merge $lib . }}
  {{ warnf "## collate-code: %+v" . }}
{{ end }}
{{ end }}

{{ $options = merge $options ( dict "targetPath" $_path "minify" "true" ) }}
{{ $lib := delimit $lib  "\n\n/*===*/\n" | resources.FromString $_path }}
{{ warnf "## collate-code: js code::\n%s" $lib.Content }}
{{ $lib := $lib | js.Build $options }}
<script  type="module" src="{{ $lib.Permalink }}"></script>

content page

This page calls a shortcode that displays a custom picture gallery. The shortcode and partial can be found immediately below.

---
categories:
  - error-pages
  - long-form
date: 2020-08-04
lastmod: 2020-08-04
description: Bork Mart has been providing quality services since 2001
title: "we're serving peanuts today!"

hero_pic: hero

boot_js:
  - stacks/gallery

debugger: none

---

# This server cannot find the page you're looking for.

Check whether there is a typo in the address requested.

## Here goes nothing.

{{< stacks/gallery/c
  hed="optional headline for gallery section"
  path="Yong" >}}

picture gallery

{{< /stacks/gallery/c >}}

Hackathon value backing innovator release management mvp non-disclosure
rate stock, crowdfunding proposition lean influencer deployment stealth
validation handshake learning, infrastructure advisor on customer prototype
client angel burn, shift mailing plan alpha chain channels marketing churn.
Validation adopters technology proof investment success party niche startup
ramen, market business backing direct monetization customer hackathon fruit
facebook, beta crowdsource equity effects venture disruptive tail deployment
ecosystem, ownership entrepreneur proposition business-to-consumer scrum
development virality mover. Investor deployment shift rockstar technology
advisor on focus curve, testing learning paradigm rate leverage startup
marketing crowdsource, infographic sales ownership proposition accelerator
effects backing ramen entrepreneur, channels grail canvas seed return alpha
hacking.

shortcodes/stacks/gallery/c.html

Shortcode that passes page reference and pics to gallery building partial.

{{ $_pics := ( .Get "path" ) | default "pic" }}
{{ $_class := ( .Get "class" ) | default "up" }}

{{ warnf "$$$$ stacks/gallery/ %v" $.Page.File.Path }}

{{ /* ################# */}}
{{ $pics := partial "utils/get_resources.gtml" ( dict "page" $.Page "resource" $_pics "type" "image" ) }}
{{ /* ################# */}}
{{ if $pics }}
  {{ partial "stacks/gallery/c.gtml" ( dict "stack" $pics "class" $_class "page" . ) }}
{{ else }}
  {{ errorf "Failed to get pictures for gallery at assets/*%q*" $_pics }}
{{ end }}

{{ with .Inner }}
  {{ . | markdownify }}
{{ end }}

partials/stacks/gallery/c.gtml

partial that build html and adds any related js code to page’s .Scratch.

{{ $_id := .id | default "gallery" }}
{{ $_class := .class }}
{{ $_page := .page }}

{{ warnf "## stacks/gallery:: processing %v" $_page }}

<section id="{{ $_id }}" class="{{ $_class }} by-{{ $_max_pix }}">
<ul class="pics {{ $_class }}">
{{ range .stack }}
<li class="thumb">[ ... ]</li>
{{ end }}
</ul>
</section>

{{ /* ################# */}}
{{ $opts := ( dict "path" "stacks/gallery/" "page" $_page ) }}
{{ partial "utils/get-code.gtml" $opts }}
{{ /* ################# */}}

{{ with $_page }}
{{ warnf "## stacks/gallery:: page scratch js:\n%v" (.Scratch.Get "js") }}
{{ end }}
<script> [ ... ] </script>

partials/utils/get-code.gtml

Partial getting js code from disk and putting it into page .Scratch

{{ $_path := .path }}
{{ $_name := .name | default "code.js" }}
{{ $_page := .page }}
{{ warnf "## get-code:: processing %s" $_path }}
{{ $code :=  path.Join "/partials" $_path $_name }}
{{ $code := ( resources.Get $code ).Content }}
{{ warnf "## get-code:: %s" $code }}

{{ with $_page }}
{{ .Scratch.SetInMap "js" $_path $code }}
{{end}}

abbreviated debugging logs

output in js/lib.js

(()=>{
  console.log("testing embedded js in page _index.md");
  console.log("testing embedded js in page about/index.md");
  console.log("testing embedded js in page services/_index.md");
  console.log("testing embedded js in page services/act/index.md");
  console.log("testing embedded js in page services/craft/_index.md");
  console.log("testing embedded js in page services/store/_index.md");
  console.log("testing embedded js in page contact/index.md");
  console.log("testing embedded js in page err/_index.md");
  console.log("testing embedded js in page err/404/index.md");
  console.log("testing embedded js in page err/409/index.md");
  console.log("testing embedded js in page err/500/index.md");
  console.log("testing embedded js in page err/607/index.md");
  console.log("testing embedded js in page err/crop_test/index.md");
  console.log("testing embedded js in page facilities/_index.md");
  console.log("testing embedded js in page facilities/internal/index.md");
  console.log("testing embedded js in page facilities/prex/index.md");
  console.log("testing embedded js in page facilities/tox/index.md");
  console.log("testing embedded js in page impressum/_index.md");
  console.log("testing embedded js in page impressum/copyright/index.md");
  console.log("testing embedded js in page impressum/legal/index.md");
  console.log("testing embedded js in page impressum/privacy/index.md");
  console.log("testing embedded js in page impressum/sitemap/index.md");
  console.log("testing embedded js in page testing/index.md");
  document.addEventListener("DOMContentLoaded", m=>{
    console.log("DOM fully loaded and parsed"),
    (i=>{  /* bunch of events */  })(MENU)
  }
  );
  /* missing a similar function called by a content page shortcode building the gallery here */
}
)();

debugging output in CLI:

WARN 2021/10/30 13:21:49 ## collate-code: map[/err/crop_test/:console.log('testing embedded js in page err/crop_test/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/err/404/:console.log('testing embedded js in page err/404/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/err/409/:console.log('testing embedded js in page err/409/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/err/500/:console.log('testing embedded js in page err/500/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/err/607/:console.log('testing embedded js in page err/607/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/err/:console.log('testing embedded js in page err/_index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/:console.log('testing embedded js in page _index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/about/:console.log('testing embedded js in page about/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/services/craft/:console.log('testing embedded js in page services/crafts/_index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/services/:console.log('testing embedded js in page childcare/_index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/contact/:console.log('testing embedded js in page contact/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/impressum/copyright/:console.log('testing embedded js in page impressum/copyright/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/facilities/:console.log('testing embedded js in page facilities/_index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/impressum/:console.log('testing embedded js in page impressum/_index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/impressum/legal/:console.log('testing embedded js in page impressum/legal/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/impressum/privacy/:console.log('testing embedded js in page impressum/privacy/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/impressum/sitemap/:console.log('testing embedded js in page impressum/sitemap/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/services/story-telling/:console.log('testing embedded js in page childcare/story telling/_index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/impressum/legal/:console.log('testing embedded js in page impressum/legal/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/testing/:console.log('testing embedded js in page testing/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: map[/testing/:console.log('testing embedded js in page testing/index.md');]
WARN 2021/10/30 13:21:49 ## collate-code: js code::
console.log('testing embedded js in page _index.md');
console.log('testing embedded js in page about/index.md');
console.log('testing embedded js in page services/_index.md');
console.log('testing embedded js in page services/act/index.md');
console.log('testing embedded js in page services/craft/_index.md');
console.log('testing embedded js in page services/store/_index.md');
console.log('testing embedded js in page contact/index.md');
console.log('testing embedded js in page err/_index.md');
console.log('testing embedded js in page err/404/index.md');
console.log('testing embedded js in page err/409/index.md');
console.log('testing embedded js in page err/500/index.md');
console.log('testing embedded js in page err/607/index.md');
console.log('testing embedded js in page err/crop_test/index.md');
console.log('testing embedded js in page facilities/_index.md');
console.log('testing embedded js in page facilities/internal/index.md');
console.log('testing embedded js in page facilities/prex/index.md');
console.log('testing embedded js in page facilities/tox/index.md');
console.log('testing embedded js in page impressum/_index.md');
console.log('testing embedded js in page impressum/copyright/index.md');
console.log('testing embedded js in page impressum/legal/index.md');
console.log('testing embedded js in page impressum/privacy/index.md');
console.log('testing embedded js in page impressum/sitemap/index.md');
console.log('testing embedded js in page testing/index.md');
document.addEventListener('DOMContentLoaded', (event) => {
  console.log('DOM fully loaded and parsed');
  ( menu_id => {  ...  } );
} ) ( MENU )
});
DEBUG 2021/10/30 13:21:49 Render page 404: page not found to "/err/404/index.html"
[ ... ]
DEBUG 2021/10/30 13:22:36 Render page assorted error pages to "/err/index.html"
WARN 2021/10/30 13:22:37 $$$$ stacks/gallery/ _index.md
WARN 2021/10/30 13:22:37 setting up simple menu %!s(int64=1635592957)
WARN 2021/10/30 13:22:39 ## get-code:: processing stacks/gallery/
WARN 2021/10/30 13:22:39 ## get-code:: document.addEventListener('DOMContentLoaded', (event) => {
  console.log('DOM fully loaded and parsed, setting events for gallery');
  ( gallery_id => { [...] } ) ( GALLERY )
});
WARN 2021/10/30 13:22:39 ## stacks/gallery:: page scratch js:
map[stacks/gallery/:document.addEventListener('DOMContentLoaded', (event) => {
  console.log('DOM fully loaded and parsed, setting events for gallery');
  ( gallery_id => {  [...]  } ) ( GALLERY )
});
]

conclusion

As can be seen from the warnf output in the above compile log, the gallery js is put in the page’s .Scratch after the collate-code partial is processed.

Possible solution: defer any code that reads .Scratch from outside the page context until all site-wide .Scratch writes are done. Writing from site context to a .page.Scratch should cause an error, thus reducing the chance of a race condition.

Usefulness: It allows ‘dynamic’ bubbling of content related features to bubble up to the site templates.

Feasible? I don’t know, but it would be great to have.

Parting shot: had to get this out of my system or I wouldn’t be able to let it go. Hope whoever reads till the end finds something useful here.

5

There may be more than one issue here, but there’s no way this is going to work unless you…

6

I did add the.content bit as described to the template but it didn’t work.

The template partial still got processed before the page shortcode.

I’ll try it again just in case.

7

Here’s a simplified example.

git clone --single-branch -b hugo-forum-topic-35379 https://github.com/jmooring/hugo-testing hugo-forum-topic-35379
cd hugo-forum-topic-35379
hugo server
1 Like
8

tried it again just to make sure. The debugging CLI code is coming out the same. Adding the $noop:=.Content before doesn’t force the page content to render before the rest of the template.

9

Can you share your project?

10

a bit hard as there’s a lot of experimental gunk in it.

11

Without seeing your code base, its a bit hard to identify the problem. The only other thing I can suggest is to examine the simplified example that I provided, and to see how it differs from your project. I’m sure I missed some key details, but I think the concept is essentially the same.

12

I’ll try to fit in the above in a simple template. Will send you feedback on your example later today.

Thanks

13

ok. First off thanks for taking the time to try and understand my conundrum. Your solution on github only covers half my example. It’s the half that works. Calling a partial from the master templates is working. The problem resides with a shortcode that is called from the contents and updates the page scratch AFTER the aforementioned template partial is called.

I am suggesting that partials that access site-wide scratch should be evaluated at the end of the processing pipeline. Currently partials called from templates are evaluated first no matter what. THis results in the partial reading page.Scratch BEFORE it is updated by the content shortcode.

14

Maybe a shorter description: the site compiler should resolve all scratch variables in the content to static data before compiling templates.

As things stand, templates seem to get compiled first and then the bits related to content are processed and filled into the template placeholders.