URL/ToC issues when inserting one page into another

support
1

My general use case is that I want to have a repository of “hidden” markdown snippets that I combine into one aggregate markdown page which is shown on the website. For that I have the following setup:

  • A headless page bundle (i.e. a directory) called “snippets” which contains the markdown files I want to combine
  • An include.html shortcode that takes the .Content of a page inside the page bundle and inserts it into a different page
  • An aggregate page that just has a bunch of {{% include "somefile" %}} calls to insert the snippets into one big page

However, I seem to hit a couple of issues that I need help with:

  1. ToC does not get updated with the newly imported headings. AFAIU using % should be the solution to this, but it does not seem to work.
  2. Image references inside the snippets lead to an “Image X is missing and not external for page Y” error, when included in another page. The referenced image does seem to appear on the website (in the example below, at /test_content_dir/snippets/test_bundle/test_png.PNG), and yet I cannot seem to find any src that correctly inserts it into the combined page.

Not sure if these two issues belong in the same post and I can split them if need be, It just seems to me that more context usually helps, they can both be reproduced with the links below, and the way I understand it, they may share a root cause.

I’ve linked a repro below that you can look at, but the relevant files should be:
headless bundle
file inside the bundle with a broken image reference
Page with broken ToC
ToC shortcode
include shortcode

Repro:
This repro uses a fork of the “doks” theme for Hugo with a minimal amount of content files added to reproduce the issue. Steps to reproduce:

  1. npm install (if you have any “package-lock.json was created with an earlier version” issues, you can delete it and reinstall)
  2. Any old npm run start, hugo server, etc. command
  3. You should immediately see the image reference error (“Image test_png.PNG is missing and not external for page test_content_dir\snippets\test_bundle\index.md”)
  4. If you delete the reference to the image inside test_bundle/index.md the site should start and you can navigate to the “Aggregate page” to see that its ToC does not get updated properly with the imported headings.
2

This error is coming from one of Node dependencies, @hyas/images. Your package JSON file looks like:

"@hyas/images": "^0.2.2",

I changed it to the current version:

"@hyas/images": "^0.3.2",

Then I did:

rm -rf node_modules
npm i

The site builds without errors, but I did not spend any time looking at the site.

3

Regarding the TOC problem, you were on the right track with using the {{% %}} notation to call the shortcode, but there’s one piece missing. In the shortcode, you need to render .RawContent not .Content.

Assuming this structure:

content/
├── snippets/
│   ├── index.md
│   ├── snippet-1.md
│   └── snippet-2.md
└── _index.md

And this in your _index.md page:

{{% include "snippet-1.md" %}}
{{% include "snippet-2.md" %}}

Then the TOC is properly generated with a shortcode like:

{{ with $path := .Get 0 }}
  {{ with $.Page.GetPage "snippets" }}
    {{ with .Resources.Get $path }}
      {{ .RawContent }}
    {{ else }}
      {{ errorf "The %s shortcode was unable to get %s. See %s" $.Name $path $.Position }}
    {{ end }}
  {{ else }}
    {{ errorf "The %s shortcode was unable to find the snippets page. See %s" $.Name $.Position }}
  {{ end }}
{{ else }}
  {{ errorf "The % shortcode requires a single positional parameter. See %s" .Name .Position }}
{{ end }}
4

First of all, thank you for the fast and correct replies - you are awesome!

I tested both solutions and I have a couple more things to ask about.
As far as the images go - upgrading the @hyas/images works, but you end up with some path complexity issues. While you would normally just need bundle-relative source (i.e. src="test_png.PNG"), when you include the snippet in another page you need the relative path from the aggregate page inside the snippet page, and you end with something like

inside test_content_dir/snippets/test_bundle

`{{< figure src="../snippets/test_bundle/test_png.PNG" >}}`

This is not optimal, since I’m developing something for other people to use and this is not a very intuitive path structure. Is this the best way to approach this?

As far as the ToC issue, switching from .Content to .RawContent does indeed populate the ToC properly, but it does not render any shortcodes inside it. This can be fixed using | markdownify, but it is not ideal because it leads to some issues with randomly inserted content from shortcodes being treated as markdown, i.e. if a shortcode inside the snippet page produces <tab>SomeInlineHTML, the tab will get markdownified into a code block and SomeInlineHTML will be rendered as text, as opposed to being treated as HTML.

In general, as far as nesting pages into one another, what is the best approach? I originally had it in my head as “the snippet will get fully rendered, then the output of it will get sent to the aggregate page”. The suggestions you are giving me seem to point to the reverse approach - “the raw contents will get concatenated into one page, then rendered as if they were a part of it”). Would this be the preferred way to go about this? Is the other way even possible?

5

So, this is the classic challenge of displaying the content of Page A on Page B, when the link and image destinations in the content of Page A are relative to Page A.

Let’s say your markdown includes:

[Relative destination: another page](another-page)
![Relative destination: image in current page bundle](a.jpg)

Although you could resolve the link to the site root with the relref shorcode…

[Relative destination: another page]({{< relref "another-page" >}})

… the syntax is ugly, the markdown isn’t portable, and the relref shortcode cannot be used with images.

The right way to properly resolve these destinations is with link and image render hooks that use the .GetPage and .ResourcesGet methods (respectively).

In your case, with a shortcode, you’ll need to create your own “figure” shortcode, using the .Resources.Get method to capture the image as a resource, and then use its .RelPermalink method to get the resolved URL.

Something like…

{{ with .Get "src" }}
  {{ with $.Page.Resources.Get . }}
    <figure>
      <img src="{{ .RelPermalink }}" width="{{ .Width }}" height="{{ .Height }}" alt="">
    </figure>
  {{ else }}
    {{ errorf "The %s shortcode was unable to find %s. See %s" $.Name . $.Position }}
  {{ end }}
{{ else }}
  {{ errorf "The %s shortcode requires a parameter named 'src'. See %s" $.Name $.Position }}
{{ end }}

The built-in figure shortcode just uses the destination as-is, with no attempt to resolve it to the site root.
https://github.com/gohugoio/hugo/blob/master/tpl/tplimpl/embedded/templates/shortcodes/figure.html

6

The next Hugo version will improve the ToC situation.

7

Thank you for the responses! Unfortunately, I’m still having trouble with these issues, namely:

  • For the ToC:
    • Using RawContent for the ToC causes any shortcodes from the included page to not get rendered. Using | markdownify on the raw content does not seem to fix this, and even if it did there are some issues with | markdownify that I’ve mentioned above.
    • You’ve mentioned that a new release might help, but the Doks theme which we are using only supports 0.107.0 at the moment, so the upgrade option is quite bothersome.
  • For the images figure shortcode:
    • $.Page.Resources.Get . leads to “…can’t evaluate field Get in type resource.Resources”
    • There don’t seem to be any resources inside Page.Resources (trying to print them in a range results in a no-op). In case it happens to be relevant, my setup consists of a headless leaf bundle, containing other leaf bundles (as linked above), i.e.
|   AggregatePage.md
|   _index.md
|
\---snippets (headless)
    |   index.md
    |
    \---test_bundle (headless)
            index.md
            test_png.PNG

This is not really a typical setup I suppose, but having subdirectories helps sort the snippets, since they might come from different people in different teams, etc., and they all need to be hidden from the production website. Any chance this is the reason why resources are not found?

Tldr: Using RawContent seems to fix the ToC but brings more issues along. Image pathfinding would be a good solution if there were any resources in sight. Any ideas?

8

Quick update:

  • On the missing resources thing - the pages inside the bundle indeed not have resources, but the root-level bundle (“snippets” in this case) does - as long as you get it first, you can get the resources with their relative paths, i.e. “test_bundle/index.md”, so you can consider this fixed
  • On the ToC thing - turns out it’s not that hard to update to the latest version, so @bep how did you mean that the situation will be improved? I assume you meant the page .Fragments feature, but it does not seem to work. Putting something like:
    {{ range .Fragments.Headings }}
    <h3>{{ .Title }}</h3>
    {{ end }}

in the docs-toc.html (from the links at the top), results in an empty array for the page which is dynamically populated, and a correct array of headings for other pages that are more “static”

9

The release is out, so it’s past tense. I said that the “ToC situation will be improved”, which is true. I have not read your post in detail to determine how/what.