Spurious behavior: code highlighter renders bare HTML based on content and percent/angle bracket shortcode

Reproduction repo is https://github.com/agronskiy/hugo-experimentation/tree/main/shortcode-rendering-markdown and I just replicate readme here:

An issue with rendering code in golang highlight rendering which I encountered in one of my projects and boiled down to this minimal example.

➙  hugo version
Hugo Static Site Generator v0.78.2/extended darwin/amd64 BuildDate: unknown

TL;DR: Shorcodes are just rendering the .Inner in three flavours:

  • bare,
  • with markdownify and
  • markdownify | safeHTML ),

and then are called in two flavours

  • as {{% %}}
  • as {{< >}}

The inner code is very simple golang function with a comment.

Problem: In two cases (marked with “problem”) it is rendered with bare HTML artifact, see screenshots.

Funny fact is that if I change the code snippet slightly by removing the blank line, all goes well.


It sounds like an interplay of what highlighter does and what Hugo does when rendering.

Are there any thoughts? Appreciated!
Alex

Thanks for the repo, it makes life so much easier. But I couldn’t find a definite answer :upside_down_face:

The docs say that the {{< >}}-delimited shortcodes, .Inner is interpreted as is, while in the {{% %}}-delimited shortcodes, the delimited part gets rendered first, and then becomes available through .Inner.

That could mean that a call to

{{% innermarkdownify %}}

with layouts/shortcodes/innermarkdownify being

{{ .Inner| markdownify}}

would efectively markdownify twice your enclosed code, the consequences of which are difficult to understand but easy to see.

This theory has many loose ends, hopefully someone else has a better explanation. If no one can come up with an answer you should probably open an issue on Github.

1 Like

On another note, this is not exclusive to newlines before comments. This also happens when you insert any markdown inside your go code after four spaces. I tried

```go
 func f () {

    _hello_ := 2

    * Hello
    * World
}
```

And the result is similar in the cases you marked as problematic.

Thanks, it’s an interesting and insightful explanation.

But from what I understand, the triple-backtick codefence content should be anyway rendered as is, without applying markdown renderer?

To put the insights together and maybe somebody can confirm?

There are three potential moments where markdown inthe shortcode’s content can be rendered:

  1. Before sending to shortcode (before it becomes .Inner)
  2. When one optionally calls markdownify inside
  3. After the content of shortcode is “returned” into the main document.

After @acanalis insights, am I right that: 1. Is controlled by %% vs <> call 2. Is always true 3. Is always false?

Ok actually it sounds like with % % calling, the .Inner is still bare stuff. The rendering happens somewhere after passing it. So the theory is unfortunately wrong.

Below is experiment: I just output what is passed as .Inner as printf, as verbatim and just as {{ .Inner }}.

Check that table is not rendered into HTML at all, only inline elements…

Repo: https://github.com/agronskiy/hugo-experimentation/tree/main/shortcode-rendering-order

Code:

Trying to render table markdown with various inline elements:

### With `% debuginner %`

{{% debuginner %}}

| header with **bold** reference[^1]|

|----|

|cell `with code` |

{{% /debuginner %}}

### With `< debuginner >`

{{< debuginner >}}

| header with **bold** reference[^1] |

|----|

|cell `with code` |

{{< /debuginner >}}

[^1]: test reference

And the Screenshot:

Compete confusion! :frowning:

1 Like

First, thanks @acanalis for hinting in the right directions. I think it would be correct and useful for anybody coming here to summarize:

  1. The assumption that {{% %}} method of calling triggers markdown rendering before passing to shortcode as .Inner is wrong, see post #6 above. In short,

    • .Inner is available as verbose content, no rendering happens when calling the shortcode
    • if one calls markdownify from the context of the shortcode, this can render such things as footnotes wrongly
    • after returning from the shortcode into the calling document, the content is rendered again using the rendering context of the calling document
  2. Spurios rendering of the code snippet which I started this topic with is actually as expected, and as @acanalis put it “difficult to understand but easy to see”. Reason is double-rendering once inside the shortcode and second time outside when returning, and the fact that by coincidence, the comment line of go code is 4-space indented. See updated repo https://github.com/agronskiy/hugo-experimentation/tree/main/shortcode-rendering-order and here’s the screenshot:

Not that if I remove “xxx”, this problem disappears, so this is exactly “difficult to understand” part of the interplay between the renderer and paragraphs preceding the problematic point.

I think this is more or less what we can track here, and I think I’m satisfied by the current understanding.

1 Like

Very interesting.

At this point someone would need to look at the source and see what is actually happening and if it’s a possible pitfall that someone could run into. If anything, the docs could be clearer on the order in which {{% %}} renders markdown.

The current is

Shortcodes using the % as the outer-most delimiter will now be fully rendered when sent to the content renderer (e.g. Blackfriday for Markdown), meaning they can be part of the generated table of contents, footnotes, etc.

When it could be

The output of a shortcode using the % as the outer-most delimiter will be inserted into the calling document, and then be fully rendered when sent to the content renderer (e.g. Blackfriday for Markdown), meaning they can be part of the generated table of contents, footnotes, etc.

(If that’s what it actually does)

1 Like

Damn it, I can’t believe that after all that research it AGAIN turns out that some specific bahavior is related to footnotes! I ran a last check, just added markdownify to the inner and tried to render a test table with a reference, an inline code and inline italic.

Imagine what? Reference renders half the times and does not render another half.

Shortcode code:

What `inner` is actually (debug):
{{ printf "%#v" ( .Inner | markdownify ) }}

Verbatim `inner`
    ```
    {{ ( .Inner | markdownify ) }}
    ```

Now just rendering it:
{{ ( .Inner | markdownify ) }}

Output:

I just give up. Honestly, I don’t see any logic in that.

Now I’m thinking that it (also) has to do with the way that markdown treats html blocks.
This is inserting a footnote is valid in the middle of a normal string (first red circle in your picture) but it’s not between HTML blocks (2nd circle). The rest of the circles… I don’t have a clue.
The worse is that probably this is all expected behavior :upside_down_face: so we are just increasing the entropy of the universe for free

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.