"Panic" with JSON and workaround, but better to fix it

support
1

I am converting huge 10,000-50,000 articles websites to Hugo, and of course some pages have errors and I can find page and fix. But from time to time I have just “Panic” and there is no report which page breaks the build.

I found that in my case it happens 100% time with malformed ````json` - not closing end tag for example, or not closing properly, or in some cases JSON content inside is malformed. In case of “panic”, I simply remove half of pages, then start again, then remove half and start again, Log(n), until I find suspicious page. It is not fast, especially if I have 100 such faulty pages.

Fortunately, I found it is related to malformed ````json` (or, [```json] in case if formatting here malformed; code snippet) => so that I can search in IDE patterns such as [``\n] (two ticks instead of three) and so on. It takes hours.

WOuld be nice if Hugo reports such issues politely instead of panic.

2

Can you provide a simple and explicit example that we can use to reproduce the problem?

5

Here is the file; malformed JSON after Mermaid; if I remove “quizzes” section issue disappears; it is really hard to reproduce, complex page needed, but what I noticed it is always malformed JSON or sometimes (maybe) two backticks instead of three.
https://raw.githubusercontent.com/FuadEfendi/gohugo_bug_report/main/index.md

ANd this is “panic”, I use most recent Hugo (upgraded few days ago):

funnyfigures.com git:(main) ✗ hugo server                                               
port 1313 already in use, attempting to use an available port
Watching for changes in /Users/fefendi/java/git/tokenizer-inc-canada/funnyfigures.com/{archetypes,assets,content,layouts,package.json}
Watching for config changes in /Users/fefendi/java/git/tokenizer-inc-canada/funnyfigures.com/config/_default, /Users/fefendi/java/git/tokenizer-inc-canada/funnyfigures.com/config/development, /Users/fefendi/java/git/tokenizer-inc-canada/funnyfigures.com/go.mod
Start building sites … 
hugo v0.126.1+extended darwin/arm64 BuildDate=2024-05-15T10:42:34Z VendorInfo=brew

panic: unknown shortcode token "HAHAHUGOSHORTCODE</span><span class=\"mi\">6</span><span class=\"err\">s</span><span class=\"mi\">0</span><span class=\"err\">HBHB" (number of tokens: 1)

goroutine 1593 [running]:
github.com/gohugoio/hugo/hugolib.(*cachedContent).contentRendered.func1.3({0x108b29750, 0x14003e7b170}, {0x14005b58480, 0x7a})
        github.com/gohugoio/hugo/hugolib/page__content.go:572 +0x17c
github.com/gohugoio/hugo/hugolib.expandShortcodeTokens({0x108b29750, 0x14003e7b170}, {0x14001e76000, 0x6889, 0x8000}, 0x14000dabbb8)
        github.com/gohugoio/hugo/hugolib/shortcode.go:738 +0x194
github.com/gohugoio/hugo/hugolib.(*cachedContent).contentRendered.func1({0x14000a34758?, 0x108b54e20?})
        github.com/gohugoio/hugo/hugolib/page__content.go:575 +0x388
github.com/bep/lazycache.(*Cache[...]).GetOrCreate(0x108b4ab80, {0x140038ca7e0, 0x63}, 0x14003cbdd78)
        github.com/bep/lazycache@v0.4.0/lazycache.go:125 +0x13c
github.com/gohugoio/hugo/cache/dynacache.(*Partition[...]).doGetOrCreate(...)
        github.com/gohugoio/hugo/cache/dynacache/dynacache.go:400
github.com/gohugoio/hugo/cache/dynacache.(*Partition[...]).GetOrCreate(0x108b4f7e0, {0x140038ca7e0, 0x63}, 0x14000dabd78)
        github.com/gohugoio/hugo/cache/dynacache/dynacache.go:388 +0x44
github.com/gohugoio/hugo/hugolib.(*cachedContent).contentRendered(0x14000efe9f0, {0x108b29750?, 0x14003b6dad0?}, 0x140010ec980)
        github.com/gohugoio/hugo/hugolib/page__content.go:510 +0x134
github.com/gohugoio/hugo/hugolib.(*cachedContent).contentPlain.func1({0x14000a347b8?, 0x108b54be0?})
        github.com/gohugoio/hugo/hugolib/page__content.go:763 +0xf0
github.com/bep/lazycache.(*Cache[...]).GetOrCreate(0x108b4aa00, {0x140038ca770, 0x63}, 0x14003e7b110)
        github.com/bep/lazycache@v0.4.0/lazycache.go:125 +0x13c
github.com/gohugoio/hugo/cache/dynacache.(*Partition[...]).doGetOrCreateWitTimeout.func1()
        github.com/gohugoio/hugo/cache/dynacache/dynacache.go:423 +0x50
created by github.com/gohugoio/hugo/cache/dynacache.(*Partition[...]).doGetOrCreateWitTimeout in goroutine 1592
        github.com/gohugoio/hugo/cache/dynacache/dynacache.go:422 +0xf4
6

Can you reduce this down to something even shorter? I doubt we need this much markdown to reproduce the problem.

7

And this is the issue, when I reduce the issue diappears

8

I got it, reduced, when I remove “quizdown” (or, when I fix malformed JSON) issue disappears:

---

---

```json
{
   myObject = [
      "abcd,
      "efgh"
   ]
}
``

## abcsd

{{< quizdown >}}

{{< /quizdown >}}

Source file:
https://raw.githubusercontent.com/FuadEfendi/gohugo_bug_report/main/index.md

9

The simplest reproducer is:

```json
{{< sc >}}
```

Error: error building site: render: failed to render pages: render of “home” failed: “/home/jmooring/code/hugo-testing/layouts/_default/home.html:11:5”: execute of template failed: template: _default/home.html:11:5: executing “main” at <.Content>: error calling Content: unknown shortcode token “HAHAHUGOSHORTCODE<span style="color:#ae81ff">1<span style="color:#960050;background-color:#1e0010">s<span style="color:#ae81ff">0<span style="color:#960050;background-color:#1e0010">HBHB” (number of tokens: 1)

Your example, with an un-closed fenced code block, is equivalent to the above. The parser doesn’t read ahead to look for a matching closer (see this thread for details), and your markdown is valid despite the lack of a matching closer.

There may be other highlighting languages that trigger the same problem. I tested about 20 other languages, including some likely suspects, but was unable to reproduce the problem.

This isn’t a problem if you call the shortcode using the {{% %}} notation.

Since your markdown is valid, you can’t use a markdown linter to find un-closed fenced code blocks. So, as you have found, your best option is a regex search (e.g., ^``$).

I’ll log a bug for this, but it is so edge-casey I wouldn’t be surprised if it lingers in the issue queue until it dies of neglect.

https://github.com/gohugoio/hugo/issues/12567

1 Like
10

I am not about markdown validation, but about “Panic” without reporting where exactly it happened (at least, file name, no need exact line number); trying to find single file causing that when you have 10,000+ files is pain… trying to find 100 such files is 100*“Log(n)” more pain :wink: considering that with disabled taxonomy server restart takes minutes…

11

Do you suggest to use {{% quizdown %}} instead of {{< quizdown >}} for shortcode? It might be best solution, I’ll try; thank you very much!

12

No, I don’t think you can change the shortcode notation in this case.

Getting the filename with or without position is probably a lot more difficult that resolving the underlying problem, which is why the logged issue is about the underlying problem.

1 Like
13

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.