Bug: Panic for resources.GetMatch("https://picsum.photos/400/300.webp") on Windows

support
1

Some users reported an error when building my theme on Windows using the exampleSite. The error doesn’t happen on Linux or Mac. I added a Windows runner to repro the error, which looks like this:

WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "paige/bootstrap-icons/bootstrap-icons.css"
WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "paige/bootstrap/bootstrap.css"
WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "paige/bootstrap/bootstrap.bundle.js"
WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "paige/katex/katex.css"
WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "paige/katex/katex.js"
WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "paige/katex/auto-render.js"
WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "https://picsum.photos/400/300.webp"
WARN 2023/02/19 20:32:55 @@@@@@@@@@@@@@@ GetMatch "landscape.webp"
Error: Error building site: "D:\a\paige\paige\exampleSite\content\shortcodes\gallery\index.md:27:1": failed to render shortcode "paige/gallery": failed to process shortcode: execute of template failed: template: shortcodes/paige/gallery.html:22:23: executing "shortcodes/paige/gallery.html" at <partial "paige/func-resource.html" (dict "page" .Page "url" $image)>: error calling partial: "D:\a\paige\paige\layouts\partials\paige\func-resource.html:15:25": execute of template failed: template: partials/paige/func-resource.html:15:25: executing "partials/paige/func-resource.html" at <resources.GetMatch>: error calling GetMatch: runtime error: invalid memory address or nil pointer dereference

Here’s that error split into lines:

Error:
Error building site: "D:\a\paige\paige\exampleSite\content\shortcodes\gallery\index.md:27:1":
failed to render shortcode "paige/gallery":
failed to process shortcode:
execute of template failed:
template: shortcodes/paige/gallery.html:22:23:
executing "shortcodes/paige/gallery.html" at <partial "paige/func-resource.html" (dict "page" .Page "url" $image)>:
error calling partial: "D:\a\paige\paige\layouts\partials\paige\func-resource.html:15:25":
execute of template failed:
template: partials/paige/func-resource.html:15:25:
executing "partials/paige/func-resource.html" at <resources.GetMatch>:
error calling GetMatch:
runtime error: invalid memory address or nil pointer dereference

To clarify that a bit, paige\exampleSite\content\shortcodes\gallery\index.md:27:1 is the first line of this:

{{< paige/gallery >}}
    {{< paige/gallery image="https://picsum.photos/400/300.webp"  />}}
{{< /paige/gallery >}}

I assume the actual line in question is the second line containing https://picsum.photos/400/300.webp. (This seems like a bug in line reporting.)

shortcodes/paige/gallery.html:22:23 is the first line of

        {{ $resource = partial "paige/func-resource.html" (dict
            "page" .Page
            "url" $image
        ) }}

paige\layouts\partials\paige\func-resource.html:15:25 is the second line of

        {{ warnf "@@@@@@@@@@@@@@@ GetMatch %#v" $url }}
        {{ with resources.GetMatch $url }}

As you saw in the raw output at the top, the value of $url was (probably) https://picsum.photos/400/300.webp (the last value printed was technically landscape.webp, but the picsum URL matches the outermost context; perhaps the landscape string was from parallel work?).

The exact source is at GitHub - willfaught/paige at a62912535e57532a5da5d4743ae2d743e81a245b.

So, it seems to me that resources.GetMatch("https://picsum.photos/400/300.webp") works on Linux and Mac (returning nil), but not on Windows (panics instead of returning nil), unless I’m missing something. Certainly, a panic shouldn’t be happening at all. I’m not sure how to debug this any further, since I don’t have access to a Windows computer right now.

I tried checking for remote resources first, so that a URL is never given to GetMatch, but it still results in a panic:

Error: Error building site: "D:\a\paige\paige\exampleSite\content\shortcodes\gallery\index.md:27:1": failed to render shortcode "paige/gallery": failed to process shortcode: execute of template failed: template: shortcodes/paige/gallery.html:49:23: executing "shortcodes/paige/gallery.html" at <partial "paige/img.html" (dict "class" "img-fluid" "height" $height "maxheight" $maxheight "maxwidth" $maxwidth "method" $method "options" $options "resource" $resource "src" $image "width" $width)>: error calling partial: "D:\a\paige\paige\layouts\partials\paige\img.html:50:35": execute of template failed: template: partials/paige/img.html:50:35: executing "partials/paige/img.html" at <$resource.Height>: error calling Height: runtime error: invalid memory address or nil pointer dereference

This time, it makes it a bit further down gallery.html to shortcodes/paige/gallery.html:49:23, which is the first line of:

              "content" (partial "paige/img.html" (dict
                "class" "img-fluid"
                "height" $height
                "maxheight" $maxheight
                "maxwidth" $maxwidth
                "method" $method
                "options" $options
                "resource" $resource
                "src" $image
                "width" $width
            ))

paige\layouts\partials\paige\img.html:50:35 is

{{ $intrinsicheight = $resource.Height }}

Here, $resource is passed in as a param from gallery.html, where it was gotten from the paige/func-resource.html partial that was causing the first problem above. So, by avoiding passing the URL to GetMatch, we seem to have just kicked the can down the road a little. The remote resource is still causing a panic, this time when trying to use its height.

The exact source for the remote resource workaround is at GitHub - willfaught/paige at e6b4ae431937d9ed075d057f4694b143b623ac47.

Any help is much appreciated!

Edit 1: Hugo 0.102.3 is used in the Windows GitHub runner.

Edit 2: Cleaned up copied log output.

Edit 3: Note that --verbose --verboseLog --log unfortunately didn’t add any extra output for debugging.

Edit 4: Updated commit for new windows branch.

Edit 5: Add remote resource workaround commit link.

2

You have exampleSite/content/content (nested content directory). To try out a theme most people do (on Windows):

cd themes\foo\exampleSite
hugo --theme ..\..

When I try this on Windows…

hugo new site mysite
cd mysite
git init
git clone --recurse-submodules https://github.com/willfaught/paige themes/paige
cd themes/paige/exampleSite
hugo --theme ..\..

hugo v0.110.0-DEV-f95fd57aaccb999b6fefe7b2ac9239a6c3325b0f+extended windows/amd64 BuildDate=2023-01-02T16:35:08Z 
Error: Error building site: "C:\temp\mysite\themes\paige\exampleSite\content\shortcodes\gallery\index.md:27:1": failed to render shortcode "paige/gallery": failed to process shortcode: execute of template failed: template: shortcodes/paige/gallery.html:49:23: executing "shortcodes/paige/gallery.html" at <partial "paige/img.html" (dict "class" "img-fluid" "height" $height "maxheight" $maxheight "maxwidth" $maxwidth "method" $method "options" $options "resource" $resource "src" $image "width" $width)>: error calling partial: execute of template failed: template: partials/paige/img.html:30:23: executing "partials/paige/img.html" at <partial "paige/func-resource.html" (dict "page" $page "url" $src)>: error calling partial: "C:\temp\mysite\themes\paige\layouts\partials\paige\func-resource.html:14:25": execute of template failed: template: partials/paige/func-resource.html:14:25: executing "partials/paige/func-resource.html" at <resources.GetMatch>: error calling GetMatch: runtime error: invalid memory address or nil pointer dereference

rename content content-bad
move content-bad\content .
hugo --theme ..\..

no errors
3

hugo --theme …..

@jmooring Do you mean --themesDir?

The issue (created Jan 10) predates the use of content/content, which used to be content/blog.

I renamed content/content to content/foo, but the same error happens:

Error:
Error building site:
"D:\a\paige\paige\exampleSite\content\shortcodes\gallery\index.md:27:1":
failed to render shortcode "paige/gallery":
failed to process shortcode:
execute of template failed:
template:
shortcodes/paige/gallery.html:49:23:
executing "shortcodes/paige/gallery.html" at <partial "paige/img.html" (dict "class" "img-fluid" "height" $height "maxheight" $maxheight "maxwidth" $maxwidth "method" $method "options" $options "resource" $resource "src" $image "width" $width)>:
error calling partial:
"D:\a\paige\paige\layouts\partials\paige\img.html:50:35":
execute of template failed:
template:
partials/paige/img.html:50:35:
executing "partials/paige/img.html" at <$resource.Height>:
error calling Height:
runtime error:
invalid memory address or nil pointer dereference

I don’t understand how content/content could cause this issue. Note that moving content/content to content disables content\shortcodes\gallery\index.md, which is the file where this error is originating.

Edit: Added commit link.

4

No, I do not.

If you want people to build your example site w/o error, create an example site that users expect.

5

I don’t follow. --theme is for specifying a theme by name. This is the hugo -h doc for it:

-t, --theme strings              themes to use (located in /themes/THEMENAME/)

The way to use it for my theme would be hugo --theme paige, but that assumes that the theme is in the exampleSite/themes directory, which it isn’t. You have to use --themesDir ../.. in exampleSite for theme: paige in exampleSite/config.yaml to work. This is what I use locally, and what the theme’s GitHub Action uses. This has always worked on Linux and Mac. Does it work differently on Windows?

6

In case it isn’t clear, it’s not a bug that content/content exists; it’s one of the sections, alongside layouts and shortcodes sections. See the menu in No Borders, No Limits for an example.

7

Linux

hugo new site mysite
cd mysite
git init
git clone --recurse-submodules https://github.com/willfaught/paige themes/paige
cd themes/paige/exampleSite
hugo --theme ..\..

Error: module “…” not found; either add it as a Hugo Module or store it in “/home/jmooring/temp/mysite/themes/paige/exampleSite/themes”.: module does not exist
Total in 0 ms

mv content/ content-bad
mv content-bad/content/ .
hugo --theme ../..

                   | EN   
-------------------+------
  Pages            |  38  
  Paginator pages  |   0  
  Non-page files   |   0  
  Static files     | 149  
  Processed images |   0  
  Aliases          |  12  
  Sitemaps         |   1  
  Cleaned          |   0  

Total in 126 ms
8

Disregard previous comment about Linux testing. Bad slash.

9

On Mac, this works:

hugo new site mysite
cd mysite
git init
git clone --recurse-submodules https://github.com/willfaught/paige themes/paige
cd themes/paige/exampleSite

On Mac, both --theme ../.. and --themesDir ../.. do the same thing:

~/tmp/mysite/themes/paige/exampleSite master
❯ hugo --theme ../..
Start building sites … 
hugo v0.110.0+extended darwin/amd64 BuildDate=unknown

                   | EN   
-------------------+------
  Pages            |  73  
  Paginator pages  |   0  
  Non-page files   |  39  
  Static files     | 149  
  Processed images |  39  
  Aliases          |  25  
  Sitemaps         |   1  
  Cleaned          |   0  

Total in 2638 ms

~/tmp/mysite/themes/paige/exampleSite master
❯ hugo --themesDir ../..
Start building sites … 
hugo v0.110.0+extended darwin/amd64 BuildDate=unknown

                   | EN   
-------------------+------
  Pages            |  73  
  Paginator pages  |   0  
  Non-page files   |  39  
  Static files     | 149  
  Processed images |  39  
  Aliases          |  25  
  Sitemaps         |   1  
  Cleaned          |   0  

Total in 195 ms

In my opinion, the doc says the former is incorrect. Anyway, we’re getting the same results regardless of the option used.

mv content/ content-bad
mv content-bad/content/ .

As I wrote above, this is removing the file in which the Windows error originates from rendering, so it’s no surprise that doing this “removes” the error.

10

First, your example site directory is confusing. I see now what you mean. But when I look in content and see what looks like a site structure, well… Maybe I didn’t look at it as carefully as I should have, but I wouldn’t be surprised if others were similarly confused.

Second, you need to refactor func-resource.html so that you’re not passing a URL to a method that expects a path.

{{ $u := $url | urls.Parse }}

{{ if $u.IsAbs }}
    {{ with resources.GetRemote $u.String }}
        {{ $result = . }}
    {{ end }}
{{ else }}
    {{ with $page.Resources.GetMatch $u.Path }}
        {{ $result = . }}
    {{ else }}
        {{ with $page.Resources.Get $u.Path }}
            {{ $result = . }}
        {{ else }}
            {{ with resources.GetMatch $u.Path }}
                {{ $result = . }}
            {{ else }}
                {{ with resources.Get $u.Path }}
                    {{ $result = . }}
                {{ end }}
            {{ end }}
        {{ end }}
    {{ end }}
{{ end }}

It aslo wouldn’t hurt if the remote call had some error or warning handlers, something like:

{{ $r := "" }}
{{ $u := urls.Parse "https://example.org/kitten.jpg" }}

{{ if $u.IsAbs }}
  {{ with resources.GetRemote $u.String }}
    {{ with .Err }}
      {{ errorf "%s" . }}
    {{ else }}
      {{ $r = . }}
    {{ end }}
  {{ else }}
    {{ errorf "Unable to get remote resource %q" $u.String }}
  {{ end }}
{{ end }}
11

First, your example site directory is confusing. I see now what you mean. But when I look in content and see what looks like a site structure, well… Maybe I didn’t look at it as carefully as I should have, but I wouldn’t be surprised if others were similarly confused.

The only thing that’s perhaps unusual about it is the name of the “content” section. If you can think of a better word for the typical Hugo demo pages, I’m all ears. Everything else is normal, it seems to me. Perhaps if the only thing in /content was content/, that might throw people off, but there’s other stuff too. The original issue report even showed the user doing

cd exampleSite
hugo --themesDir ../..

which is correct. Anyway, it seems now we both think this isn’t the source of the error.

Second, you need to refactor func-resource.html so that you’re not passing a URL to a method that expects a path.

As I wrote in the OP, I did try this:

I tried checking for remote resources first, so that a URL is never given to GetMatch, but it still results in a panic:

It prevented a panic from happening in GetMatch, but a panic happened later when the resource .Height was accessed. A panic shouldn’t be happening in GetMatch, regardless of the input.

(As an aside, it would be much more helpful if the panic stack trace is printed.)

It aslo wouldn’t hurt if the remote call had some error or warning handlers

Good point, however, in this case, that isn’t the source of the error.

12

On Windows, I started from scratch.

After cloning your repo I changed one file — func-resource.html.

I pasted in what I previously posted.

Then…

C:\temp\mysite\themes\paige\exampleSite> hugo --theme ..\..

                   | EN
-------------------+------
  Pages            |  73
  Paginator pages  |   0
  Non-page files   |  39
  Static files     | 149
  Processed images |  39
  Aliases          |  25
  Sitemaps         |   1
  Cleaned          |   0

Running from CMD.

13

Your solution fails on the Windows runner, unfortunately (using hugo v0.110.0-e32a493b7826d02763c3b79623952e625402b168 windows/amd64).

I pasted your code:

{{ $params := . }}

{{ $page := $params.page }}
{{ $url := $params.url }}

{{ $result := "" }}

{{ $u := $url | urls.Parse }}

{{ if $u.IsAbs }}
    {{ with resources.GetRemote $u.String }}
        {{ $result = . }}
    {{ end }}
{{ else }}
    {{ with $page.Resources.GetMatch $u.Path }}
        {{ $result = . }}
    {{ else }}
        {{ with $page.Resources.Get $u.Path }}
            {{ $result = . }}
        {{ else }}
            {{ with resources.GetMatch $u.Path }}
                {{ $result = . }}
            {{ else }}
                {{ with resources.Get $u.Path }}
                    {{ $result = . }}
                {{ end }}
            {{ end }}
        {{ end }}
    {{ end }}
{{ end }}

{{ if not $result }}
    {{ errorf "invalid resource: %q" $url }}
{{ end }}

{{ return $result }}
14

I don’t know what to tell you. Works for me. Windows 11.

15

Thanks for helping to debug. I’ll ship a new theme version with these changes and see if the issue reporter can still repro the error.

16

Looks like that probably works as a workaround. Thanks again, @jmooring!

17

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.