Panic: runtime error when using garbage collector

support
1

I’m struggling with an error that comes and go… on several projects.

It only happens when using --gc or after having deleted the resources dir, so I suppose it involves Hugo trying to regenerate resources.

I had it on several repos (I can’t share unfortunately), one whith only a scss resource in there (no json, image, or csv).

The problem solves itself when I delete the project and clone it again from remote… :man_shrugging:

I understand the following log may not be very useful but it really is all I got.

If it’s bound to memory, is there any way to clear this memory so Hugo can start on a clean slate?

panic: runtime error: slice bounds out of range

goroutine 1 [running]:
github.com/spf13/afero.(*UnionFile).Readdir(0xc0004cc640, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/afero@v1.2.0/unionFile.go:192 +0x516
github.com/spf13/afero.(*UnionFile).Readdirnames(0xc0004cc640, 0x1, 0x402b8d2, 0xc0000dc800, 0xc0006677c0, 0x18, 0xc0006677b8)
	/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/afero@v1.2.0/unionFile.go:196 +0x4d
github.com/gohugoio/hugo/cache/filecache.Caches.Prune.func1(0x4ca8565, 0x1, 0x4ef2820, 0xc00000d3a0, 0x0, 0x0, 0x0, 0x0)
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/cache/filecache/filecache_pruner.go:45 +0x12b
github.com/spf13/afero.walk(0x4efada0, 0xc00000c500, 0x0, 0x0, 0x4ef2820, 0xc00000d3a0, 0xc00000d360, 0x0, 0xc00000d360)
	/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/afero@v1.2.0/path.go:44 +0x8a
github.com/spf13/afero.Walk(0x4efada0, 0xc00000c500, 0x0, 0x0, 0xc00000d360, 0x0, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/afero@v1.2.0/path.go:105 +0xe5
github.com/gohugoio/hugo/cache/filecache.Caches.Prune(0xc0001e4360, 0xc0004568f0, 0x48b57e1, 0xc00056ba48)
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/cache/filecache/filecache_pruner.go:31 +0x167
github.com/gohugoio/hugo/hugolib.(*HugoSites).GC(0xc0000b3640, 0xc000069260, 0x4ca88d0, 0x2)
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/hugolib/prune_resources.go:18 +0x33
github.com/gohugoio/hugo/commands.(*commandeer).fullBuild(0xc00024e240, 0x0, 0x0)
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/commands/hugo.go:333 +0x26c
github.com/gohugoio/hugo/commands.(*commandeer).build(0xc00024e240, 0x0, 0x0)
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/commands/hugo.go:350 +0xaf
github.com/gohugoio/hugo/commands.(*commandsBuilder).newHugoCmd.func1(0xc000265b80, 0xc00034f2e0, 0x0, 0x1, 0x0, 0x0)
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/commands/commands.go:161 +0xc3
github.com/spf13/cobra.(*Command).execute(0xc000265b80, 0xc0000ba010, 0x1, 0x1, 0xc000265b80, 0xc0000ba010)
	/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:762 +0x473
github.com/spf13/cobra.(*Command).ExecuteC(0xc000265b80, 0xc00034efb0, 0xc00014a960, 0x0)
	/Users/brew/Library/Caches/Homebrew/go_cache/pkg/mod/github.com/spf13/cobra@v0.0.3/command.go:852 +0x2fd
github.com/gohugoio/hugo/commands.Execute(0xc0000ba010, 0x1, 0x1, 0x8, 0xc00048df88, 0x4007700, 0xc00009c058)
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/commands/hugo.go:86 +0xad
main.main()
	/private/tmp/hugo-20181224-98242-12trytm/hugo-0.53/src/github.com/gohugoio/hugo/main.go:27 +0x7a
2

Seems to be a bug in the virtual filesystem we use. I will eventually fix.

But you should be able to fix this by deleting the file cache folder(s), probably resources/_gen?

3

Thanks for opening the issue.

But you should be able to fix this by deleting the file cache folder(s), probably resources/_gen ?

When I tried that I couldn’t even run hugo (without --gc), hence my thinking it was about trying to generate resources rather than using their cached version.

Question: do you think it’s possible a project has been eating memory and the error gets thrown while building another project?

4

No. I have fixed the bug upstream. It was a stupid code error from … someone.

5

I have pushed a fix to this issue to Hugo. I can add that this error situation would happen when you get to an empty directory in your cache. The code path in the virtual composite file system we use crashed in this case … When “cache pruning” we remove empty directories (which should be rare, which is, I guess, the reason we have not heard about this before).

3 Likes
6

Might happen if you interrupt builds too often I guess (ctr + c).

7

Yes, maybe. My best guess would be that it happens when you rename your resource target folders. I will put out a patch release with this and some other fixes in one of the next few days.

1 Like
8

Ok, so now that your fix provides a nice message I can grasp what is happening yet I don’t see how I can prevent it:

Error: failed to prune cache "images": remove /Users/cheribibi/Boulot/projets/hugo/freewind_project/resources/_gen/images: no such file or directory

As I don’t have any image resources in my project, Hugo does not create any resources/_gen/images dir. So everytime I run --gc I end up with the message, unless I manually create said dir. :thinking:

9

As it happens, I had some resources in a submodule (hugo component)… probably from having mistakenly run hugo from the component instead of the project.

Since I removed this resources dir, I don’t have the error anymore.

10

@regis note that my fix fixed the very specific error you got, I didn’t test your specific situation (I really don’t know). If you still think there is a bug here, please open another GH issue.

1 Like