How to "ignore" intermediate images

support
1

I have a slightly complex render-image hook, it takes a large image, resizes, sharpens at the reduced size then resize to the same size to lower quality. Images look perfectly crisp, couldn’t do it server side, loving the power of Hugo.

So, unsurprisingly, on top of the original image the processing chain creates three images for each final output size.

  1. Original
  2. Resized
  3. Resized and sharpened
  4. Final output (which gets the .RelPermalink)

I’ve been trying to only have the final output get copied to public

This…

cascade:
  _build:
    publishResources: false

…deals with not copying over the original image which is a great start. Reading around that, I wasn’t sure if it was only supposed to publish the images with a RelPermalink? Maybe I am doing something incorrect.
Old discussion here: Don’t put unprocessed resources in /public - support - HUGO (gohugo.io)

So then I tried to get rid of the second largest with ignoreFiles: but can’t work out how to match the files with “resize_q100” in the filename (has to be simple!).

I don’t have all my content up yet and my public folder currently ends up at 10GB, of which only 1GB is actually needed. I currently tackle this with a grunt script but it would be great not to be generating so much stuff that was the deleted.

Thanks for any help.

2

Is this a problem because you transfer the contents of public somewhere to be served?

3

Currently I run a grunt script to clean up public folder of the unused images.
I sftp sync with the host, it does work ok.

The problem is on next the build it will copy everything back from the cached resources folder, even on a nvme drive it does take some time, about 20 seconds after the publishResources: false was set.
About the same time for the grunt script to run also, so lets say 40s.
I’d guess I currently have a fifth or less of the content. So it could get up to 50GB and 200s very easily. I’m not sure I should be too worried about the writes to the SSD but if there is a simple way to reduce the load and the time, that would be great.

4

I may not understand the problem, but try this:

git clone --single-branch -b hugo-forum-topic-42640 https://github.com/jmooring/hugo-testing hugo-forum-topic-42640
cd hugo-forum-topic-42640
rm -rf public/ && hugo && tree public
1 Like
5

OK, got it.

First run → one image published
Second run, without clearing the resources directory → many images published

This is a known issue:
https://github.com/gohugoio/hugo/issues/10255#issuecomment-1241350155

2 Likes
6

Right! So it is sort of working both your way and mine (pipe rather than “with”).
I didn’t know it was coming from the cache.

Will ignoreFiles: be of any use in that case?

Problem to clear the cache is the build times are really long without it, all that resizing and sharpening lots of images (not a hugo performance issue, I’m really impressed the speed, I think it is at least as fast as photoshop and far more reliable and the resizing method is excellent).

Thank you for getting to the bottom of it.

7

No.

Exactly.

I don’t know of a work around, other than something external to Hugo.

If public looks like:

public/
├── posts/
│   ├── post-1/
│   │   ├── a_hua03266d9251670c666624a396dbef1d5_42764_2bc65ed9939d3a398dabc8494558febe.jpg
│   │   ├── a_hua03266d9251670c666624a396dbef1d5_42764_300x0_resize_q75_box.jpg
│   │   ├── a_hua03266d9251670c666624a396dbef1d5_42764_68d019d35705d0de0c94346e2a6478b0.jpg
│   │   └── index.html
│   └── index.html
├── favicon.ico
└── index.html

… how do you determine which one to keep with your Grunt task?

8

Super, I’ll be patient, wait for a fix and carry on as I have.
The grunt task looks at all the html files and only removes unused images:
grunt-unused - npm (npmjs.com)
Very useful!

2 Likes
9

Nice.

I assume if you process a global resource (assets directory) and then pass the final permalink to be used in a Sass file (e.g. background image url), that file will be removed because it is not in the HTML? Rare case, but I actually use that approach.

1 Like
10

It does have excludes, since I have it just looking in **/*.html I had to exclude some favicon images in manifest. I think you can probably point it at style sheets in addition.

2 Likes