Don't put unprocessed resources in /public

support
1

I store big images (in their original format and size) to process them and deliver lower qualities and smaller sizes where I need them.
But the unprocessed, original jpgs get put into the /public folder although I don’t use them on the page.

Since they are pretty big, I would love to get by without having them copied to the public folder but only have the processed versions.

Is there a way to do it?

This is my file structure:

content/
├── site1.md
├── site2.md
├── site3.md
├── site4.md
└── media
     |    └── index.md
     └── img
          |   ├── img0.jpg
          |   └── img00.jpg
          ├── folder1
          |   ├── image1.jpg
          |   ├── image2.jpg
          |   └── image3.jpg
          ├── folder2
          |   ├── image1.jpg
          |   ├── image2.jpg
          |   └── image3.jpg
          ├── folder3
          |   ├── image1.jpg
          |   ├── image2.jpg
          |   └── image3.jpg

The code on my site:

–

layouts/photo/create.html

<!--    config    -->
{{- $imageBox_1x := ( print .Site.Params.images.boxsize "x" .Site.Params.images.boxsize ) -}}
{{- $imageBox_2x := ( print (mul .Site.Params.images.boxsize 2) "x" (mul .Site.Params.images.boxsize 2) ) -}}

<!--    embed    -->

{{- $name := .Params.img -}}
{{- $images := (.Site.GetPage "page" "media").Resources.Match (print "img/" $name) -}}

{{- range $image := $images -}}

    {{- $image_1x := ($image.Fit $imageBox_1x ) -}}
    {{- $image_2x := ($image.Fit $imageBox_2x ) -}}

    {{- $imagedict := (dict "url_1x"  $image_1x.RelPermalink "url_2x" $image_2x.RelPermalink "height" $image_1x.Height "width" $image_1x.Width ) -}}

    {{- $.Scratch.Add "images" (slice $imagedict) -}}

{{- end -}}

–

layouts/photo/embed.html

{{- .Render "create" -}}
<div id="photo-container">
{{ range $key, $data := ( $.Scratch.Get "images" ) }}
    <img class="fade"
    alt = "{{ .Params.title }}"
    width="{{ $data.width }}px"
    height="{{ $data.height }}px"
    src="{{ $data.url_1x }}"
    srcset="{{ $data.url_1x }} 1x, {{ $data.url_2x }} 2x" >
{{ end }}
</div>

–

layouts/photo/thumb.html

<!--    config    -->

{{- $thumbnailBox_1x := ( print .Site.Params.thumbs.boxsize "x" .Site.Params.thumbs.boxsize ) -}}
{{- $thumbnailBox_2x := ( print (mul .Site.Params.thumbs.boxsize 2) "x" (mul .Site.Params.thumbs.boxsize 2) ) -}}
{{- $thumbnailQuality := .Site.Params.thumbs.quality -}}

<!--    thumb    -->

{{- with .Params.thumb | default .Params.img -}}
    {{- $.Scratch.Set "thumb" . -}}
{{- end -}}

{{- $thumb := (.Site.GetPage "page" "media").Resources.GetMatch (print "img/" ( .Scratch.Get "thumb" ) ) -}}

{{- $thumb_1x := ($thumb.Fit (print $thumbnailBox_1x " q" $thumbnailQuality)) -}}
{{- $thumb_2x := ($thumb.Fit (print $thumbnailBox_2x " q" $thumbnailQuality)) -}}

{{- .Scratch.Set "thumbnail_1x" $thumb_1x.RelPermalink -}}
{{- .Scratch.Set "thumbnail_2x" $thumb_2x.RelPermalink -}}

{{- .Scratch.Set "thumbnail_width" $thumb_1x.Width -}}
{{- .Scratch.Set "thumbnail_height" $thumb_1x.Height -}}

and in the partial:

<div id="modal-content" class="loading">
  <div class="loadstatus blink" hidden="hidden">
    <p>#loading</p>
  </div>
  {{- if .IsPage -}}
    {{- if isset .Params "type" -}}
      {{- .Render "embed" -}}
    {{- end -}}
  {{- end -}}
</div>

Is there a way to not have the originals copied to public even though I don’t need/use them?

Is there another way / a more simple way to do what I’m doing?
I would like to keep the site itself and it’s resources in separate folders. Since I don’t only have “photo” types, I would prefer to not hav folders for some sites and files for others.

Am I doing it wrong?
(I’m on 0.37)

2

and this is my content/media/index.md

+++

title = "sub"
date = 2018-01-25
headless = true

+++
3

Currently no. There is no way for us (me) to know if you “need/use” them.

4

Proposal: If a dir contains a cookie file .hugoignore, don’t copy that dir and its contents to public (but do process them as needed by Resources methods).

5

The foundation of that proposal is good, the name of the files is, however, not perfect … Also, what about JSON files etc? SVG images?

6

What if the .hugoignore file had syntax like .gitignore?

3 Likes
7

I like the the proposal of @RickCogley

something like

content/.hugoignore

/media/**.jpg
/header.jpg

But (maybe this is a stupid question) I thought huge might / could know which files are being references within the site…

Another proposal:
Let us put files in resources/ and reference them via .Resources or a special front matter like

[extResources]
1 Like
8

We have no idea what files are referenced from the internet.

9

yes, of course.
Maybe have a setting ignoreUnprocessedResources = true (the name might be suboptimal) in config.tml?

10

I think I have the same issue. I’m running a photo blog and from a workflow perspective, it would be best to save full-resolution images with my content. During the processing, the various downscaled versions should get created and only those should be uploaded to save webspace. This way, I’m able to adjust web layout and thus resolutions later-on without having to re-export all images from Lightroom.
So far, I’ve been doing this prior to hugo processing by storing them together in a large folder and then generating the low-resolution versions and moving only those into the static folder.
It would be nice to be able to store them with the individual posts and use hugo’s built-in resizing feature but without a way to “hide” the originals, this doesn’t seem to help me.
Is anyone using an alternate workflow to make this work?

11

So, I have been doing image processing / resource work outside of the issue discussed here. When you do transformations in chains as described in some other thread, you would potentially end up with lots of garbage if you pubiish all the steps, so for those I decided to do “lazy publishing”; i.e. only publish resources which is referenced. Also, since your source is (normally) in /assets, the original will never be published.

But for the images in /content that cannot be the default, because I have no idea if the image is used or not. So, we probably need a very simple way to tell Hugo about that … And I would love if that “simple thing” could be something that could be used for other /content items that you do not want to publish. I.e. you may want the .Content, but don’t care about the .Permalink.

12
  1. where can we find more about /assets
  2. working with page bundles, can we use a sub-directory ./ignore ore ./private for this? or ./assets`?
13

I have the same issue here.

In my case, I also have my images in the same folder as the respective content

content/
    A/
        - image1.png
        - index.md
    B/
        - image1.png
        - index.md

I currently have code in my theme that does this:

{{ $scaled := .Resize "x800 q90" }}
<img style="margin: 1px 0 1px 0 ; width: 100%" src="{{ $scaled.Permalink }}" />

Because I am asking Hugo to scale my images in my current directory of the content directory, I don’t want to keep the original images in the public folder.

I’d be ok with a config option that when set to true it will remove all original files. Maybe even something as simple as a naming semantic, e.g. all image files with hugo_ prefixed in front of the images will not get transferred over to the public folder?

In the interim, I am thinking about standardizing a naming schema and writing a separate shell script that will scan my public directory and remove these images based on a regex.

It’d be nice if Hugo offered something similar out of the box.

14

Is there any news about this? I couldn’t find an active issue on GitHub, so I thought I’d check in here.
I found myself wanting to exclude content files I bundle with resources.Match so they don’t end up in public/

15

See https://gohugo.io/content-management/build-options/#readout

3 Likes
16

Thank you for this, usefull and elegant! Will try it tomorrow! Thanks😊

1 Like
17

For reference:

bep’s link above no longer goes to the correct anchor in the docs. The relevant build option is publishResources.

Another option for this use case is to use the global ignoreFiles configuration setting to exclude files.

Also note that hugo ignores any files whose name begins with . or # or whose name ends with ~.

Proposal:

While these features cover many use cases, there are still a few gaps and ugly corner cases.

For example, it is not currently possible to override the ./#/~ filename filter, so e.g. you cannot create a .htaccess file that is published directly as a resource. (You can work around this using a Custom Output Format with basename = "", but that may not work well if you need a dot file containing binary data).
For another example, if you have many resources for a page and want to exclude only one, you must either use the global ignoreFiles, or use publishResources: false and ensure that you use .Permalink or .RelPermalink for all of the others.

What do you think of:

  1. Creating an includeFiles global configuration setting which can be used to override the hard-coded filename filter?
  2. Creating a "publish" resource metadata value that allows publishing to be explicitly enabled or disabled for a resource? This would achieve the same goal as the .hugoignore proposal above, but would allow either setting publishResources: true and explicitly marking resources to be ignored or setting publishResources: false and explicitly marking resources to be published.
1 Like