Cache between builds a remote resource from resources.GetRemote

support
1

Hi! Is there a way to cache (or rather store) between builds a resource from resources.GetRemote?

I started using resources.GetRemote two days ago, but build time is getting much longer than before, (due to the remote fetching, I guess). I tried processing the remote resources (they are images) to create processed images in /resources/_gen/images, but it doesn’t bypass remote fetching, just image processing.

So it’d be nice if Hugo could “cache” remote resource — just like it “caches” processed images storing them in /resources/_gen/images to bypass processing on following builds. It could create local resources in, for instance, /resources/_gen/remote and fetch from there in following builds.

Possible problem: a remote resource could change, turning the “cached” resource somewhat wrong. Perhaps all this “caching” could be enabled via a second argument to .GetRemote…

Does this make sense?

2

(I know at first it seems simpler to just download directly the remote resource and store it in Git. But I’m using .GetRemote to get YouTube/Vimeo thumbnails via oEmbed, which allows me — and content creators — to just paste a video link to get a nice thumbnail after build.)

3

See https://gohugo.io/getting-started/configuration/#configure-file-caches.

For example, on Ubuntu the remote resource is cached here:

/tmp/hugo_cache/my-project/filecache/getresource
2 Likes
4

Thank you, @jmooring !

A bit off-topic: since I’m using Hugo on Netlify, how to know whether Netlify is persisting that folder between builds?

5

https://github.com/gohugoio/hugo/blob/master/helpers/path.go#L415-L423

1 Like
6

Thank you again, @jmooring ! But perhaps I found a problem:

To ensure cache existence, I changed config to store it in a project folder:

[caches]
  [caches.getjson]
    dir = ':resourceDir/_gen'
    maxAge = 2592000
  [caches.getresource]
    dir = ':resourceDir/_gen'
    maxAge = 2592000

Then committed it, along with created files in those folders. But now, each build changes all files in getresource, which seems to mean Hugo is ignoring cache (getjson files don’t change). Is this correct?

7

https://gohugo.io/getting-started/configuration/#the-keywords-explained

maxAge

This is the duration before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off. Uses Go’s time.Duration , so valid values are "10s" (10 seconds), "10m" (10 minutes) and "10h" (10 hours).

I think your current value is 0.002592 seconds.

2 Likes
8

Thank you again and again, @jmooring . I didn’t read that part, just googled for max-age in HTML.

Now it all works (with -1 in maxAge)!

9

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.