How to Instruct Hugo Pipe not to cache?

[hollowaykeanho]

As titled.

Currently, I’m working around with the filename random generator:

        {{- $filename = printf "deletable-%v-%v-%v-%v"
                                (mod (add (mul 13 now.Unix) 97) 400000)
                                (sha256 (string $Page.URL.Current.Absolute))
                                (now.Format "200601021504050700")
                                (crypto.FNV32a $dataList)
        -}}
        {{- $dataList = resources.FromString $filename $dataList -}}
        {{- $dataList = resources.ExecuteAsTemplate $filename $Page $dataList -}}
        {{- $dataList = string $dataList.Content -}}

Use case:

1. String processing with Go template capability. The function is too primitive for caching every string the module processes (and noticeable memory consumption).
2. The random number + URL fields are required due to uncontrollable caching not able to update certain components in the module.

Problem trying to solve:

1. High memory consumption (about 240MB, 2% of 12GB RAM).

Reference:

1. Hestia/Sanitize at experimental · ZORALab/Hestia · GitHub
2. hestiaHUGO: fixed hugo's auto caching problem in hestiaSTRING/Sanitize · ZORALab/Hestia@d77b81f · GitHub

[bep]

You cannot instruct Hugo Pipe not to cache, which has proven to be a problem when doing transform.Unmarshal on large amount of big JSON files. I have not seen your particular problem reported before, and I wouldn’t call 240MB of 12GB “high memory consumption”.

That said, I have a release planned in the near future (it’s called the “million pages” release in the road map) which introduces a LRU cache for most of Hugo’s caching – which gets dynamically resized in low memory situations.

[hollowaykeanho]

@bep

You cannot instruct Hugo Pipe not to cache, which has proven to be a problem when doing transform.Unmarshal on large amount of big JSON files.

OK. Got it.

@bep

I have not seen your particular problem reported before, and I wouldn’t call 240MB of 12GB “high memory consumption”.

Yeap. I haven’t report anything yet. You got a very helpful and knowledgeable communities here so wondering did others face the same problem.

I see. It’s quite scary because the total size (as in include those Blink and Gecko engine browsers) for testing, I barely can move around in my 12GB RAM 8-years old laptop haha.

Partly I’m to be blamed because I managed to turn Hugo into a web UI component compiler rather than just compiling page contents alone (as in compile compliant favicons, PWA, openGraph, sitemaps, robots, CSS, JS, hosting WASM, and etc) without any external dependencies. Can’t spoil it yet… coming soon. Currently porting the documentations.

@bep

That said, I have a release planned in the near future (it’s called the “million pages” release in the road map) which introduces a LRU cache for most of Hugo’s caching – which gets dynamically resized in low memory situations.

Noted. No hush. Status quo can still work. I’m just looking for ways to further optimise the module.

[bep]

@bep

That said, I have a release planned in the near future (it’s called the “million pages” release in the road map) which introduces a LRU cache for most of Hugo’s caching –

I can add that the concept with the LRU cache really do work – Go’s garbage collector is incredibly effective.

[hollowaykeanho]

@bep

I can add that the concept with the LRU cache really do work – Go’s garbage collector is incredibly effective.

Ok. Ahh… anything I can do now on my side? 0.o I just added a follow up ticket in the GitHub repo here "Memory Leak" Problem from hestiaHUGO · Issue #106 · ZORALab/Hestia · GitHub.

Will follow up first thing when you released the feature. =)