Is there anyway to profile memory usage and consumption for hugo server? I need to trace what exactly occupying the memory space and does not allow Go’s garbage collection to collect back.
Issue trying to solve:
Worst case impact: Hugo memory consumption keeps growing to 100% and even entering swap partition.
opened 08:09AM - 22 May 23 UTC
Enhancement
With Workaround
HestiaHUGO
Stuck
# Description
```
Please provide a short description of what feature you're lo… oking forward to
enhance below. Please include the story behind your idea as well to give a
better visualization of your idea.
```
Follow up with Bep and his latest release about Go's LRU optimization. It has the ability to garbage collect at will to reduce the memory consumption down effectively.
Discussion: https://discourse.gohugo.io/t/how-to-instruct-hugo-pipe-not-to-cache/44467/3
UPDATE:
It was detected that some part of the contents in `hestiaHUGO` is holding up the memory but inaccessible, behaving like a "memory leak".
# Expected Behavior
```
Please specify the expected behavior of your requested enhancement. Some great
and helpful pointers are your expected interface (e.g. command patterns,
simple sketches of the user interface, etc).
```
Lower memory consumption usage.
# Current Behavior
```
Please specify the current behavior (e.g. workaround, blockage, etc).
```
Hugo + Blink Engine Brower + Gecko Engine Browser can really saturate the RAM easily when used simultenously for building the page contents.
Using ZORALab's Hestia `1.2.0` with the following stat:
```
Start building sites …
hugo v0.111.3-5d4eb5154e1fed125ca8e9b5a0315c4180dab192 linux/amd64 BuildDate=2023-03-12T11:40:50Z VendorInfo=gohugoio
| EN
-------------------+------
Pages | 6
Paginator pages | 0
Non-page files | 97
Static files | 574
Processed images | 27
Aliases | 0
Sitemaps | 0
Cleaned | 0
Built in 6503 ms
Watching for changes in /home/u0/Documents/zoralab/Hestia/{hestiaHUGO,sites}
Watching for config changes in /home/u0/Documents/zoralab/Hestia/sites/config/_default, /home/u0/Documents/zoralab/Hestia/sites/config/development, /home/u0/Documents/zoralab/Hestia/hestiaHUGO/config/_default, /home/u0/Documents/zoralab/Hestia/hestiaHUGO/config/development
Environment: "development"
Serving pages from memory
Web Server is available at http://localhost:8080/ (bind address 127.0.0.1)
Press Ctrl+C to stop
```
Memory consumption is at: (about 240MB, 2% of 12GB RAM)
# Attachment
```
Please drag and drop the necessary data files (e.g. screenshot, logs, etc)
below.
```
Current workaround: periodically kill and start the server via cmd script (server.cmd)
committed 09:36AM - 06 Jun 23 UTC
Appearently, Hugo somehow still fails to garbage collect properly
in time until … the memory reaches 99% consumption and entering swap
file. This is not usable at all and it's outside ZORALab's Hestia
control. Fortunately, killing the server allows the memory restoration
immediately.
We already purged all memory related algorithms (.Store &
partialCached) already so the bug should highly related to Hugo Pipe
itself. We do not have a choice but to periodically kill the hugo
server.
This patch applies 3rd attempt to workaround hugo memory leak
symptom in hestiaHUGO/ directory.
Co-authored-by: Shuralyov, Jean <jean.shuralyov@proton.me>
Co-authored-by: Galyna, Cory <cory.galyna@zoralab.com>
Co-authored-by: (Holloway) Chew, Kean Ho <kean.ho.chew@zoralab.com>
Signed-off-by: (Holloway) Chew, Kean Ho <kean.ho.chew@zoralab.com>
Attempted: tried all the commands in here but hugo does not generate the mystats.txt output.
Is there a way to measure memory usage with Hugo?
That would be ideal to find potential memory spikes in themes etc…
Thanks a lot!