How to diagnose which template comsume a lot of memories?

support
1

Hi, I found that the theme uses a lot of memories for building site, even I’ve specified the --gc flag.
Once the pages over 10k, the process was killed by OS, since the Out of memory error is reached.

[  291.214115] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/,task=hugo,pid=4795,uid=1000
[  291.214267] Out of memory: Killed process 4795 (hugo) total-vm:11842260kB, anon-rss:7361308kB, file-rss:0kB, shmem-rss:0kB, UID:1000 pgtables:18356kB oom_score_adj:0
[  291.466886] oom_reaper: reaped process 4795 (hugo), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

I believed there is something need to improve in the theme’s template, how can I determine which part/template using a lot of memories?

image
image268×940 10.1 KB

EDIT:

I enabled the metrics, but there’s no intuitive parameters for memories.

image
image1091×138 3.18 KB

2

use option
–renderToDisk → serve all files from disk (default is from memory)

This helped other users :wink:

3

@ju52 is right. You could also try

hugo  --printMemoryUsage

I can also add that the next Hugo version will be much more effective when it comes to memory handling.

4

@ju52 @bep Thanks for your reply, but I’m using hugo for generating the static files in production, the renderToDisk seems not available for hugo command.

$ hugo --gc --renderToDisk
Error: unknown flag: --renderToDisk

I also use --templateMetrics --templateMetricsHints --printMemoryUsage flags for diagnosing the problem, but it’s hard for me to find out the root cause of the problem. Are there extra tools or tricks to troubleshoot this problem?

$ hugo --gc --minify --templateMetrics --templateMetricsHints  --printMemoryUsage
...

Alloc = 973.4 MB
TotalAlloc = 52.6 GB
Sys = 2.3 GB
NumGC = 158

Alloc = 1.3 GB
TotalAlloc = 54.6 GB
Sys = 2.3 GB
NumGC = 161

Alloc = 1.4 GB
TotalAlloc = 55.9 GB
Sys = 2.3 GB
NumGC = 163

Template Metrics:

     cumulative       average       maximum      cache  percent  cached  total
       duration      duration      duration  potential   cached   count  count  template
     ----------      --------      --------  ---------  -------  ------  -----  --------
  4m43.85896635s   84.835315ms  353.735075ms          0        0       0   3346  _default/single.html
  2m13.132415616s   39.788528ms  126.314977ms          2        0       0   3346  partials/post.html
  1m59.990751916s   50.736047ms  418.872067ms          0        0       0   2365  _default/list.html
  1m25.00948187s   13.385212ms  6.279195158s         30        0       0   6351  partials/head.html
  1m0.91849912s    2.492267ms  371.406788ms         12        0       0  24443  partials/list-item.html
   ...

I also use partialCached for caching partial when possible, but there was no significant effect, the memory’s usage still too high.

BTW, what does the Alloc，TotalAlloc, Sys mean?

Parameter Meaning
Alloc Current memory usage?
TotalAlloc Total memory allocation during the site build?
Sys Available memory for current build process?

Thanks.

1 Like
5

The hugo command (aka production) always renders to disk.