Why does hugo need so much memory for big site

support
1

I am testing hugo for building site with lots of content (~350K pages). I am happy with the speed (~ 2 min using morden desktop). However I am confused by the memory usage which is about 15G. The content folder is only 1.1G and the generated public folder is only 3.4G. I don’t have any static files. I did a memory profile and the top results are

(pprof) top
Showing nodes accounting for 26882.13MB, 60.16% of 44684.51MB total
Dropped 607 nodes (cum <= 223.42MB)
Showing top 10 nodes out of 209
      flat  flat%   sum%        cum   cum%
 9126.85MB 20.43% 20.43% 29361.37MB 65.71%  reflect.Value.call
 5549.01MB 12.42% 32.84%  5549.01MB 12.42%  reflect.FuncOf
 2422.04MB  5.42% 38.26%  7971.04MB 17.84%  reflect.(*rtype).Method
 2041.32MB  4.57% 42.83%  2041.32MB  4.57%  bytes.makeSlice
 1660.06MB  3.72% 46.55% 30601.43MB 68.48%  github.com/gohugoio/hugo/tpl/internal/go_templates/texttemplate.(*state).evalCall
 1484.71MB  3.32% 49.87%  1484.71MB  3.32%  reflect.unsafe_NewArray
 1322.34MB  2.96% 52.83%  1322.34MB  2.96%  github.com/gohugoio/hugo/helpers.(*PathSpec).UnicodeSanitize
 1234.04MB  2.76% 55.59%  2718.75MB  6.08%  reflect.MakeSlice
 1160.66MB  2.60% 58.19%  1160.66MB  2.60%  net/url.parse
  881.11MB  1.97% 60.16%   881.11MB  1.97%  bytes.(*Buffer).String

Why does hugo need so much memory? Is it possible to reduce it? As I plan to run hugo hourly in production and reducing the memory requirement will reduce lots of cost. Btw, anohter great feature I am hopping is incremental build even in non-watching mode.

2

As to the second question, I’m about to, if not fixing it, so at least wastly improve it But as it is now that is how it is.

1 Like