I am testing hugo for building site with lots of content (~350K pages). I am happy with the speed (~ 2 min using morden desktop). However I am confused by the memory usage which is about 15G. The content folder is only 1.1G and the generated public folder is only 3.4G. I don’t have any static files. I did a memory profile and the top results are
(pprof) top Showing nodes accounting for 26882.13MB, 60.16% of 44684.51MB total Dropped 607 nodes (cum <= 223.42MB) Showing top 10 nodes out of 209 flat flat% sum% cum cum% 9126.85MB 20.43% 20.43% 29361.37MB 65.71% reflect.Value.call 5549.01MB 12.42% 32.84% 5549.01MB 12.42% reflect.FuncOf 2422.04MB 5.42% 38.26% 7971.04MB 17.84% reflect.(*rtype).Method 2041.32MB 4.57% 42.83% 2041.32MB 4.57% bytes.makeSlice 1660.06MB 3.72% 46.55% 30601.43MB 68.48% github.com/gohugoio/hugo/tpl/internal/go_templates/texttemplate.(*state).evalCall 1484.71MB 3.32% 49.87% 1484.71MB 3.32% reflect.unsafe_NewArray 1322.34MB 2.96% 52.83% 1322.34MB 2.96% github.com/gohugoio/hugo/helpers.(*PathSpec).UnicodeSanitize 1234.04MB 2.76% 55.59% 2718.75MB 6.08% reflect.MakeSlice 1160.66MB 2.60% 58.19% 1160.66MB 2.60% net/url.parse 881.11MB 1.97% 60.16% 881.11MB 1.97% bytes.(*Buffer).String
Why does hugo need so much memory? Is it possible to reduce it? As I plan to run hugo hourly in production and reducing the memory requirement will reduce lots of cost. Btw, anohter great feature I am hopping is incremental build even in non-watching mode.