Running with chache disabled slows down sites with many images

@bep Version .52 is slower by factor 10 to 50

Metrics for 0.51

render and write pages:
	3.1959111s (4.3879638s)	  202.53 MB 	2126411 Allocs
render and write Sitemap:
	0s (4.3879638s)	    0.00 MB 	0 Allocs
render and write robots.txt:
	0s (4.3879638s)	    0.01 MB 	54 Allocs
render and write 404:
	2.001ms (4.3899648s)	    0.17 MB 	2767 Allocs
DEBUG 2018/12/05 11:00:06 Render home to "\\feed.json" with layouts ["_text/page/index.de.json.json" "_text/page/home.de.json.json" "_text/page/list.de.json.json" "_text/page/index.json.json" "_text/page/home.json.json" "_text/page/list.json.json" "_text/page/index.de.json" "_text/page/home.de.json" "_text/page/list.de.json" "_text/page/index.json" "_text/page/home.json" "_text/page/list.json" "_text/index.de.json.json" "_text/home.de.json.json" "_text/list.de.json.json" "_text/index.json.json" "_text/home.json.json" "_text/list.json.json" "_text/index.de.json" "_text/home.de.json" "_text/list.de.json" "_text/index.json" "_text/home.json" "_text/list.json" "_text/_default/index.de.json.json" "_text/_default/home.de.json.json" "_text/_default/list.de.json.json" "_text/_default/index.json.json" "_text/_default/home.json.json" "_text/_default/list.json.json" "_text/_default/index.de.json" "_text/_default/home.de.json" "_text/_default/list.de.json" "_text/_default/index.json" "_text/_default/home.json" "_text/_default/list.json"]
render and write pages:
	4.9989ms (4.3959658s)	    1.23 MB 	27479 Allocs
render and write pages:
	0s (4.3959658s)	    0.23 MB 	2666 Allocs
DEBUG 2018/12/05 11:00:06 Render home to "\\feed.xml" with layouts ["page/index.de.atom.xml" "page/home.de.atom.xml" "page/list.de.atom.xml" "page/index.atom.xml" "page/home.atom.xml" "page/list.atom.xml" "page/index.de.xml" "page/home.de.xml" "page/list.de.xml" "page/index.xml" "page/home.xml" "page/list.xml" "index.de.atom.xml" "home.de.atom.xml" "list.de.atom.xml" "index.atom.xml" "home.atom.xml" "list.atom.xml" "index.de.xml" "home.de.xml" "list.de.xml" "index.xml" "home.xml" "list.xml" "_default/index.de.atom.xml" "_default/home.de.atom.xml" "_default/list.de.atom.xml" "_default/index.atom.xml" "_default/home.atom.xml" "_default/list.atom.xml" "_default/index.de.xml" "_default/home.de.xml" "_default/list.de.xml" "_default/index.xml" "_default/home.xml" "_default/list.xml"]
render and write pages:
	86.0291ms (4.4819949s)	    7.58 MB 	77675 Allocs

Template Metrics:

     cumulative       average       maximum         
       duration      duration      duration  count  template
     ----------      --------      --------  -----  --------
     1.5650503s   55.894653ms    925.0252ms     28  photo\gallery.html
     1.0733759s   214.67518ms    1.0062787s      5  photo\single.html
     1.0673813s   30.496608ms    876.0912ms     35  shortcodes/imagelist.html
      233.029ms    1.125743ms      6.0009ms    207  tag\list.html
     160.0275ms     393.187µs      6.0004ms    407  partials/meta.html
      92.0318ms    1.095616ms       3.001ms     84  post\single.html
      85.0311ms      87.211µs       2.002ms    975  _default/summary.html
      85.0294ms     85.0294ms     85.0294ms      1  _default/home.atom.xml
      71.0124ms     174.477µs      1.9995ms    407  partials/favicon.html
      61.9952ms     152.322µs      1.9989ms    407  partials/header.html
      57.0233ms      58.485µs      1.0048ms    975  partials/summary-head.html
      51.0087ms    4.250725ms     17.0003ms     12  intern\single.html
      49.0343ms     207.772µs      4.0005ms    236  partials/pagination.html
      47.0204ms    1.621393ms      7.0013ms     29  _default\list.html
      47.0189ms     1.74144ms     17.0021ms     27  photo\panorama.html
      45.0159ms     298.118µs      2.0019ms    151  partials/single-head.html
      39.0082ms      5.5726ms      9.0018ms      7  _default\single.html
      38.9894ms      95.797µs      3.9994ms    407  partials/css/css.html
      21.9956ms    3.142228ms      6.0022ms      7  shortcodes/linux/shortlist.html
      15.9642ms      39.417µs      1.0028ms    405  partials/footer.html
        8.003ms       8.208µs      1.0022ms    975  partials/symbol.html
       4.9989ms      4.9989ms      4.9989ms      1  _default/home.json.json
       3.0031ms       7.378µs      1.0022ms    407  partials/feeds.html
       3.0022ms      3.0022ms      3.0022ms      1  tag\tags.html
       3.0015ms     1.50075ms      2.0008ms      2  shortcodes/linux/list.html
       3.0004ms      3.0004ms      3.0004ms      1  shortcodes/vcard.html
       2.9999ms      2.9999ms      2.9999ms      1  index.html
       2.0031ms     100.155µs      1.0018ms     20  shortcodes/maplink.html
       1.9963ms      1.9963ms      1.9963ms      1  partials/xml/atom.html
       1.0024ms       501.2µs      1.0024ms      2  shortcodes/msg/todo.html
        1.002ms     111.333µs       1.002ms      9  shortcodes/readcode.html
       1.0019ms      200.38µs      1.0019ms      5  _internal/shortcodes/relref.html
       1.0015ms        2.46µs      1.0015ms    407  partials/css/tachyons.html
        1.001ms       1.001ms       1.001ms      1  404.html
       1.0008ms      14.937µs      1.0008ms     67  partials/meta-photo.html
       1.0006ms      1.0006ms      1.0006ms      1  partials/xml/icon.html
       1.0001ms      500.05µs      1.0001ms      2  _default/menu.html
        999.7µs        76.9µs       999.7µs     13  _internal/shortcodes/ref.html
        999.6µs       999.6µs       999.6µs      1  links\list.html
        998.3µs       998.3µs       998.3µs      1  shortcodes/staticindex.html
             0s            0s            0s      1  series\series.html
             0s            0s            0s      5  shortcodes/frac.html
             0s            0s            0s     14  shortcodes/fa/r.html
             0s            0s            0s      1  shortcodes/fa/rf.html
             0s            0s            0s      1  shortcodes/version.html
             0s            0s            0s      2  shortcodes/msg/note.html
             0s            0s            0s      3  shortcodes/msg/warn.html
             0s            0s            0s      6  shortcodes/fa/s.html
             0s            0s            0s      1  robots.txt
             0s            0s            0s      1  intern\list.html
             0s            0s            0s      3  shortcodes/msg/tipp.html
             0s            0s            0s     14  shortcodes/tube.html
             0s            0s            0s    407  partials/css/awesome.html
             0s            0s            0s      3  shortcodes/note.html
             0s            0s            0s      1  shortcodes/footnotes.html
             0s            0s            0s     11  shortcodes/figure.html
             0s            0s            0s      2  shortcodes/fa/sf.html


                   | DE   
+------------------+-----+
  Pages            | 360  
  Paginator pages  |  50  
  Non-page files   | 506  
  Static files     |  79  
  Processed images | 506  
  Aliases          | 186  
  Sitemaps         |   0  
  Cleaned          |   0  

Total in 4334 ms

Metrics for 0.52

render and write pages:
	27.5430537s (29.1825557s)	 3100.42 MB 	2151313 Allocs
render and write Sitemap:
	0s (29.1825557s)	    0.00 MB 	0 Allocs
render and write robots.txt:
	999.5µs (29.1835552s)	    0.01 MB 	54 Allocs
render and write 404:
	1.0013ms (29.1845565s)	    0.17 MB 	2767 Allocs
DEBUG 2018/12/05 10:57:36 Render home to "\\feed.json" with layouts ["_text/page/index.de.json.json" "_text/page/home.de.json.json" "_text/page/list.de.json.json" "_text/page/index.json.json" "_text/page/home.json.json" "_text/page/list.json.json" "_text/page/index.de.json" "_text/page/home.de.json" "_text/page/list.de.json" "_text/page/index.json" "_text/page/home.json" "_text/page/list.json" "_text/index.de.json.json" "_text/home.de.json.json" "_text/list.de.json.json" "_text/index.json.json" "_text/home.json.json" "_text/list.json.json" "_text/index.de.json" "_text/home.de.json" "_text/list.de.json" "_text/index.json" "_text/home.json" "_text/list.json" "_text/_default/index.de.json.json" "_text/_default/home.de.json.json" "_text/_default/list.de.json.json" "_text/_default/index.json.json" "_text/_default/home.json.json" "_text/_default/list.json.json" "_text/_default/index.de.json" "_text/_default/home.de.json" "_text/_default/list.de.json" "_text/_default/index.json" "_text/_default/home.json" "_text/_default/list.json"]
render and write pages:
	5.9997ms (29.1905562s)	    1.29 MB 	27481 Allocs
render and write pages:
	0s (29.1905562s)	    0.23 MB 	2667 Allocs
DEBUG 2018/12/05 10:57:36 Render home to "\\feed.xml" with layouts ["page/index.de.atom.xml" "page/home.de.atom.xml" "page/list.de.atom.xml" "page/index.atom.xml" "page/home.atom.xml" "page/list.atom.xml" "page/index.de.xml" "page/home.de.xml" "page/list.de.xml" "page/index.xml" "page/home.xml" "page/list.xml" "index.de.atom.xml" "home.de.atom.xml" "list.de.atom.xml" "index.atom.xml" "home.atom.xml" "list.atom.xml" "index.de.xml" "home.de.xml" "list.de.xml" "index.xml" "home.xml" "list.xml" "_default/index.de.atom.xml" "_default/home.de.atom.xml" "_default/list.de.atom.xml" "_default/index.atom.xml" "_default/home.atom.xml" "_default/list.atom.xml" "_default/index.de.xml" "_default/home.de.xml" "_default/list.de.xml" "_default/index.xml" "_default/home.xml" "_default/list.xml"]
render and write pages:
	9.810685s (39.0012412s)	  290.65 MB 	87346 Allocs

Template Metrics:

     cumulative       average       maximum         
       duration      duration      duration  count  template
     ----------      --------      --------  -----  --------
  1m29.2906788s  3.188952814s   13.3466626s     28  photo\gallery.html
    15.7300728s  449.430651ms    8.1036009s     35  shortcodes/imagelist.html
    15.6250781s   3.12501562s   10.6635393s      5  photo\single.html
     9.8086853s    9.8086853s    9.8086853s      1  _default/home.atom.xml
     260.0477ms    1.256269ms     16.0024ms    207  tag\list.html
     160.0588ms     393.264µs     15.0023ms    407  partials/meta.html
     148.0082ms   12.334016ms    113.9986ms     12  intern\single.html
      90.9854ms      93.318µs      1.0024ms    975  _default/summary.html
      80.0966ms      953.53µs      3.0014ms     84  post\single.html
      72.0064ms     176.919µs     15.0023ms    407  partials/favicon.html
      69.0045ms    2.555722ms     47.0007ms     27  photo\panorama.html
      64.0048ms      65.645µs       1.002ms    975  partials/summary-head.html
      62.9704ms    2.171393ms      5.0008ms     29  _default\list.html
      62.0372ms     152.425µs      1.0309ms    407  partials/header.html
      57.0023ms     241.535µs      5.0008ms    236  partials/pagination.html
       45.032ms     110.643µs      3.9989ms    407  partials/css/css.html
      40.0306ms     265.103µs      1.0281ms    151  partials/single-head.html
      27.0068ms      66.683µs      1.0021ms    405  partials/footer.html
      26.9973ms    3.856757ms      5.9992ms      7  _default\single.html
       17.003ms       2.429ms      5.9971ms      7  shortcodes/linux/shortlist.html
      13.0003ms      13.333µs       1.002ms    975  partials/symbol.html
       7.0013ms      7.0013ms      7.0013ms      1  tag\tags.html
       5.0006ms      5.0006ms      5.0006ms      1  _default/home.json.json
       4.0029ms       9.835µs      1.0019ms    407  partials/feeds.html
       3.9992ms      1.9996ms      2.0002ms      2  shortcodes/linux/list.html
       3.0316ms      3.0316ms      3.0316ms      1  index.html
       2.9994ms      2.9994ms      2.9994ms      1  partials/xml/atom.html
       2.0017ms      2.0017ms      2.0017ms      1  links\list.html
       1.9974ms     221.933µs       998.9µs      9  shortcodes/readcode.html
       1.0013ms      1.0013ms      1.0013ms      1  shortcodes/staticindex.html
       1.0004ms      14.931µs      1.0004ms     67  partials/meta-photo.html
       1.0003ms      50.015µs      1.0003ms     20  shortcodes/maplink.html
            1ms         200µs           1ms      5  _internal/shortcodes/relref.html
            1ms      90.909µs           1ms     11  shortcodes/figure.html
        999.2µs       499.6µs       999.2µs      2  shortcodes/fa/sf.html
        998.8µs       998.8µs       998.8µs      1  intern\list.html
        998.8µs       998.8µs       998.8µs      1  partials/xml/icon.html
        997.7µs       997.7µs       997.7µs      1  shortcodes/vcard.html
        993.7µs      496.85µs       993.7µs      2  _default/menu.html
             0s            0s            0s     14  shortcodes/fa/r.html
             0s            0s            0s     13  _internal/shortcodes/ref.html
             0s            0s            0s      1  robots.txt
             0s            0s            0s      1  404.html
             0s            0s            0s    407  partials/css/awesome.html
             0s            0s            0s      6  shortcodes/fa/s.html
             0s            0s            0s      3  shortcodes/note.html
             0s            0s            0s      2  shortcodes/msg/note.html
             0s            0s            0s      2  shortcodes/msg/todo.html
             0s            0s            0s     14  shortcodes/tube.html
             0s            0s            0s    407  partials/css/tachyons.html
             0s            0s            0s      1  shortcodes/version.html
             0s            0s            0s      1  shortcodes/footnotes.html
             0s            0s            0s      3  shortcodes/msg/tipp.html
             0s            0s            0s      1  series\series.html
             0s            0s            0s      5  shortcodes/frac.html
             0s            0s            0s      3  shortcodes/msg/warn.html
             0s            0s            0s      1  shortcodes/fa/rf.html


                   | DE   
+------------------+-----+
  Pages            | 360  
  Paginator pages  |  50  
  Non-page files   | 506  
  Static files     |  79  
  Processed images | 506  
  Aliases          | 186  
  Sitemaps         |   0  
  Cleaned          |   0  

Total in 38824 ms

That is obviously a false statement. There may be an issue with your particular site, but that is impossible to tell without … seing your site.

1 Like

And note that the only relevant change in 0.52 was a re-implementation of the file caches. If you do lots of getJSON or similar, that could be the culprit (that it somehow does not work).

Are you running this on Windows?

Yes, Windows 10 64x

Same effect on Raspbian!

I have looked at your project. You have configured the caches with a maxAge of 2.097152ms, which, not surprisingly, have an … impact.

It is easier to reason about if you use a string, e.g.:

maxAge = "300h" # 300 hours. Or 60m = 60 minutes, -1 = infinite etc.
2 Likes

Thanks

Please add the default value and unit for maxAge to the docs.

https://golang.org/pkg/time/#Duration

A Duration represents the elapsed time between two instants as an int64 nanosecond count. The representation limits the largest representable duration to approximately 290 years.

I started with seconds in my mind.

This is the duration before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off. Uses Go’s time.Duration , so valid values are "10s" (10 seconds), "10m" (10 minutes) and "10h" (10 hours).

This is the current docs which looks good to me. If you have a suggested improvement to the above, I suggest you create a PR.