Running with chache disabled slows down sites with many images


#1

@bep Version .52 is slower by factor 10 to 50

Metrics for 0.51

render and write pages:
	3.1959111s (4.3879638s)	  202.53 MB 	2126411 Allocs
render and write Sitemap:
	0s (4.3879638s)	    0.00 MB 	0 Allocs
render and write robots.txt:
	0s (4.3879638s)	    0.01 MB 	54 Allocs
render and write 404:
	2.001ms (4.3899648s)	    0.17 MB 	2767 Allocs
DEBUG 2018/12/05 11:00:06 Render home to "\\feed.json" with layouts ["_text/page/index.de.json.json" "_text/page/home.de.json.json" "_text/page/list.de.json.json" "_text/page/index.json.json" "_text/page/home.json.json" "_text/page/list.json.json" "_text/page/index.de.json" "_text/page/home.de.json" "_text/page/list.de.json" "_text/page/index.json" "_text/page/home.json" "_text/page/list.json" "_text/index.de.json.json" "_text/home.de.json.json" "_text/list.de.json.json" "_text/index.json.json" "_text/home.json.json" "_text/list.json.json" "_text/index.de.json" "_text/home.de.json" "_text/list.de.json" "_text/index.json" "_text/home.json" "_text/list.json" "_text/_default/index.de.json.json" "_text/_default/home.de.json.json" "_text/_default/list.de.json.json" "_text/_default/index.json.json" "_text/_default/home.json.json" "_text/_default/list.json.json" "_text/_default/index.de.json" "_text/_default/home.de.json" "_text/_default/list.de.json" "_text/_default/index.json" "_text/_default/home.json" "_text/_default/list.json"]
render and write pages:
	4.9989ms (4.3959658s)	    1.23 MB 	27479 Allocs
render and write pages:
	0s (4.3959658s)	    0.23 MB 	2666 Allocs
DEBUG 2018/12/05 11:00:06 Render home to "\\feed.xml" with layouts ["page/index.de.atom.xml" "page/home.de.atom.xml" "page/list.de.atom.xml" "page/index.atom.xml" "page/home.atom.xml" "page/list.atom.xml" "page/index.de.xml" "page/home.de.xml" "page/list.de.xml" "page/index.xml" "page/home.xml" "page/list.xml" "index.de.atom.xml" "home.de.atom.xml" "list.de.atom.xml" "index.atom.xml" "home.atom.xml" "list.atom.xml" "index.de.xml" "home.de.xml" "list.de.xml" "index.xml" "home.xml" "list.xml" "_default/index.de.atom.xml" "_default/home.de.atom.xml" "_default/list.de.atom.xml" "_default/index.atom.xml" "_default/home.atom.xml" "_default/list.atom.xml" "_default/index.de.xml" "_default/home.de.xml" "_default/list.de.xml" "_default/index.xml" "_default/home.xml" "_default/list.xml"]
render and write pages:
	86.0291ms (4.4819949s)	    7.58 MB 	77675 Allocs

Template Metrics:

     cumulative       average       maximum         
       duration      duration      duration  count  template
     ----------      --------      --------  -----  --------
     1.5650503s   55.894653ms    925.0252ms     28  photo\gallery.html
     1.0733759s   214.67518ms    1.0062787s      5  photo\single.html
     1.0673813s   30.496608ms    876.0912ms     35  shortcodes/imagelist.html
      233.029ms    1.125743ms      6.0009ms    207  tag\list.html
     160.0275ms     393.187µs      6.0004ms    407  partials/meta.html
      92.0318ms    1.095616ms       3.001ms     84  post\single.html
      85.0311ms      87.211µs       2.002ms    975  _default/summary.html
      85.0294ms     85.0294ms     85.0294ms      1  _default/home.atom.xml
      71.0124ms     174.477µs      1.9995ms    407  partials/favicon.html
      61.9952ms     152.322µs      1.9989ms    407  partials/header.html
      57.0233ms      58.485µs      1.0048ms    975  partials/summary-head.html
      51.0087ms    4.250725ms     17.0003ms     12  intern\single.html
      49.0343ms     207.772µs      4.0005ms    236  partials/pagination.html
      47.0204ms    1.621393ms      7.0013ms     29  _default\list.html
      47.0189ms     1.74144ms     17.0021ms     27  photo\panorama.html
      45.0159ms     298.118µs      2.0019ms    151  partials/single-head.html
      39.0082ms      5.5726ms      9.0018ms      7  _default\single.html
      38.9894ms      95.797µs      3.9994ms    407  partials/css/css.html
      21.9956ms    3.142228ms      6.0022ms      7  shortcodes/linux/shortlist.html
      15.9642ms      39.417µs      1.0028ms    405  partials/footer.html
        8.003ms       8.208µs      1.0022ms    975  partials/symbol.html
       4.9989ms      4.9989ms      4.9989ms      1  _default/home.json.json
       3.0031ms       7.378µs      1.0022ms    407  partials/feeds.html
       3.0022ms      3.0022ms      3.0022ms      1  tag\tags.html
       3.0015ms     1.50075ms      2.0008ms      2  shortcodes/linux/list.html
       3.0004ms      3.0004ms      3.0004ms      1  shortcodes/vcard.html
       2.9999ms      2.9999ms      2.9999ms      1  index.html
       2.0031ms     100.155µs      1.0018ms     20  shortcodes/maplink.html
       1.9963ms      1.9963ms      1.9963ms      1  partials/xml/atom.html
       1.0024ms       501.2µs      1.0024ms      2  shortcodes/msg/todo.html
        1.002ms     111.333µs       1.002ms      9  shortcodes/readcode.html
       1.0019ms      200.38µs      1.0019ms      5  _internal/shortcodes/relref.html
       1.0015ms        2.46µs      1.0015ms    407  partials/css/tachyons.html
        1.001ms       1.001ms       1.001ms      1  404.html
       1.0008ms      14.937µs      1.0008ms     67  partials/meta-photo.html
       1.0006ms      1.0006ms      1.0006ms      1  partials/xml/icon.html
       1.0001ms      500.05µs      1.0001ms      2  _default/menu.html
        999.7µs        76.9µs       999.7µs     13  _internal/shortcodes/ref.html
        999.6µs       999.6µs       999.6µs      1  links\list.html
        998.3µs       998.3µs       998.3µs      1  shortcodes/staticindex.html
             0s            0s            0s      1  series\series.html
             0s            0s            0s      5  shortcodes/frac.html
             0s            0s            0s     14  shortcodes/fa/r.html
             0s            0s            0s      1  shortcodes/fa/rf.html
             0s            0s            0s      1  shortcodes/version.html
             0s            0s            0s      2  shortcodes/msg/note.html
             0s            0s            0s      3  shortcodes/msg/warn.html
             0s            0s            0s      6  shortcodes/fa/s.html
             0s            0s            0s      1  robots.txt
             0s            0s            0s      1  intern\list.html
             0s            0s            0s      3  shortcodes/msg/tipp.html
             0s            0s            0s     14  shortcodes/tube.html
             0s            0s            0s    407  partials/css/awesome.html
             0s            0s            0s      3  shortcodes/note.html
             0s            0s            0s      1  shortcodes/footnotes.html
             0s            0s            0s     11  shortcodes/figure.html
             0s            0s            0s      2  shortcodes/fa/sf.html


                   | DE   
+------------------+-----+
  Pages            | 360  
  Paginator pages  |  50  
  Non-page files   | 506  
  Static files     |  79  
  Processed images | 506  
  Aliases          | 186  
  Sitemaps         |   0  
  Cleaned          |   0  

Total in 4334 ms

Metrics for 0.52

render and write pages:
	27.5430537s (29.1825557s)	 3100.42 MB 	2151313 Allocs
render and write Sitemap:
	0s (29.1825557s)	    0.00 MB 	0 Allocs
render and write robots.txt:
	999.5µs (29.1835552s)	    0.01 MB 	54 Allocs
render and write 404:
	1.0013ms (29.1845565s)	    0.17 MB 	2767 Allocs
DEBUG 2018/12/05 10:57:36 Render home to "\\feed.json" with layouts ["_text/page/index.de.json.json" "_text/page/home.de.json.json" "_text/page/list.de.json.json" "_text/page/index.json.json" "_text/page/home.json.json" "_text/page/list.json.json" "_text/page/index.de.json" "_text/page/home.de.json" "_text/page/list.de.json" "_text/page/index.json" "_text/page/home.json" "_text/page/list.json" "_text/index.de.json.json" "_text/home.de.json.json" "_text/list.de.json.json" "_text/index.json.json" "_text/home.json.json" "_text/list.json.json" "_text/index.de.json" "_text/home.de.json" "_text/list.de.json" "_text/index.json" "_text/home.json" "_text/list.json" "_text/_default/index.de.json.json" "_text/_default/home.de.json.json" "_text/_default/list.de.json.json" "_text/_default/index.json.json" "_text/_default/home.json.json" "_text/_default/list.json.json" "_text/_default/index.de.json" "_text/_default/home.de.json" "_text/_default/list.de.json" "_text/_default/index.json" "_text/_default/home.json" "_text/_default/list.json"]
render and write pages:
	5.9997ms (29.1905562s)	    1.29 MB 	27481 Allocs
render and write pages:
	0s (29.1905562s)	    0.23 MB 	2667 Allocs
DEBUG 2018/12/05 10:57:36 Render home to "\\feed.xml" with layouts ["page/index.de.atom.xml" "page/home.de.atom.xml" "page/list.de.atom.xml" "page/index.atom.xml" "page/home.atom.xml" "page/list.atom.xml" "page/index.de.xml" "page/home.de.xml" "page/list.de.xml" "page/index.xml" "page/home.xml" "page/list.xml" "index.de.atom.xml" "home.de.atom.xml" "list.de.atom.xml" "index.atom.xml" "home.atom.xml" "list.atom.xml" "index.de.xml" "home.de.xml" "list.de.xml" "index.xml" "home.xml" "list.xml" "_default/index.de.atom.xml" "_default/home.de.atom.xml" "_default/list.de.atom.xml" "_default/index.atom.xml" "_default/home.atom.xml" "_default/list.atom.xml" "_default/index.de.xml" "_default/home.de.xml" "_default/list.de.xml" "_default/index.xml" "_default/home.xml" "_default/list.xml"]
render and write pages:
	9.810685s (39.0012412s)	  290.65 MB 	87346 Allocs

Template Metrics:

     cumulative       average       maximum         
       duration      duration      duration  count  template
     ----------      --------      --------  -----  --------
  1m29.2906788s  3.188952814s   13.3466626s     28  photo\gallery.html
    15.7300728s  449.430651ms    8.1036009s     35  shortcodes/imagelist.html
    15.6250781s   3.12501562s   10.6635393s      5  photo\single.html
     9.8086853s    9.8086853s    9.8086853s      1  _default/home.atom.xml
     260.0477ms    1.256269ms     16.0024ms    207  tag\list.html
     160.0588ms     393.264µs     15.0023ms    407  partials/meta.html
     148.0082ms   12.334016ms    113.9986ms     12  intern\single.html
      90.9854ms      93.318µs      1.0024ms    975  _default/summary.html
      80.0966ms      953.53µs      3.0014ms     84  post\single.html
      72.0064ms     176.919µs     15.0023ms    407  partials/favicon.html
      69.0045ms    2.555722ms     47.0007ms     27  photo\panorama.html
      64.0048ms      65.645µs       1.002ms    975  partials/summary-head.html
      62.9704ms    2.171393ms      5.0008ms     29  _default\list.html
      62.0372ms     152.425µs      1.0309ms    407  partials/header.html
      57.0023ms     241.535µs      5.0008ms    236  partials/pagination.html
       45.032ms     110.643µs      3.9989ms    407  partials/css/css.html
      40.0306ms     265.103µs      1.0281ms    151  partials/single-head.html
      27.0068ms      66.683µs      1.0021ms    405  partials/footer.html
      26.9973ms    3.856757ms      5.9992ms      7  _default\single.html
       17.003ms       2.429ms      5.9971ms      7  shortcodes/linux/shortlist.html
      13.0003ms      13.333µs       1.002ms    975  partials/symbol.html
       7.0013ms      7.0013ms      7.0013ms      1  tag\tags.html
       5.0006ms      5.0006ms      5.0006ms      1  _default/home.json.json
       4.0029ms       9.835µs      1.0019ms    407  partials/feeds.html
       3.9992ms      1.9996ms      2.0002ms      2  shortcodes/linux/list.html
       3.0316ms      3.0316ms      3.0316ms      1  index.html
       2.9994ms      2.9994ms      2.9994ms      1  partials/xml/atom.html
       2.0017ms      2.0017ms      2.0017ms      1  links\list.html
       1.9974ms     221.933µs       998.9µs      9  shortcodes/readcode.html
       1.0013ms      1.0013ms      1.0013ms      1  shortcodes/staticindex.html
       1.0004ms      14.931µs      1.0004ms     67  partials/meta-photo.html
       1.0003ms      50.015µs      1.0003ms     20  shortcodes/maplink.html
            1ms         200µs           1ms      5  _internal/shortcodes/relref.html
            1ms      90.909µs           1ms     11  shortcodes/figure.html
        999.2µs       499.6µs       999.2µs      2  shortcodes/fa/sf.html
        998.8µs       998.8µs       998.8µs      1  intern\list.html
        998.8µs       998.8µs       998.8µs      1  partials/xml/icon.html
        997.7µs       997.7µs       997.7µs      1  shortcodes/vcard.html
        993.7µs      496.85µs       993.7µs      2  _default/menu.html
             0s            0s            0s     14  shortcodes/fa/r.html
             0s            0s            0s     13  _internal/shortcodes/ref.html
             0s            0s            0s      1  robots.txt
             0s            0s            0s      1  404.html
             0s            0s            0s    407  partials/css/awesome.html
             0s            0s            0s      6  shortcodes/fa/s.html
             0s            0s            0s      3  shortcodes/note.html
             0s            0s            0s      2  shortcodes/msg/note.html
             0s            0s            0s      2  shortcodes/msg/todo.html
             0s            0s            0s     14  shortcodes/tube.html
             0s            0s            0s    407  partials/css/tachyons.html
             0s            0s            0s      1  shortcodes/version.html
             0s            0s            0s      1  shortcodes/footnotes.html
             0s            0s            0s      3  shortcodes/msg/tipp.html
             0s            0s            0s      1  series\series.html
             0s            0s            0s      5  shortcodes/frac.html
             0s            0s            0s      3  shortcodes/msg/warn.html
             0s            0s            0s      1  shortcodes/fa/rf.html


                   | DE   
+------------------+-----+
  Pages            | 360  
  Paginator pages  |  50  
  Non-page files   | 506  
  Static files     |  79  
  Processed images | 506  
  Aliases          | 186  
  Sitemaps         |   0  
  Cleaned          |   0  

Total in 38824 ms

#2

That is obviously a false statement. There may be an issue with your particular site, but that is impossible to tell without … seing your site.


#3

And note that the only relevant change in 0.52 was a re-implementation of the file caches. If you do lots of getJSON or similar, that could be the culprit (that it somehow does not work).


#4

Are you running this on Windows?


#5

Yes, Windows 10 64x

Same effect on Raspbian!


#6

I have looked at your project. You have configured the caches with a maxAge of 2.097152ms, which, not surprisingly, have an … impact.

It is easier to reason about if you use a string, e.g.:

maxAge = "300h" # 300 hours. Or 60m = 60 minutes, -1 = infinite etc.

#7

Thanks

Please add the default value and unit for maxAge to the docs.

https://golang.org/pkg/time/#Duration

A Duration represents the elapsed time between two instants as an int64 nanosecond count. The representation limits the largest representable duration to approximately 290 years.

I started with seconds in my mind.


#8

This is the duration before a cache entry will be evicted, -1 means forever and 0 effectively turns that particular cache off. Uses Go’s time.Duration , so valid values are "10s" (10 seconds), "10m" (10 minutes) and "10h" (10 hours).

This is the current docs which looks good to me. If you have a suggested improvement to the above, I suggest you create a PR.


#9