Then the next most costly item is footer.html, I wonder why, I’ll look through that too but could use another pair of eyes:
<footer data-pagefind-ignore id="footer" class="footer bg4 shadow">
<div class="container">
<div class="row">
{{ if isset .Site.Params "about_us" }}
<div class="col-md-2 col-sm-6">
<div class="h3">Psychedelics</div>
<ul class="">
{{ range .Site.Menus.footer }}
<li class="" style="margin: 10px 0">
<a href="{{ .URL }}">{{ .Pre }}{{ .Name }}</a>
</li>
{{ end }}
</ul>
</div>
<!-- /.col-md-3 -->
{{ end }}
<div class="col-md-3 col-sm-6">
{{ if isset .Site.Params "recent_posts" }}
{{ if .Site.Params.recent_posts.enable }}
<div class="h3">{{ i18n "recentPosts" }}</div>
<div class="blog-entries">
{{ range first 6 (where $.Site.RegularPages "Section" "!=" "") }}
<div class="item same-height-row clearfix">
<div class="image same-height-always">
<a href="{{ .Permalink }}">
{{ if isset .Params "banner" }}
<img src="{{ .Site.BaseURL}}img/placeholder.png" data-src="{{.Site.BaseURL}}{{ .Params.banner }}" class="img-responsive" alt="{{.Title}}" title="{{.Title}}" width="255" height="192" loading="lazy" />
{{ else }}
<img src="{{ .Site.BaseURL}}img/placeholder.png" class="img-responsive" alt="{{.Title}}" title="{{.Title}}" width="255" height="192" loading="lazy"/>
{{ end }}
</a>
</div>
<div class="name same-height-always">
<p style=""><a href="{{ .Permalink }}">{{ .Title }}</a></p>
</div>
</div>
{{ end }}
</div>
<hr class="hidden-md hidden-lg">
{{ end }}
{{ end }}
</div>
<!-- /.col-md-3 -->
<div class="col-md-3 col-sm-6">
<div class="h3">About Hash B.</div>
<div style="">{{ .Site.Params.about_us | safeHTML }}</div>
<hr class="hidden-md hidden-lg hidden-sm">
</div>
{{ if isset .Site.Params "address" }}
<div class="col-md-3 col-sm-6">
<div class="h3">{{ i18n "contactTitle" }}</div>
<p> Message Hash Borgir:</p>
<ul>
<li>If you want to write for PsychedelicsDaily</li>
<li>For any questions related to psychedelics & Microdosing</li>
<li>To book Hash for lectures, talks, or Trip Sitting.</li>
<li>Send your comments, questions, or concerns.</li>
<li>Or if you just want to say hi :)</li>
</ul>
<!-- {{ .Site.Params.address | safeHTML }} -->
<a href="/contact" class="btn btn-small btn-template-main shadow">Contact Hash</a>
<a class='read-more btn btn-small btn-template-main shadow' href='/about/'>About Us</a>
{{ partial "site_social.html" . }}
<hr class="hidden-md hidden-lg hidden-sm">
</div>
<!-- /.col-md-3 -->
{{ end }}
</div>
</div>
<!-- /.container -->
<div style="padding: 12px;">
<p class="" style="text-align: center;">I recently started to display some links to relevant products on amazon in order to try and cover server and hosting costs. I do sincerely apologize about the advertising and we hope that you understand.</p>
<p class="center">Design, development, server administration, research and writing powered by potent psychedelics.</p>
</div>
</footer>
<!-- /#footer -->
<!-- *** FOOTER END *** -->
<!-- *** COPYRIGHT ***-->
<div id="copyright">
<div class="container">
<div class="col-md-12">
{{ if isset .Site.Params "copyright" }}
<p class="pull-left">{{ .Site.Params.copyright | safeHTML }}
<a class="w3c" href="https://jigsaw.w3.org/css-validator/check/referer">
<span class="w3">
W3C
</span>
Valid CSS 3
</a>
<a class="w3c" href="https://validator.w3.org/nu/?doc=https%3A%2F%2Fwww.psychedelicsdaily.com%2F">
<span class="w3">
W3C
</span>
Valid HTML5
</a>
</p>
{{ end }}
<a class="subscribe-button icon-feed" href='/index.xml'>RSS Feed</a>
<p class="pull-right">
Design By: <a href="https://stoned.io">Hash C. Borgir</a>
</p>
</div>
</div>
</div>
<!-- /#copyright -->
<!-- *** COPYRIGHT END *** -->
Look for other suspects where you might ranging through every page on every page.
50,000 x 50,000 = 2.5 Billion
You’re doing this 50,000 times:
{{ range first 6 (where $.Site.RegularPages "Section" "!=" "") }}
I’m not sure I follow the last bit. How would I display the last 6 recent articles in footer without costing it 50k times? Cache the footer? Or…?
Or give a slice of sections so it doesn’t have to do a check condition?
Hash_Borgir:
Cache the footer?
Yes, or just this portion of it. You should be looking for caching possibilities everywhere.
Think about building a house. Are you going to walk back out to the truck every time you need a nail?
I gotcha. How would I cache just this iteration in footer.html? I’ve partialCache’d the footer everywhere. Every dynamic module/wiget (not many) left alone and caching now every static object that doesn’t need to change form page to page.
Here’s the next most costly item: author.html
{{ if .Site.Params.widgets.authors }}
{{ $baseurl := .Site.BaseURL }}
<div data-pagefind-ignore class="panel sidebar-menu bg4">
<div class="panel-heading">
<p class="h4 panel-title"> {{ i18n "authors" }}</p>
</div>
<div class="panel-body">
<ul class="authors-list nav nav-pills nav-stacked sidebar-authors">
{{ range $name, $items := $.Site.Params.authors }}
{{ range $key, $val := $items }}
{{ if eq $key "name" }}
<li>
<a href="{{ $baseurl }}authors/{{ $val | urlize | lower }}">
🧔 {{ $items.display_name }}
({{ len (where $.Site.Pages ".Params.author" $name) }})
</a>
</li>
{{ end }}
{{ end }}
{{ end }}
</ul>
</div>
</div>
{{ end }}
And the one after that most costly, random.html
{{ if .Site.Params.widgets.featured }}
<div class="panel panel-default sidebar-menu bg4">
<div data-pagefind-ignore class="panel-heading">
<p class="h4 panel-title"> You might also like</p>
</div>
<div class="sidebar-featured">
{{ range first 24 (where .Site.RegularPages "Section" "not in" (slice "faq" "community")) | shuffle }}
<div class="item same-height-row clearfix">
{{ $slice := slice "🍄" "🌿" "🌱" "🌵" "🧘" "🙏" "🍂" "🌈" "❓" "✨" "🌟"}}
{{ $shuffled := shuffle $slice }}
{{ $random := index $shuffled 0 }}
<!--<div class="name same-height-always">
<h4><a href="{{ .Permalink }}">{{ .Title }}</a></h4>
</div>-->
<div class="image">
<a href="{{ .Permalink }}">
{{ if isset .Params "banner" }}
<!--<img src="{{ .Site.BaseURL}}img/placeholder.png" data-src="{{ .Site.BaseURL}}{{ .Params.banner }}" class="img-circle shadow" alt="{{.Title}}" title="{{.Title}}" width="40" height="40">-->{{$random}} {{ .Title }}
{{ else }}
<!--<img src="{{ .Site.BaseURL}}img/placeholder.png" class="img-circle" alt="{{.Title}}" title="{{.Title}}" width="40" height="40">-->🌿 {{ .Title }}
{{ end }}
</a>
</div>
<!--<div><h4 >{{.Description}}</h4></div>-->
</div>
{{ end }}
</div>
<hr class="hidden-md hidden-lg">
</div>
{{end}}
You’re looking at every page, on every page. Come on…
I honestly can’t even remember what I was doing years ago. Wow.
Same. That’s 2.5 billion comparisons.
How might I optimize that? Caching again? Which I just learned about tonight. Which is fantastic!
Ok I’ll get all the changes in and do another templte metrics test.
As for the goofy stuff I did, checking things on every page, seemed to never notice it on a few hundred pages, but yeah, those are problems. I gotta figure out what I was trying to do a few years ago and optimize/recode some stuff.
Thank you very much. Most helpful.
I remember what I was trying to do. Display the total # of articles by that author, next to the author name. What would be the most optimal way to do that?
{{ range $name, $items := $.Site.Params.authors }}
{{ range $key, $val := $items }}
{{ if eq $key "name" }}
<li>
<a href="{{ $baseurl }}authors/{{ $val | urlize | lower }}">
🧔 {{ $items.display_name }}
({{ len (where $.Site.Pages ".Params.author" $name) }})
</a>
</li>
{{ end }}
{{ end }}
{{ end }}
Please advise. For the time being, it’s not necessary or vital to display this on the sidebar, so I’ll disable it, but I’d like to find out how to do this optimally in case I might need to. Thank you so much! You’re the best.
Pull that into a partial, call it with partialCached, using the author as as the cache key.
Or cache the whole thing, again, using the author as the cache key. Hard to say looking a bits and pieces of your code.
Template Metrics:
cumulative average maximum cache percent cached total
duration duration duration potential cached count count template
---------- -------- -------- --------- ------- ------ ----- --------
47m56.335089852s 177.354488ms 456.765463ms 0 0 0 16218 _default/single.html
33m29.994986524s 121.193547ms 337.596316ms 63 0 0 16585 partials/sidebar.html
16m24.853623451s 59.38219ms 254.237421ms 26 0 0 16585 partials/widgets/random.html
11m59.087840608s 44.338872ms 237.344581ms 28 0 0 16218 partials/related.html
7m48.157319884s 28.226053ms 138.492169ms 94 0 0 16586 partials/widgets/recent.html
7m0.47508267s 25.352733ms 121.680465ms 19 0 0 16585 partials/widgets/faq.html
49.061556268s 2.95801ms 149.760643ms 9 1 188 16586 partials/head.html
32.337059797s 212.743814ms 290.734373ms 0 0 0 152 drugs/single.html
14.183324094s 854.88µs 71.089107ms 100 99 16421 16591 partials/footer.html
12.878318069s 130.08402ms 195.939541ms 0 0 0 99 section/community.html
12.471682324s 138.574248ms 226.036178ms 0 0 0 90 _default/list.html
11.362771499s 684.958µs 79.662173ms 12 99 16421 16589 partials/nav.html
10.511584784s 557.377µs 92.615181ms 98 86 16227 18859 partials/share.html
8.293333075s 499.869µs 67.060804ms 99 100 16584 16591 partials/widgets/featured.html
8.03943046s 485.033µs 99.583655ms 100 99 16420 16575 partials/author.html
7.452620441s 42.344434ms 100.862753ms 0 0 0 176 shortcodes/related.html
7.416729476s 447.168µs 57.011919ms 10 99 16421 16586 partials/scripts.html
4.847572701s 292.304µs 73.886ms 37 98 16321 16584 partials/breadcrumbs.html
3.255965748s 196.426µs 65.461373ms 100 99 16421 16576 partials/tagcloud.html
3.18862018s 192.259µs 67.09023ms 100 100 16584 16585 partials/widgets/categories.html
2.349002122s 141.634µs 59.739518ms 100 100 16584 16585 partials/widgets/search.html
2.262385987s 10.982456ms 85.9734ms 96 0 0 206 partials/carousel_featured.html
1.870572301s 207.841366ms 217.782361ms 0 0 0 9 page/single.html
1.697442792s 102.341µs 54.891913ms 100 100 16584 16586 partials/widgets/tags.html
1.414451255s 86.405µs 54.237217ms 99 99 16217 16370 partials/most_read.html
1.300384083s 6.668636ms 315.546508ms 0 0 0 195 _internal/_default/rss.xml
1.143645666s 127.07174ms 163.62196ms 0 0 0 9 section/psychedelics.html
1.080139314s 65.127µs 53.822834ms 100 100 16584 16585 partials/widgets/facebook.html
753.373165ms 45.942µs 10.601996ms 10 0 0 16398 partials/seo/twitter.html
671.651535ms 167.912883ms 219.22018ms 0 0 0 4 section/blog.html
628.141178ms 89.734454ms 122.718859ms 0 0 0 7 shortcodes/list_tags.html
449.91717ms 449.91717ms 449.91717ms 0 0 0 1 index.html
397.724421ms 198.86221ms 215.712384ms 0 0 0 2 section/drugs.html
336.631852ms 575.439µs 68.200926ms 71 66 384 585 partials/recent.base.html
277.267742ms 17.012µs 237.514µs 0 0 0 16298 _internal/alias.html
231.170778ms 231.170778ms 231.170778ms 0 0 0 1 _default/index.json
194.270434ms 194.270434ms 194.270434ms 0 0 0 1 section/news.html
175.347749ms 175.347749ms 175.347749ms 0 0 0 1 _internal/_default/sitemap.xml
154.776519ms 874.443µs 15.815592ms 0 0 0 177 shortcodes/signup.html
119.693133ms 119.693133ms 119.693133ms 100 0 0 1 partials/sections_list.html
98.241437ms 498.687µs 21.706955ms 100 98 194 197 partials/drugs.html
65.852407ms 144.73µs 10.24792ms 0 0 0 455 shortcodes/notice.html
54.046377ms 54.046377ms 54.046377ms 100 0 0 1 partials/recent_psychedelics.html
39.138739ms 39.138739ms 39.138739ms 100 0 0 1 partials/recent_faq.html
34.188942ms 34.188942ms 34.188942ms 100 0 0 1 partials/recent_blog.html
26.858911ms 95.244µs 4.792496ms 0 0 0 282 shortcodes/imgcap.html
24.682914ms 24.682914ms 24.682914ms 100 0 0 1 partials/recent_drugs.html
13.735335ms 83.244µs 1.656472ms 16 0 0 165 partials/seo/schema.html
5.702445ms 34.56µs 297.099µs 100 0 0 165 partials/seo/google_analytics.html
4.338365ms 173.534µs 455.703µs 0 0 0 25 shortcodes/blockquote.html
1.131105ms 1.131105ms 1.131105ms 0 0 0 1 404.html
560.212µs 560.212µs 560.212µs 100 0 0 1 partials/testimonials.html
517.491µs 517.491µs 517.491µs 100 0 0 1 partials/features.html
512.149µs 3.012µs 261.16µs 100 0 0 170 partials/site_social.html
432.358µs 432.358µs 432.358µs 100 0 0 1 partials/clients.html
398.483µs 398.483µs 398.483µs 100 0 0 1 partials/team.html
268.485µs 67.121µs 151.65µs 0 0 0 4 shortcodes/radio.html
241.252µs 241.252µs 241.252µs 100 0 0 1 partials/signup.html
230.821µs 230.821µs 230.821µs 0 0 0 1 _internal/shortcodes/youtube.html
208.75µs 208.75µs 208.75µs 100 0 0 1 partials/welcome.html
172.431µs 974ns 41.73µs 100 0 0 177 partials/mailchimp.html
168.633µs 168.633µs 168.633µs 0 0 0 1 shortcodes/aboutimg.html
165.577µs 165.577µs 165.577µs 0 0 0 1 shortcodes/calc.html
120.511µs 120.511µs 120.511µs 100 0 0 1 partials/hero.html
3.407µs 3.407µs 3.407µs 0 0 0 1 robots.txt
| EN
-------------------+--------
Pages | 16772
Paginator pages | 11
Non-page files | 28
Static files | 2451
Processed images | 0
Aliases | 16298
Sitemaps | 1
Cleaned | 0
Total in 193567 ms
Got it down to 3 minutes, that’s not bad. I’ll keep working at trying to optimize it and rethink what I need/want to display or not.
Thank you. If you have any other tips, that’d be helpful too.
If you’d like to see the source, please let me know your email and I can add you to the repository, I don’t want to post my code/link to repository publicly. Thank you.
I have to figure out why so many aliases are being generated. Where might I look for that?
AFAIK, only a few pages have aliases, rest should not.
+++
author = "hash"
authors = "hash-borgir"
banner = "https://news.psychedelicsdaily.com/cache/thumb/0ae76c673b9a854b04c783087597a8eb.jpg"
date = "2020-11-07T06:18:18-07:00"
description = ""
featured = false
images = ["https://patch.com/img/cdn20/users/57069/20201107/013927/styles/patch_image/public/marijuana-shutterstock-126159383___07133914296.jpg"]
reviewCount = "169814"
title = "5-states-pass-pot-ballot-measures-including-new-jersey"
tags = ["News", "cannabis"]
aliases = [""]
+++
Here is the front matter for the news pages. It seems like an empty “” alias tag is causing that? Could that be it?
find . -type f -name '*.md' -exec sed -i 's/aliases = \[""\]//g' {} + removed all empty alias front matter things and will try again.
Ok, that was it. Empty alias was cuasing empty alias pages to be generated.
Error: Error building site: open /home/stoned/websites/websites/psychedelicsdaily.com-hugo/public/about/index.html: no such file or directory
Also unsure why I get this error. All the markdown files are correctly in place.
But with Caching and giving up some few things, I was able to get
Start building sites …
hugo v0.110.0-e32a493b7826d02763c3b79623952e625402b168+extended linux/amd64 BuildDate=2023-01-17T12:16:09Z VendorInfo=snap:0.110.0
| EN
-------------------+--------
Pages | 17999
Paginator pages | 464
Non-page files | 28
Static files | 2451
Processed images | 0
Aliases | 418
Sitemaps | 1
Cleaned | 0
Total in 23505 ms
Arif
February 19, 2023, 11:05am
38
Just a tip. Throrougly check if adding partialCached to the post share and random partials breaks those partials. For example, the post share in all posts will only point to one post, and the random posts will not be ‘random’ per page since they will be the same.
Also, I’m not sure how this might help, but you can try to cache the images on your site. E.g.
[caches]
[caches.images]
dir = ':resourceDir/_gen'
maxAge = "10h"
I also have one site with over 50k pages (no images), but since it is using pages from data , it takes over 3 minutes to build. I have cached a lot of the templates and resources, but even without caching, the build was just 10 minutes!
Yes, I figured as much so now I am having to carefully decide where I want randomness and where I don’t. Thanks for the tip.
Can you tell me about your dev workflow with so many files? hugo serve must be very slow.
I’m thinking after the initial build, don’t build them again, using some front matter.So the content directory with tens of thousands of pages doesn’t get regenerated evertytime I launch hugo serve
Start building sites …
hugo v0.110.0-e32a493b7826d02763c3b79623952e625402b168+extended linux/amd64 BuildDate=2023-01-17T12:16:09Z VendorInfo=snap:0.110.0
| EN
-------------------+--------
Pages | 49286
Paginator pages | 2438
Non-page files | 28
Static files | 2451
Processed images | 0
Aliases | 419
Sitemaps | 1
Cleaned | 0
Total in 443982 ms