Stuck solving long render times for Section/single.html

I’m trying to figure out what could be delaying the render of a single.html file I have for the “instagram” section.

From the templateMetricsHints output it looks like it is the one dragging the build longer. But as far as code goes, it’s also quite simple: https://github.com/brunoamaral/Hugo-Now-UI/blob/master/layouts/instagram/single.html

Any idea on where I should look for the problem?

      cache     cumulative       average       maximum
  potential       duration      duration      duration  count  template
      -----     ----------      --------      --------  -----  --------
          0 38.972509678s   13.351322ms  289.713251ms   2919  instagram/single.html
          0 16.497959827s   16.869079ms  446.078343ms    978  _default/single.html
          0   5.62059637s   17.294142ms   38.549714ms    325  instagram/list.html
          0  4.040344179s      539.72µs   98.507677ms   7486  _default/summary.html
          0  3.774279443s    1.293004ms   16.319046ms   2919  instagram/summary.html
          0  3.252707149s    2.094466ms   22.354221ms   1553  _default/list.html
          0  2.976099571s     7.59209ms   20.101125ms    392  index.html
          0  1.966794213s    13.28915ms   40.015377ms    148  _default/section.html
          0  1.703706672s  1.703706672s  1.703706672s      1  json/single.html
          0  1.269414191s  634.707095ms  1.193737823s      2  _default/index.json
          0  1.196933861s   32.349563ms   88.728044ms     37  shortcodes/gallery.html
          0  1.179877149s     847.612µs   89.729679ms   1392  _default/rss.xml
         46  1.169003337s     433.285µs   30.227835ms   2698  partials/header-small.html
         84  1.088891277s     359.133µs    30.17324ms   3032  partials/header-image.html
          0   437.08682ms   218.54341ms  360.239307ms      2  _default/archives.html
          0  264.912572ms   66.228143ms  102.168454ms      4  _default/terms.html
          0  222.980947ms   13.116526ms   25.126626ms     17  _default/page.html
         97  219.412367ms      34.591µs    9.263806ms   6343  partials/footer.html
          0  195.188369ms  195.188369ms  195.188369ms      1  unlisted/list.html
         77  115.659815ms      38.489µs   16.708928ms   3005  partials/pagination.html
          0   106.15005ms   53.075025ms   54.347309ms      2  sitemap.xml
          0   99.709031ms     898.279µs    7.342004ms    111  _internal/shortcodes/tweet.html
          0   92.390005ms   10.265556ms   85.768361ms      9  _internal/shortcodes/highlight.html
         48   41.072835ms     651.949µs    3.383131ms     63  partials/story-card.html
          0   36.541723ms   18.270861ms   19.678373ms      2  _default/profile.html
          0     2.70357ms    1.351785ms    1.414312ms      2  404.html
          0    1.366089ms      75.893µs     172.152µs     18  _internal/shortcodes/youtube.html
        100    1.316505ms       1.315µs      22.063µs   1001  partials/description.html
          0     330.018µs     330.018µs     330.018µs      1  episodes/section.rss.xml
          0     313.718µs     313.718µs     313.718µs      1  _internal/_default/sitemapindex.xml
          0     273.828µs      45.638µs     132.946µs      6  shortcodes/image.html
          0     218.127µs     218.127µs     218.127µs      1  shortcodes/carousel.html
          0     179.385µs      89.692µs     130.882µs      2  shortcodes/panorama.html
          0      97.986µs      24.496µs      54.302µs      4  shortcodes/image-sidebyside.html
          0      97.974µs      48.987µs       69.58µs      2  shortcodes/aside.html
          0      66.233µs      66.233µs      66.233µs      1  shortcodes/bootstrap/youtube.html
          0      65.066µs      65.066µs      65.066µs      1  shortcodes/bootstrap/iframe.html
          0      63.592µs      63.592µs      63.592µs      1  shortcodes/image-floatleft.html
          0      61.682µs      61.682µs      61.682µs      1  shortcodes/image-fullwidth.html
          0      51.435µs      51.435µs      51.435µs      1  shortcodes/image-floatright.html


                   |  EN  |  PT
+------------------+------+------+
  Pages            | 2929 | 3784
  Paginator pages  |  451 |  581
  Non-page files   | 3380 | 1958
  Static files     |  761 |  761
  Processed images | 6613 | 4445
  Aliases          |  746 |  894
  Sitemaps         |    2 |    1
  Cleaned          |    0 |    0

Total in 32595 ms

any virus scanner ??

@ju52 this happens both on my development machine (in the post above) and the server, where it actually takes longer, around 150 000ms. I don’t see the machines’ resources being at risk for the moment.

Not sure how big your instagram images are, but setting them as the background image may take some time.

I just ran a test, without images and the same thing happens. Could the problem be in the baseof.html ?
Maybe if I could set the header image for posts and instagram photos there, saving it as a variable it would cut down on render time.

Do you have the full repo (with hugo and content dir) so we could test locally.

How are you syndicating your photos from Instagram to your Hugo website?

On your local machine is there some kind of an automated trigger that generates the site whenever a new Instagram photo is published? If yes then that might be the culprit for the abnormal render time.

Anyway it is difficult to say what might be going on without seeing an example of the actual use case and not just the theme.

instagram photos are fetched by 4k Stogram. And then I use a modification of a script that I saw here on the forum to convert the photos to page bundles.

I think you were the one that shared the script I based on.

And the github link I shared contains an exampleSite with a reduced version of my content. The full source code contains content I can’t make public.

These last few hours I was looking into the way I manage page resources, because there are a few repeat calls to .Resouces.GetMatch. Since I use blocks, I tried to save them as a scratch variable to avoid repeating the same procedure. No luck. It just outputs an error when I try to save the object in scratch.

I tried cloning your repo, but got this error:

$ git clone https://github.com/brunoamaral/Hugo-Now-UI.git
Cloning into 'Hugo-Now-UI'...
remote: Enumerating objects: 1162, done.
remote: Counting objects: 100% (1162/1162), done.
remote: Compressing objects: 100% (1074/1074), done.
remote: Total 1162 (delta 78), reused 1146 (delta 62), pack-reused 0
Receiving objects: 100% (1162/1162), 7.94 MiB | 4.38 MiB/s, done.
Resolving deltas: 100% (78/78), done.
git-lfs filter-process: git-lfs: command not found
fatal: The remote end hung up unexpectedly
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

It didn’t clone all your files:

$ tree
.
├── README.md
├── archetypes
│   ├── default.md
│   ├── episodes.md
│   └── stories.md
└── assets
    ├── css
    │   ├── bootstrap.min.css
    │   ├── bootstrap.min.css.map
    │   ├── now-ui-kit.css
    │   ├── now-ui-kit.css.map
    │   └── now-ui-kit.min.css
    ├── fonts
    │   ├── nucleo-license.md
    │   ├── nucleo-outline.eot
    │   ├── nucleo-outline.ttf
    │   ├── nucleo-outline.woff
    │   └── nucleo-outline.woff2
    └── img

5 directories, 14 files

Ok. That was a bash script to generate the bundles before generating a Hugo Site. So it seems that you are not doing anything funky with automated triggers etc.

Then I suppose that you can try to clean up that /instagrams/single.html template.

At first glance the following seems like a double call:
{{ $instagram := (.Page.Resources.GetMatch "header").Permalink | relURL }}

Why use that and not the following since you want a relative URL:
{{ $instagram := (.Page.Resources.GetMatch "header").RelPermalink }}

Again these seem like double calls:
{{ .PrevInSection.URL | relLangURL }}
{{ .NextInSection.URL | relLangURL }}

(although I admit I am not familiar with Hugo’s multilingual features)

Try out these small changes and then check out those metrics again.

Also a final tip: stop using .URL and instead use .Permalink or .RelPermalink

It works on my end though:

git clone https://github.com/brunoamaral/Hugo-Now-UI.git
Cloning into 'Hugo-Now-UI'...
remote: Enumerating objects: 1162, done.
remote: Counting objects: 100% (1162/1162), done.
remote: Compressing objects: 100% (1074/1074), done.
remote: Total 1162 (delta 78), reused 1146 (delta 62), pack-reused 0
Receiving objects: 100% (1162/1162), 7.94 MiB | 888.00 KiB/s, done.
Resolving deltas: 100% (78/78), done.

@alexandros hmm okay something on my end then, thanks

@brunoamaral just ended up downloading a zip of your repo. But when I try to build your example site, I get an error because you don’t have a content/ folder. Maybe just add a dummy one for now, with a .gitkeep inside

Edit: so I added a dummy content/ folder, but when rebuilding your example site, I get this now

$ hugo server --source exampleSite --config exampleSite/config.toml --themesDir ../.. --theme Hugo-Now-UI-master
Building sites … ERROR 2018/11/27 16:10:15 render of "taxonomyTerm" failed: execute of template failed: panic in Execute: runtime error: invalid memory address or nil pointer dereference. See "https://github.com/gohugoio/hugo/issues/5327" for the reason why we cannot provide a better error message for this.
ERROR 2018/11/27 16:10:15 render of "home" failed: execute of template failed: panic in Execute: runtime error: invalid memory address or nil pointer dereference. See "https://github.com/gohugoio/hugo/issues/5327" for the reason why we cannot provide a better error message for this.
Total in 131 ms
Error: Error building site: failed to render pages: render of "taxonomyTerm" failed: execute of template failed: panic in Execute: runtime error: invalid memory address or nil pointer dereference. See "https://github.com/gohugoio/hugo/issues/5327" for the reason why we cannot provide a better error message for this.

Edit 2: link from error message https://github.com/gohugoio/hugo/issues/5327

@brunoamaral As @zwbetz said you need to have all those folders in your exampleSite under /content/ otherwise the test site will not generate.

Anyway I already did that but now I’m getting the following for every single image in your bundles:

ERROR 2018/11/28 00:07:32 failed to render pages: render of "page" failed: "/home/alex/Desktop/Hugo-Now-UI/layouts/_default/baseof.html:95:16": execute of template failed: template: _default/single.html:95:16: executing "_default/single.html" at <.Fill>: error calling Fill: fill /home/alex/Desktop/Hugo-Now-UI/exampleSite/content/page/elements/images/IMG_0160.JPG: image: unknown format

Something is wrong with the JPGs in your bundles. They’re not recognized on my end in Xubuntu 18.04

that could be because I use git lfs to host the jpg image. try git lfs install or git lfs fetch.

When you create your theme’s docs, that’ll be a good thing to add :slight_smile: