Large taxonomy does result in empty frontpage

[EstherM]

Hi all,
we are running a really large website with Hugo with more than 40,000 images ftcommunity.de. The website content has mostly been imported from a previous website. The website is in German only.

We already use 2 taxonomies: one for “legacy-id” (legacy URLs which are forwarded to the current content) and one for “konstrukteure” (designers). Both work pretty well.

About 2000 images have “schlagworte” (keywords). We now want to add a new taxonomy “schlagworte” for those keywords. However, when I run hugo-0.66.0, the keyword list as prepared in the layout file builds nicely. But the front page of our website is empty.

When I start the latest hugo-0.78.2, I get the following error message:
“Error: Error building site: missing taxonomy: schlagworte”.

Please let us know where we missed something.
The sources for the relevant branch are on Gitlab
Thank you for any help

Cheers
Esther

[ju52]

Hi,

how looks your config file?

[taxonomies]
schlagwort = “schlagworte”

anything under [permalinks] ?

If you generate the site in publicDIr, what is in the HTML file?

If you have a reporitory on github … would help.

[EstherM]

Dear ju52,
thanks for answering.
This is the relevant part of config.toml: (lines commented out are here given in `` for better readability)

[taxonomies]
‘# category = “categories”’
‘# tag = “tags”’
konstrukteur = “konstrukteure”
legacy_id = “legacy_id”
schlagwort = “schlagworte”`

We have no permalinks section in our config.toml.

I tried to link to our Gitlab repo. Sorry if this did not work. Here is the full link to the branch under discussion:
https://gitlab.com/ftcommunity/website-hugo-poc/-/tree/new-taxonomy-keywords

Here is an example of a frontmatter containing the “schlagworte”-taxonomy:
https://gitlab.com/ftcommunity/website-hugo-poc/-/tree/new-taxonomy-keywords/content/bilderpool/modelle/optische-instrumente/47687

The index.html as generated with hugo-0.66.0 contains an empty “body-inner”:

 <div id="body-inner">
   <h1>..</h1>

</div>

Cheers
Esther

[ju52]

Tryed to download, ~5GB is a lot. Can you make a smaller sample?

[EstherM]

Dear ju52,
thanks for your feedback. Yes, we have a large amount of data on our website. We have a smaller subset for testing at https://gitlab.com/ftcommunity/testsite, without the branch for schlagworte taxonomy.
However, the tests with this small subset show no problems.
The test website builds without any issues both on hugo-0.66.0 and on 0.78.2.
Unfourtunately, the issue is only reproducible with the full data set. Sorry for that.
Thank you again.
Cheers
Esther

[ju52]

Run Hogo and check the used memory.
Slow version with –renderToDisk - could help?

[EstherM]

I always run ‘hugo’ to render the site to disk, because it is too large to keep it in memory.
This is the command and the output:

~/bin/hugo_0.66.0_Linux-64bit/hugo

                   |   EN    
-------------------+---------
  Pages            | 168571  
  Paginator pages  |      0  
  Non-page files   |  83637  
  Static files     |     80  
  Processed images |      2  
  Aliases          |      0  
  Sitemaps         |      1  
  Cleaned          |      0  

Total in 523158 ms

The memory consumption never exceeded 72 %. The behaviour of my local computer during the build process was in no way different from the ordinary build process (without the additional taxonomy).
I have no idea, why the content of the startpage is not build with the additional taxonomy. For further reference, I now pushed the test data into a new branch: https://gitlab.com/ftcommunity/testsite/-/tree/new-taxonomy-keyword.
Perhaps you could advice me on the difference except for the mass of data. The theme is exactly the same in both cases. Is there an upper limit to the number of taxonomy terms?
Thanks for your help.
Cheers
Esther

[ju52]

user github example, runs nice for me, here my first comments

— config.toml —

baseURL= "/"

Do not use this in config.toml, set it in hugo command line like
hugo server --watch -b http://localhost/ --noHTTPCache --navigateToChanged

[blackfriday]

blackfriday is gone in new hugo versions, please update!
Newer versions have less errors and are faster,

— basof.html —

<link href="{{"css/nucleus.css" | relURL}}{{ if $assetBusting }}?{{ now.Unix }}{{ end }}" rel="stylesheet">

better way to use

{{ $nuccss := resources.Get "css/nucleus.css" }}
{{ if hugo.IsProduction }} {{ $nuccss  = $nuccss | minify | resources.Fingerprint site.Params.digest }} {{end}}
<link type=text/css rel=stylesheet href={{ $nuccss.RelPermalink }} {{ if hugo.IsProduction }} integrity={{ $nuccss.Data.Integrity }} {{end}} media="screen, print" />

nucleus.css must be in an asset directory testsite\themes\website-layout\layouts\assets

put this in a partial template and call it with {{ partialCached allcss . }} to transform it only one time. Here allcss.html
Same for scripts …

— bilderpool —

very time consuming generation for breadcrumb and navigation, for maintenance better to use patials!
This can break the build process … (?)

---

In my own testsite I tryed to use a lot of hugo tricks …
GitHub - gj52/HugoSample: Hugo Sample

[EstherM]

Thank you for having a closer look at our Hugo setup.

Do I understand correctly, that you discovered a number of major issues which have to be resolved before the issue concerning th taxonomomy build can be further investigated?

I will discuss this insight with the other development team members. Resolving these issues will certainly take some time. I will be back with additional questions or any results.
If there is any quicker fix for the issue described in my first post, I will be more than happy to learn about it. Thank you.

Cheers
Esther

[EstherM]

The solution might be very simple:
one of the data files contains the following line:

schlagworte: [".."]

More tests will follow.

[EstherM]

Hi all,

The answer for the questions asked above is:
Tags must not contain "..". Two dots (..) is most probably interpreted as link to the parent directory, which then results in a strange behaviour.
Perhaps it is a bug in Hugo that it does not check for correct strings when building the taxonomy. I did not test, if this behaviour can also be produced with other characters used in directory names such as slash, backslash, asterisk, etc.

The problem is not related to the number of tags, the number of taxonomies or the number of pages. It is also not caused by the config.toml used or any of the templates.

What was done to find this solution? To start with, the site builds fine with a small subset of data. I tryed to identify the maximum number of possible pages, so I build the master site with only a small number of pages and then added more. When the problem occured, I took away some of the pages whitch caused no problem. Quickly, it became clear, that the issue is not caused by the number of pages, but with one single page, which was then easy to idenitfy. When this line is removed, the website works as intended.

Lesson learned: do systematic tests. Don’t waste your time on unrelated issues.

Cheers
Esther