Generating taxonomy pages in a content adapter seems to break the taxonomy

hugo v0.138.0+extended+withdeploy darwin/arm64 BuildDate=unknown VendorInfo=nixpkgs

Title does the heavy lifting on describing the problem, but I can get more specific here:

I’m attempting to use my Twitter export JSON file to generate an archive. Each tweet is its own page in a “tweets” section. Mentions and hashtags are taxonomies for tweets. The tweet pages are working well, and there is no metadata associated with hashtags, so I don’t bother generating pages for them, but mentions are another story. I want to generate pages for each item in the mentions taxonomy because I want to make use of the name and screen_name properties.

The relevant parts of the data look like this:

[
  {
    "tweet": {
        "entities" : {
          "user_mentions" : [
            {
              "name" : "@ramsey@phpc.social",
              "screen_name" : "ramsey",
              "indices" : [
                "0",
                "7"
              ],
              "id_str" : "7794552",
              "id" : "7794552"
            }
          ]
        }
    }
]

I’ve put together the following content adapter, which lives in /content/mentions/_content.gotempl:

{{ $data := slice }}
{{ with resources.Get "json/tweets.json" }}
    {{ $data = . | transform.Unmarshal }}

    {{ $mentions := dict }}

    {{ range $data }}
        {{ range .tweet.entities.user_mentions }}
            {{ $mentions = merge $mentions (dict .screen_name (dict
                "name" .name
                "screen_name" .screen_name
                "id" .id_str
            ))}}
        {{ end }}
    {{ end }}

    {{ range $key, $value := $mentions }}
        {{ $mentionParams := dict
            "screen_name" $value.screen_name
        }}

        {{ $mentionPage := dict
            "title" $value.name
            "kind" "taxonomy"
            "path" $value.screen_name
            "params" $mentionParams
        }}

        {{ $.AddPage $mentionPage }}
    {{ end }}
{{ end }}

This produces the pages for the taxonomy, but in the list.html template for my tweets section, I have the following:

       <div>
            <h3>Top mentions</h3>
            <ol>
                {{ range first 5 .Site.Taxonomies.mentions.ByCount }}
                <li><a href="{{ .Page.RelPermalink }}">{{ .Page.Title }}</a> ({{ .Count }})</li>
                {{ end }}
            </ol>
            <p><a href="{{ .Site.Taxonomies.mentions.Page.RelPermalink }}">All mentions</a></p>
        </div>

The range gets an empty slice, and the .Site.Taxonomies.mentions.Page.RelPermalink line throws the following error:

ERROR Rebuild failed: render: failed to render pages: render of "section" failed: "/.../website/layouts/tweets/list.html:28:32": execute of template failed at <.Site.Taxonomies.mentions.Page.RelPermalink>: nil pointer evaluating page.Page.RelPermalink

In a previous iteration, I used Python to turn the archive into pages and this exact template code worked. I’m using the same templates and have made an effort to keep the content the same. My goal is to share this project with others, and I hoped to eliminate the Python dependency. I also wanted an excuse to learn about content adapters.

Anyway, curious if anyone has any idea what might be happening here. Any help is very appreciated.

Appending some additional info.

Here is my config.toml

baseURL = "https://nicksloan.com/"
languageCode = "en-us"
title = "Nick Sloan"
enableRobotsTXT = true

[params]
  [params.author]
    name = 'Nick Sloan'

[outputs]
home = ['html']
section = ['html', 'rss']
taxonomy = ['html', 'rss']
term = ['html', 'rss']

[taxonomies]
hashtag = "hashtags"
mention = "mentions"

[markup]
  [markup.highlight]
    style = 'dracula'
    lineNos = true
    noClasses = false

  [markup.goldmark.renderer]
    unsafe = true

Here is the tweets section _content.gotempl

{{ $data := slice }}
{{ with resources.Get "json/tweets.json" }}
    {{ $data = . | transform.Unmarshal }}

    {{ $pages := slice }}

    {{ range $data }}

        {{ $fullText := .tweet.full_text }}

        {{ $replacements := slice }}

        {{ $tweetMentions := slice }}
        {{ $tweetHashTags := slice }}

        {{ range .tweet.entities.user_mentions }}
            {{ $tweetMentions = $tweetMentions | append .screen_name }}

            {{ $substr := delimit (slice "@" .screen_name) "" }}
            {{ $replacement := delimit (slice "<a href=\"/mentions/" .screen_name "\">" $substr "</a>") "" }}
            {{ $fullText = replace $fullText $substr $replacement }}
        {{ end }}

        {{ range .tweet.entities.hashtags }}
            {{ $tweetHashTags = $tweetHashTags | append .text }}

            {{ $substr := delimit (slice "#" .text) "" }}
            {{ $replacement := (delimit (slice "<a href=\"/hashtags/" .text "\">" $substr "</a>") "") }}
            {{ $fullText = replace $fullText $substr $replacement }}
        {{ end }}

        {{ range .tweet.entities.urls }}
            {{ $replacement := (delimit (slice "<a href=\"" .expanded_url "\">" .display_url "</a>") "") }}
            {{ $fullText = replace $fullText .url $replacement }}
        {{ end }}

        {{ $content := dict
            "value" $fullText
            "mediaType" "text/html"
        }}

        {{ $dates := dict "date" (time .tweet.created_at) }}

        {{ $params := dict 
            "hashtags" $tweetHashTags
            "mentions" $tweetMentions
        }}

        {{ $page := dict 
            "content" $content
            "kind" "page"
            "path" .tweet.id_str
            "title" ($fullText | plainify)
            "dates" $dates
            "params" $params
        }}

        {{ $.AddPage $page }}
    {{ end }}
{{ end }}

Secondarily, if I’ve done something wrong by using go-html-template for my code fences in that post, let me know what I should have done instead. I was hoping the discourse rules would have something to say on the subject, but I can’t seem to find them again.

Hugo version?

Added to the post, but here it is: hugo v0.138.0+extended+withdeploy darwin/arm64 BuildDate=unknown VendorInfo=nixpkgs

What does the taxonomies key in your site configuration contain?

@jmooring Updated the top post again with that and the tweets/_content.gotmpl (just in case). Sorry for making you nickel and dime the details out of me. Didn’t know how much context would be relevant.

OK, so you have two content adapters:

  • content/mentions/_content.gotmpl (these are page kind “taxonomy”)
  • content/tweets/_content.gotmpl (these are page kind “page”)

And they each use different data sources. Correct?

Same data source. I redacted the tweet bits. A question I had while doing this was whether I could use the same content adapter to generate the tweets pages and the mentions pages, but it didn’t seem to like ../mentions/{{ screen_name}} as a path. Here’s a full tweet example:

[
    {
      "tweet" : {
        "edit_info" : {
          "initial" : {
            "editTweetIds" : [
              "1604701767249313796"
            ],
            "editableUntil" : "2022-12-19T05:24:49.000Z",
            "editsRemaining" : "5",
            "isEditEligible" : false
          }
        },
        "retweeted" : false,
        "source" : "<a href=\"http://twitter.com/download/iphone\" rel=\"nofollow\">Twitter for iPhone</a>",
        "entities" : {
          "hashtags" : [ ],
          "symbols" : [ ],
          "user_mentions" : [
            {
              "name" : "Edward Z. Yang",
              "screen_name" : "ezyang",
              "indices" : [
                "0",
                "7"
              ],
              "id_str" : "14930686",
              "id" : "14930686"
            }
          ],
          "urls" : [ ]
        },
        "display_text_range" : [
          "0",
          "49"
        ],
        "favorite_count" : "0",
        "in_reply_to_status_id_str" : "1604701662714773504",
        "id_str" : "1604701767249313796",
        "in_reply_to_user_id" : "16835872",
        "truncated" : false,
        "retweet_count" : "0",
        "id" : "1604701767249313796",
        "in_reply_to_status_id" : "1604701662714773504",
        "created_at" : "Mon Dec 19 04:54:49 +0000 2022",
        "favorited" : false,
        "full_text" : "@ezyang Maybe look at SQLAlchemy for inspiration?",
        "lang" : "en",
        "in_reply_to_screen_name" : "nicksloan",
        "in_reply_to_user_id_str" : "16835872"
      }
    }
]

I think it would save us both some time if you could just share your repository, or a repository that reliably reproduces the problem you’re experiencing.

The repo is not public. I’ll come back to this and share a minimal reproduction repo, but I’ll have to do that later this evening. Thanks for taking a look.

OK, I see now what you’re trying to do.

In content/mentions/_content.gotmpl change this:

"kind" "taxonomy"

To this:

"kind" "term"
1 Like

Oh no.

I made that same mistake months ago with the templates, I think. Figured it out on my own that time.

This fixed it. Thank you so much for your help @jmooring!

Just one more question to put a bow on this:

Should I be processing the same dataset twice in separate content adapters, or is there a way to build the mentions term pages in the same _content.gotmpl as the tweets? Is there a way to nest the mentions term pages in the tweets section instead?

No, because …

When a content adapter creates a page, the page’s logical path will be relative to the content adapter.

https://gohugo.io/content-management/content-adapters/#overview

1 Like

Cool. I assumed that was the case, but glad to have it confirmed by a pro.

Thanks again for your help. Really appreciate it.

1 Like

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.