How to make a recursive content adapter

I am opening a new topic here to discuss and learn how to create a recursive go template using a JSON file- with a tree like nested structure.

This topic stems from a support post I made previously regarding help with content adapters on a nested JSON with defined levels. Which can be found here: https://discourse.gohugo.io/t/using-content-adapters-to-add-pages-nested-structure/52813/9

This original post has made me curious about recursively iterating over a JSON and its levels instead of manually doing so. The potential here would be to offload manual creation of pages/structures and instead allow for bulk uploads per a formatted JSON. Another potential use case would be in situations where levels of nesting differ for different child nodes- could be more flexible than a rigid hierarchy.

Below is some sample data I was considering.

{
  "topics": [
    {
      "id": "1",
      "title": "Root Topic 1",
      "description": "This is the first root topic.",
      "metadata": {
        "created_by": "User1",
        "created_at": "2024-12-20"
      },
      "children": [
        {
          "id": "1.1",
          "title": "Child Topic 1.1",
          "description": "This is the first child of Root Topic 1.",
          "metadata": {
            "created_by": "User2",
            "created_at": "2024-12-21"
          },
          "children": [
            {
              "id": "1.1.1",
              "title": "Sub-child Topic 1.1.1",
              "description": "This is the first sub-child of Child Topic 1.1.",
              "metadata": {
                "created_by": "User3",
                "created_at": "2024-12-22"
              },
              "children": []
            }
          ]
        },
        {
          "id": "1.2",
          "title": "Child Topic 1.2",
          "description": "This is the second child of Root Topic 1.",
          "metadata": {
            "created_by": "User4",
            "created_at": "2024-12-23"
          },
          "children": []
        }
      ]
    },
    {
      "id": "2",
      "title": "Root Topic 2",
      "description": "This is the second root topic.",
      "metadata": {
        "created_by": "User5",
        "created_at": "2024-12-24"
      },
      "children": []
    }
  ]
}

My limited understanding makes me think we would expect something like this: where we define how to add pages, then call it through each level

{{/* Recursive function to create pages from nested data */}}
{{ define "createPages" }}
  {{- $data := site.Data.nested_recursive_data -}}
  {{- $basePath := .basePath -}}
  
  {{ range $data }}
    {{ $title := .title }}
    {{ $content := dict
      "mediaType" "text/markdown"
      "value" .description
    }}
    {{ $params := dict }}
    {{ $path := (path.Join $basePath $title) }}
    {{ $page := dict
      "content" $content
      "kind" "section"
      "linkTitle" $title
      "path" $path
      "params" $params
      "title" $title
      "type" "docs"
    }}
    {{ $.AddPage $page }}

    {{/* Check if children exist and recursively process them */}}
    {{ if .children }}
      {{ $childContext := dict "data" .children "basePath" $path }}
      {{ template "createPages" $childContext }}
    {{ end }}
  {{ end }}
{{ end }}

{{/* Initialize and start the recursive process */}}
{{ template "createPages" (dict "data" site.Data.nested_recursive_data.children"basePath" "") }}

However I get confused when it comes to things like the final line- as I understand it - this: site.Data.nested_recursive_data.children is only storing the second level containing children- not continuing deeper.

Any help, thoughts, explantions would be super helpful here! Thanks a lot!

I would say it stops at the first level, because you are not using .data from the supplied dictionary, instead using site.Data.nested_recursive_data each time. The first line of the template (not tested) should probably be {{ $data := .data }}.

You do that with the base path, why not with data?

I find it a bit easier to break the content adapter into three pieces:

  1. The initial call to a template that walks the data
  2. A template to walk the data
  3. A template to create the page

In the template calls you need to pass:

  • The data
  • The path of the section to be created
  • The root context of the content adapter (you need this to use the AddPage method)
content/topics/_content.gotmpl
{{ with site.Data.nested_recursive_data }}
  {{ template "walk" (dict "data" .topics "path" "" "ctx" $) }}
{{ end }}

{{ define "walk" }}
  {{ range .data }}
    {{ $path := path.Join $.path .title }}
    {{ template "add-page" (dict "data" . "path" $path "ctx" $.ctx) }}
    {{ with .children }}
      {{ template "walk" (dict "data" . "path" $path "ctx" $.ctx) }}
    {{ end }}
  {{ end }}
{{ end }}

{{ define "add-page" }}
  {{ with .data }}
    {{ $content := dict "mediaType" "text/markdown" "value" .description }}
    {{ $dates := dict "date" (time.AsTime .metadata.created_at) }}
    {{ $params := dict
      "authors" (slice .metadata.created_by)
      "id" .id
      "description" .description
    }}
    {{ $page := dict
      "content" $content
      "dates" $dates
      "kind" "section"
      "params" $params
      "path" $.path
      "title" .title
    }}
    {{ $.ctx.AddPage $page }}
  {{ end }}
{{ end }}

This makes every page in the tree a section, including the leaves (i.e., the page at the end of a branch). You could easily modify this to set the page kind to “page” for the leaves, but then you’d need two templates (e.g., layouts/topics/list.html and layouts/topics/single.html). So… I’d leave it as is.

3 Likes

Thank you for the breakdown here- really helpful for my understanding!

Would we then expect the “Sub-child Topic 1.1.1” to be a section as well or the other leaves? Or do these need to be the kind “page” to be visible?

Or would I need to follow something use the layouts/topics/list.html with this logic below :

  1. The list page for the products section, by default, includes product-1 and product-2, but not their descendant pages. To include descendant pages, use the RegularPagesRecursive method instead of the Pages method in the list template. ([ref])(Sections | Hugo)

I’m not sure that I understand your question. The example I provided works fine with your data. But the dates of some of your nodes are in the future, so when you build your site you need to add the -F flag.

Ah I see, I fixed the dates!

Hi @jmooring - apologies for a slightly delayed question.

I was considering your mention of setting kind page for leaves potentially. My expected behavior would to see that child page’s children in the form of a page link- given the content of that page, so here when I click on Child Topic 1.1, I would see only Sub-child Topic 1.1.1:

Page generation is correct structurally, - so I was considering a shortcode or partial that filters down to the desired page using that as the content for displaying the link. I have been playing around with hide summary conditionals depending on if it is a parent/child page.

Right now it looks like we sort of see everything listed out rather than the specific pages based on the relationships. It seems like a small matter of link visibility but I want to make sure I get this down before adding visualizations at each level or modify how links are displayed. Some guidance here would be amazing!! :grinning:

The rendered list is determined by the page collection over which you are ranging. And from the above I have no idea what’s in the template.

I recommend sharing your repository; it looks like a test rig anyway.

Sure- the goal would be to use the docsy template.

The link is below:
https://github.com/ethanjose1/ContentAdapter_HugoSite

The logic seems to be working after I made some changes (added list and single html files):

When I click School of Mathematics I will enter the departments (ignoring css and missing partials references in the list.html):

Does this logic seem to be the way to go? If so the idea would be to do some conditional work in the go template to added content/plots/shortcode depending on the level within the hierarchy, but want to ensure this would be the approach.

I haven’t looked at your repo, but your comment above doesn’t make sense to me.

It seems to me, if you need to add something based on the level in the hierarchy, you should do this with a partial from /layouts/foo/list.html or whatever.

Shortcodes should be used to add something based on content, not structure.

Forgive my confusion here, I should have segmented my questions more clearly.

The initial issue was that when I clicked on a deeper level within the hierarchy- it would show all page links- not the specific page links for children of that page. In my above reply I seem to have fixed this.

{{ with site.Data.topics }}
  {{ template "walk" (dict "data" .roots "path" "" "ctx" $) }}
{{ end }}

{{ define "walk" }}
  {{ range .data }}
    {{ $path := path.Join $.path .title }}
    {{ template "add-page" (dict "data" . "path" $path "ctx" $.ctx) }}
    {{ with .children }}
      {{ template "walk" (dict "data" . "path" $path "ctx" $.ctx) }}
    {{ end }}
  {{ end }}
{{ end }}

{{ define "add-page" }}
  {{ with .data }}
  
    {{ $params := dict "title" .title "content" .content }}
    {{warnf "content: %s" .content}}
        {{ $content := dict "mediaType" "text/markdown" "value" .content }}

    {{$kind := ""}}
    {{$type := ""}}
    {{ if  .children}}
        {{ $kind = "section" }}
        {{ $type = "topics"}}
    {{ else }}
        {{ $kind = "page" }}
        {{ $type = "docs"}}
    {{ end }}

    
    {{ $page := dict
      "content" $content
      "kind" $kind
      "params" $params
      "path" $.path
      "title" .title
      "type" $type
    }}

    {{ $.ctx.AddPage $page }}
  {{ end }}
{{ end }}

I added a conditional here: that sets the type to “topics” (as you mentioned layouts/foo/list.html I have layouts/topics/list.html) so we are now getting our view according to that showing specific child pages for each parent).

So is this resolved?

I think so :grinning:, thanks for the help!