[Solved] GroupByParamDate produces "runtime error: index out of range"

support
1

Hugo version: v0.36.1 linux/amd64
Go version: v1.10

I have a custom date param that looks like this on my front matter in JSON format

{
   "date": "2018-02-20",
   "my_param": "2017-02-01"
}

This date param is used in many pages and I want to use it to group and order content. I tried using .GroupByParamDate like this

 {{ $grouped_pages := .Data.Pages.GroupByParamDate "my_param" "2006-01-02" }}

This produces the following error

ERROR 2018/02/21 17:39:17 Failed to render “theme/index.html”: runtime error: index out of range

I made sure the my_param parameter is set and readable. I can even use it with GroupByParam without problem.

Problem is the custom param is parsed as string in Hugo so I cannot use the “sort” function reliably. It actually seems to work following the Golang string comparison function but there is no guarantee this sort is going to work on all dates. Or am I missing something ?

What would be the best way to achieve what I need ?

Also posted as a github issue which was closed

2

You’ll need to first share your site source (at least a minimal version if you cannot share the original site), so that someone can clone it and reproduce the issue on their machine, and debug better.

3

I created a minimal website reproducing my error here

The line producing the error is here

4

Thanks for uploading that minimal example. I tried it, but I am confused too. Not sure why GroupByParamDate doesn’t work (not that I have used it before).

Hopefully someone else has experience using this function, and has a solution.

5

I don’t understand Go… but can sort of guess-read the code… from here, it looks like the string param is casted to time.Time first.

6

It looks like you can use GroupByParamDate function only for front-matter params of “date type” like date, lastmod, publishdate and expirydate.

From brief testing, I couldn’t make custom front-matter params in JSON format to be interpreted as time.Time. But in TOML front-matter, setting foo = 2018-02-21 (a valid RFC3339 format date without double quotes) make hugo auto-interpret that to be of type time.Time.

Looks like, here it checks if the type of the parameter is time.Time.

if _, ok := param.(time.Time); ok {
    r = append(r, e)
}

@bep Seems like a JSON limitation?

7

Guess what… it works if you use TOML as front-matter format instead of JSON, and make sure you do not double quote the custom date param. That way mydates will get auto parsed as a time.Time parameter, and not a string.

Here’s a little snippet to put in home/markup.html to understand the types:

{{ range .Data.Pages }}
     {{ printf "Page %s:: date: %s (%#T) mydate: %s (%#T)<br />" .Title .Params.date .Params.date .Params.mydates .Params.mydates | safeHTML }}
{{ end }}

Also you cannot set mydates as taxonomy else you will get errors like below.

ERROR 2018/02/21 16:24:26 Invalid mydates in section1/second.md
ERROR 2018/02/21 16:24:26 Invalid mydates in section1/first.md

Looks like taxonomy terms can only be strings (makes sense) and not time.Time.

So you have to remove mydate = "mydates" from config.toml.

With section1/first.md set as:

+++
title = "First"
date = 2018-02-21T21:12:49+01:00
mydates = 2018-02-01
draft = false
+++

This is content

and section1/second.md set as:

+++
title = "Second"
linktitle = "Second"
short_description = ""
date = 2018-02-21T21:15:28+01:00
aliases = 0
categories = []
mydates = 2017-01-01
draft = false
author = ""
+++

This is content

it works… hugo outputs:

                   | EN
+------------------+----+
  Pages            | 18
  Paginator pages  |  0
  Non-page files   |  0
  Static files     | 28
  Processed images |  0
  Aliases          |  8
  Sitemaps         |  1
  Cleaned          |  0

Total in 98 ms
8

Thanks for the update.

From brief testing, I couldn’t make custom front-matter params in JSON format to be interpreted as time.Time. But in TOML front-matter, setting foo = 2018-02-21 (a valid RFC3339 format date without double quotes) make hugo auto-interpret that to be of type time.Time.

Yes I found a kind of similar issue with date parsing where TOML seems to support this.

Guess what… it works if you use TOML as front-matter format instead of JSON, and make sure you do not double quote the custom date param. That way mydates will get auto parsed as a time.Time parameter, and not a string.

I need json as I have a wrapper script using jq to populate the front matter. If it really does not work with JSON it should be considered as a bug, and it works with TOML should not be the only answer.

Also you cannot set mydates as taxonomy else you will get errors like below.

If GroupByParamDate worked as specified in the documentation, this would not be an issue, as the taxonomy is still a string but GroupByParamDate formats the string into a date.

Thanks for the proposal but unfortunately I can’t use TOML and I need this as a taxonomy, so GroupByParamDate is my only solution.

If this can be reported as a bug I can try to propose a fix @bep

9

I made a fix which allows to use strings as dates in page parameters.

  • if the string can get parsed to time, it’s added to the grouped pages
  • if the string cannot be parsed to time, it’s skipped like any non time.Time parameter

Here’s the repo with the fix and the diff

Should I propose this as a PR ?

1 Like
10

After going through the tests, I understand better how GroupByParamDate is supposed to work however the docs are not all detailed enough to understand this behavior.

GroupByParamDate when applied to a date parameter (a parameter converted explicitly to date by hugo like the TOML example) will group the pages according to the format passed to it. ie if the format is “2006-01” it will group pages by year-month and if passed “2006” it will group by year.

In my situation with a custom date as string, the format is used to parse the date from the parameter, not for grouping, so it will group by the whole date as a key.

I think this could make it ever more confusing. The best would be to have a GroupByParamDateAsString that takes (“custom_param_date” “parse_format” “format” “order”). The other way would be to update the json front matter parsing side and add a way to parse some params as dates. I think the last one would be much simpler
as adding some tag on the json instead of adding a whole new set of functions.

any feedback ?

EDIT: Since the fix and @kaushalmodi solution solve the problem I mark the thread as solved.

11

I didn’t understand that. Can you explain that with an example so that we can better document it?

Also the patch, that you came up with, wouldn’t work? Is it failing existing tests?

I’m just curious, that’s all.

12

The patch is failing if I reuse the same test data. I will explain by showing how the GroupByParamDate works in hugo then explain how my fix works.

Originally in hugo

The test data in hugo is like this:

var pageGroupTestSources = []pageGroupTestObject{
	{"/section1/testpage1.md", 3, "2012-04-06", "foo"},
	{"/section1/testpage2.md", 3, "2012-01-01", "bar"},
	{"/section1/testpage3.md", 2, "2012-04-06", "foo"},
	{"/section2/testpage4.md", 1, "2012-03-02", "bar"},
	{"/section2/testpage5.md", 1, "2012-04-06", "baz"},
}
                                        ^
                                        |
                                  date field

In the test data, the custom date param is actually copied from the date parameter and casted as a date:

		p.params["custom_date"] = cast.ToTime(src.date)

The format passed to GroupByParamDate {{ .GroupByParamDate “custom_date” “format” "2006-01 }} is used to group and order the pages. In this case the format “yyyy-mm” will group the pages by year-month. Based on the test data it will give three groups: [2012-04, 2012-01, 2012-03]

If I pass the format “2006-01-02” it will group the pages by the full date, so if I have 5 pages with 5 different dates it will give 5 groups.

The custom_date param is casted to a date in GoLang so it must be parsed from TOML as date (since TOML supports time types)

With my fix

I discovered the problem by mistake when I tried to reuse the test data, I basically did the following:

		p.params["custom_date_string"] = src.date

I basically reuse the date field, but do not cast it into a date so it’s effectively a string representing a date. I duplicated the GroupByParamDates and applied them to the custom_date_param

My fix modifies the GroupByParamDate in the following way:

  • It tests if the custom_date param is a string
  • if it is a string, it tries to parse as a date using the passed format

So hugo uses the format to parse the date from the string, if he can parse it he will add the page to the candidate pages to group.

When hugo reaches the sorting step, he also uses the same format to group the pages. If the format is different than the custom string date it will not parse the string as date and the tests fail

To make the tests pass, I added an other string parameter with date strings inside looking exactly like the “format” used in GroupByParamDate

I made two branches one with tests passing and the other failing:

In the end I don’t know if there is a clean way to do it. It seems the GroupByParamDate is biased toward people using TOML. In my case I’m maybe going to migrate to TOML as I discovered rq which is equivalent to jq for toml

1 Like