Custom shortcode to display the date when a remote rss feed was last updated (e.g. Substack, Mastodon)

support
1

I’d like to display the date when writers have most recently updated Substack, Mastodon, etc. I’m using the following two guides as inspiration:

By experimenting with both, I’ve managed to get it this far (substack-updated.html):

{{ if .Get "url" }}
    {{ $url := .Get "url" }}
        {{ with resources.GetRemote $url | transform.Unmarshal }}
            {{ range first 1 .channel.item }}
                {{ dateFormat "2023-10-15" .pubDate }}
            {{ end }}
        {{ end }}
{{ end }}

and

{{< substack-updated url="https://fastwomen.substack.com/feed.rss" >}}

The error I’m getting is (I’ve put /path/to/ rather than displaying my actual paths):

Error: error building site: render: failed to render pages: render of "page" failed: "/path/to/single.html:22:5": execute of template failed: template: _default/single.html:22:5: executing "main" at <.Content>: error calling Content: "/path/to/page:80:1": failed to render shortcode "substack-updated": failed to process shortcode: "/path/to/shortcodes/substack-updated.html:4:21": execute of template failed: template: shortcodes/substack-updated.html:4:21: executing "shortcodes/substack-updated.html" at <first 1 .channel.item>: error calling first: both limit and seq must be provided

I’ve understand that I’m using range incorrectly, and have spent hours trying to figure out what I need to correct in order to make it work properly.

2

Code defensively; check for existence before doing something with it.

{{ with .Get "url" }}
  {{ with resources.GetRemote . | transform.Unmarshal }}
    {{ with .channel.item }}
      {{ range first 1 . }}
        {{ with .pubDate }}
          {{ dateFormat "2006-01-02" . }}
        {{ end }}
      {{ end }}
    {{ end }}
  {{ end }}
{{ end }}

Edit: I just tightened it up a bit.

3

Thank you! This is elegantly coded, and far easier to parse than what I was doing. I see what you mean about coding defensively.

I’m still getting an error. The relevant part of it is:

template: shortcodes/substack-updated.html:2:43: executing "shortcodes/substack-updated.html" at <transform.Unmarshal>: error calling Unmarshal: MIME "application/rss+xml" not supported

I think the issue must be with my config.toml. I’m trying to publish Atom and HTML (and not RSS) and want to be able to ingest RSS feeds from other websites (e.g. Substack, Mastodon) so I can use the shortcode above. The relevant parts of my config.toml:


[mediaTypes]
  [mediaTypes."application/atom+xml"]
    suffixes = ["xml"]
    
  [mediaTypes."application/rss+xml"]
    suffixes = ["rss"]

[outputFormats]
  [outputFormats.ATOM]
    mediaType = "application/atom+xml"
    baseName  = "feed"

[outputs]
    home = ["ATOM", "HTML"]
    page = ["HTML"]
    section = ["HTML"]
    taxonomy = ["HTML"]
    term = ["HTML"]

Apologies if it’s bad form to follow a question with a question. I thought others who wanted to use this shortcode might run into the same problem that I’m facing and would want to find out how to inplement the shortcode within the same response thread.

4 
{{ with .Get "url" }}
  {{ with resources.GetRemote . }}
    {{ with .Content | transform.Unmarshal }}
      {{ with .channel.item }}
        {{ range first 1 . }}
          {{ with .pubDate }}
            {{ dateFormat "2006-01-02" . }}
          {{ end }}
        {{ end }}
      {{ end }}
    {{ end }}
  {{ end }}
{{ end }}

https://gohugo.io/functions/resources/getremote/#remote-data

In these cases, pass the resource Content through the transform.Unmarshal function instead of passing the resource itself.

1 Like
5

Perhaps the issue is with the way Substack publishes RSS? I’m able to make the shortcode work when I test non-Substack RSS feeds. When I include Substack’s feeds, I get this error:

execute of template failed: template: shortcodes/substack-updated.html:3:12: executing "shortcodes/substack-updated.html" at <.Content>: error calling Content: error calling resources.GetRemote: failed to resolve media type for remote resource "https://fastwomen.substack.com/feed.rss"

When I review the URL, the feed looks like a proper RSS feed:
https://fastwomen.substack.com/feed.rss

It’s also validates:
Feed Validator Results: https://fastwomen.substack.com/feed.rss (albeit, with many recommendations)

Do I need to add security.http to my site configuration?

6

I can’t reproduce your problem. If you share your project repository we might be able to reproduce the problem.

The substack server includes this in its response headers:

Content-Type: application/xml; charset=utf-8
7

Thanks for volunteering to troubleshoot my hugo configuration. I hadn’t been using git and had not previously published my repository, so I had to figure it out how to use git in order to share my repository. I’ve created a publicly viewable version of the repository. Thanks for your patience and encouragement.

8

I was able to reproduce the problem. As noted above, substack serves the feeds with content-type “application/xml”. Add this to your site configuration:

[mediaTypes."application/xml"]
suffixes = ["rss"]

Some of the feeds you’re visiting (e.g., https://nitter.poast.org/tworepcave/rss) refuse the connection, causing the build to fail. Change your shortcode to emit a warning instead:

{{ with .Get "url" }}
  {{ with resources.GetRemote . }}
    {{ with .Err }}
      {{ warnf "%s" . }}
    {{ else }}
      {{ with .Content | transform.Unmarshal }}
        {{ with .channel.item }}
          {{ range first 1 . }}
            {{ with .pubDate }}
              {{ dateFormat "2006-01-02" . }}
            {{ end }}
          {{ end }}
        {{ end }}
      {{ end }}
    {{ end }}
  {{ end }}
{{ end }}

With so many resources.GetRemote calls, you may want to increase the build time-out in your site configuration:

timeout = '10m'
9

I don’t know how to begin to tell you how grateful I am. I’ve been making websites for almost 30 years, and I’ve used Hugo for 3.5 years. I haven’t had this much fun, or learned this much about Hugo, Git, or web development in a long, long time. Thank you!

2 Likes
10

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.