Data templates with other sources than JSON feed

Hello there,

I suggested the proposal to add other formats than JSON to data templates, such as RSS or Atom, to be able to do feed aggregation directly into Hugo :

I had already posted about that, with a different approach : JSON feed from Wordpress + Webhooks/Websub = Display aggregated posts automatically?
This idea has been suggested in the past : External RSS feed data - leave handling to hugo?
This guy has suggested some solutions : Feed aggregation using Hugo

But what I would like is for Hugo to directly parse RSS or Atom feed like it does with JSON feed, to be able to use only Hugo to do feed aggregation.

What do you think of this proposal ?

Yann

what works:

you can try to find an external tool to convert rss (XML) to JSON

anything like this https://jsonformatter.org/xml-to-json or what works offline.

Remark: to actualize the content from RSS, you must rebuild the complete site!

Hello,

Thank you for your answer.

However, I can’t find a way to convert automatically my RSS feed to JSON.

The website you suggested is only for one-time process, whereas I would like that every time a post appear in my RSS feed, it gets automatically converted to a blog post in my hugo blog.

Yann

It is easier to create a JSON feed instead of converting

Check my sample repo

1st define output formats in config.toml
2nd create the templates

I have there feeds with RSS, ATOM and JSON

Thank you for your answer.

However, what I’m trying to achieve is not to import the feed of a Hugo blog inside an other Hugo blog.

What I want is
to take the RSS feed, for example of my Shaarli bookmarks, or of a Wordpress website (where I do not have the authority to install a Json plugin),
and display these posts in my Hugo blog.

So what I suppose I need is to find a way to automatically parse a feed from RSS to JSON, and then update the content with data templates.
I know there are a lot of tools to parse XML or RSS feeds (for example in Go : https://github.com/ungerik/go-rss or https://github.com/mmcdole/gofeed , or in Node : https://github.com/rbren/rss-parser or https://github.com/danmactough/node-feedparser )
but I can’t figure how to take the results of these parsers and transform it (automatically) into something that my Hugo blog will understand.

Since you want the content from a RSS feed to automatically be added to Hugo and Hugo generates static sites, you’ll need some kind of automatic that’s outside of Hugo, such as a crontab entry to re-run hugo.

Given that requirement, rather than modify hugo to have the ability to parse an RSS feed directly, it would be simpler and more straight forward to simply schedule an repeating task that would parse the RSS feed into JSON or another data format, and place the output where hugo will pick it up on the next run.

So for example:

Scheduled every hour/day/etc:

  • a script or program that converts RSS to JSON, using as input source your bookmarks, and outputting to your hugo data/ directory.

Scheduled for 5min after every hour/day/etc:

  • a call to run hugo

Yes, exactly !
I’ve thought of either doing a cron job or with a webhook,
But my problem is precisely that :

I can’t figure how to “place the output” of the parsed result in a manner that “Hugo can pick it up on the next run”.

Could you help me with this bit ?

You were originally asking about Data Templates, so if you wanted to go that way still then simply direct the output to your hugosite/data/ folder and access it via a hugo template.

OK, but how can I direct the output of, say, this tool :


or this one :

to the data folder ?
That’s what I try to figure out.

Thanks a lot for your help anyway !

In the unix world, if any program of script outputs to the screen, you can redirect that output to a file. The following save a directory listing to a txt file instead of printing to the console:

ls -la > listing.txt

But the specific method would depend on the tool you choose to use, and what options it includes.

Since it’s outside Hugo, it’s less likely that someone here will know how to use it any more than you do. Once you find a tool you want to use, then you’d need to check their documentation, and the best place for help would be on their forums etc.

If I was looking for a small standalone utility to download rss and output some json on a unix type system, I’d look at python or perl first, since both are common utility scripting languages. You’d need to decide what is best for your situation.

Okay, many thanks for your answer. It’s true I dismissed a bit too quickly Perl and Python, as I want to use docker, I went directly for Node or Go.
I will try with a little script then.

If I can’t make it work, I will see if I can use one of the few “RSS to Markdown” converter i found.

This topic was automatically closed 24 hours after the last reply. New replies are no longer allowed.