Work-around for making pages from data?

I just want to say that I have used the above technique with a complex CSV to split it in various markdown files and it works flawlessly.

I did encounter a couple of caveats that are unrelated to Hugo but I managed to fix them so I’m posting these fixes here in case someone else encounters them:

  1. When exporting a CSV from Excel all UTF-8 characters turn into question marks. There are involved hacks circulating the web that seem like overkill.

Solution: Ditch Excel and use Sublime Text’s Advanced CSV Package

  1. However when working with Advanced CSV it stripped quotation marks whenever I justified the columns.

Solution: Disable auto_quotes in settings of Advanced CSV.

Tip 1: To use a comma in a description cell of your CSV you need to escape it by quoting the entire cell.

Tip 2: You can use Hugo functions outside the getCSV range to fetch whatever resources you may need and have them rendered as front matter parameters.

Tip 3: The Hugo new command can create markdown files in the specified PATH (Big time saver)

I am very happy with the above. It’s pretty fast and efficient and it shows the power of Hugo with Data.

Again my big thanks to @bep for suggesting Environment Variables.

1 Like

What exactly do you mean by this (other than what you’ve written in the code example above)?

For example if you have an Archetype called orange.md

Then you can do:

hugo new "/orange/varieties/belladonna.md"

And belladonna.md will be created in the above PATH.

Ah, yea that’s what I thought you meant. Thank you!

Nice.

Of course, you could take this further with a simple BASH, PowerShell or Node.JS script (OK, or pretty much any other scripting language too!). It would be pretty easy to automate the whole thing.

Instead, you could (to redress the Excel bash), use Excel’s PowerQuery ETL features to generate the list of commands directly from the CSV, Excel file, JSON, a database, …

Excel is out of the question, it turns UTF-8 characters to ??? on CSV export.

One could import the `.xlsx to Libre Office’s Calc and then Export the CSV. (Isn’t that ironic a free and Open Source Software can handle UTF-8 whereas the proprietary and expensive counterpart cannot).

But as I said above I’m using Sublime Text’s Advanced CSV Package and I have automated the generation of the console commands with the RegReplace package.

The main reason that I prefer using this setup than what I was doing before with Bash (posted link above) is the ability to generate the content files on the fly with Hugo Archetypes.


I am using resources.Get and .Resize from within the Archetype to populate some old-fashioned image front matter parameters.

It seems to me that using a .Params.image in the templates, is faster than any .resources.Get, .Resize or Page Bundle.


Of course if the above statement regarding the speed is wrong, I;m sure that someone else more knowledgeable, will chime in to correct me. :wink:

Sorry, I wasn’t clear. The PowerQuery tool wouldn’t use a CSV export. It would create a shell script for you. Save directly to a file or copy/paste from Excel. Though to be fair, I’d need to double check extended characters but they should work OK. Needs a new’ish version of Excel though and I don’t know if the same facilities are available on Excel for Mac. Still, PowerQuery in Excel is designed specifically as an Extract, Transformation & Load (ETL) tool.

If you have some example data, I’d be happy to give it a go to check.

Mostly, I’d probably use either PowerShell or Node.JS to do this. Others might prefer Python or BASH :slight_smile:

That would still be used. I’m only suggesting that you could automate the whole thing based on the source data. Only worth it if you have LOTS of posts to create of course.

1 Like

Thanks for offering to help. :four_leaf_clover:

But I’m not using Windows Tools for Development.

Haha, well I’ll try not to feel too sorry for you :smile:

Anyway, just thought I’d chip in with some other ideas in case they spark some thoughts in others.

1 Like

Hey,
I wrote a simple tool that solves this problem (using a JS-wrapper): kidsil/hugo-data-to-pages .

Would definitely appreciate some feedback.

1 Like