Currently as you’ve seen Hugo does NOT support the automatic generation of pages from data.
This is the most anticipated feature at the moment and you may want to track:
opened 05:20PM - 14 Aug 18 UTC
Enhancement
Currently Hugo handles internal and external data source with getJSON/getCSV whi… ch is great for using data source in a template.
But Hugo cannot, using a data set of items, build a page for each of them plus related list pages like it does from the content directory files.
Here is a fresh start on specing this important step in the roadmap.
As a user, I can only see the configuration aspect of the task.
I don’t see many configuration tied issues except for the mapping of the key/values collected from the data source and the obvious external or internal endpoint of the data set. The following are suggestions regarding how those configurations could be managed by the users followed by a code block example.
### Endpoint/URL/Local file
Depending on use cases there may be a need of one or several url/path.
For many projects, not every page types (post, page etc…) may be built from the same source. Type could be defined from a data source key or as a source parameter.
I suppose there could be other parameters per sources.
### Front Matter mapping
User must be able to map the keys from the data source to Hugo’s commonly used Front Matter Variables (title, permalink, slug, taxonomies, etc…).
Every keys not referenced in the map configuration could be stored as is a user-define Front Matter available in the `.Params` object but this should not be default as there maybe way too many.
### Example:
This is a realtor agency, a branch of a bigger one.
Their pages are built with hugo's local markdown
They have an old wordpress site whose > 100 blog posts they did not want to convert to markdown. So they load those blog posts from a local data file on top of the local Hugo's own markdown posts.
They use a tier service to create job posts when they need to fill a new position. They want to hosts those job listing on their site though. Their jobs are served by `https://ourjobs.com/api/client/george-and-son/jobs.json`
The most important part of the website are their realty listings. They add their listing to their mother company's own website whose API in turn serves those at `https://api.mtl-realtors/listings/?branch=george-and-son&status=available`
#### Configuration
```yaml
title: George and Son (A MTL Realtors Agency)
dataSources:
- source: data/old_site_posts.json
contentPath: blog
mapping:
Title: post_title
Date: post_date
Type: post_type
Content: post_content
Params.location.city: post_meta.city
Params.location.country: post_meta.country
- source: https://ourjobs.com/api/client/george-and-son/jobs.json
contentPath: jobs
mapping:
Title: job_label
Content: job_description
- source: https://api.mtl-realtors/listings/?branch=george-and-son&status=available
contentPath: listings/:Type/
grabAllFrontMatter: true
mapping:
Type: amenity_kind
Title: name
Content: description
Params.neighbourhood: geo.neighbour
Params.city: geo.city
```
#### Content structure
This results in a content "shadow" structure. Hard lines dir/files are local, while dashed ones are remote.
```
content
├── _index.md
├── about.md
├── contact.md
├── blog
│ ├─── happy-halloween.md
│ ├─── merry-christmas.md
│ ├- - nice-summer
│ └- - hello-world
├- -listings
│ ├- - appartment
│ │ ├- - opportunity-studio
│ │ ├- - mile-end-condo
│ │ └- - downtown-tower-1
│ └- - house
│ └- - cottage-green
└- - jobs
├- - marketing-director
└- - accountant-internship
```
opened 12:22PM - 05 Sep 19 UTC
Proposal
There are existing issues about this, but I prefer to start fresh when I have ne… w ideas on a subject. I thought about this again when having my hair washed at the hairdresser today. Maybe the hair massage helped.
I think I have been too hung up in the technical challenges of this (remote adapters, how to effectively do partial updates etc.), making the whole issue too big to start with. What we have talked about earlier have also been "_something different on the side of what we already have_".
But what we have is:
* A virtual `/content` directory that can be composed via Hugo Modules (with overrides on file level)
* Virtual mounts support (you can mount any directory or file, even from remote GitHub repos into `/content`)
* A front matter based metadata model with cascade keyword etc.
* Partial server updates based on filesystem events.
* ...
With that I mind, I thought about adding a new reserved filename in `/content` starting with `_content`.
Given the example below:
* `_content.json`, `_content.toml` (and YAML) would be fairly straight forward, i.e. metadata + content. We should probably support a tree structure somehow, so you can build a complete content structure from one root `_content`file. We should probably support multiple files per directory so you could do `_content_products.json` etc.
* `_content.go` would represent the dynamic content, some kind of content adapter, possibly remote. This is obviously the area with most open questions (`_content.js` would be a thought), but having the naming in place is a start.
```bash
content
├── _content.json
├── _index.md
├── blog
│ ├── _content.toml
│ ├── _index.md
│ ├── image.png
│ └── post1.md
└── docs
├── _content.go
└── _index.md
```
/cc @regisphilibert @onedrawingperday @digitalcraftsman @budparr @moorereason @kaushalmodi and gang.
However there is a workaround mentioned here: JSON as data for websites - #4 by alexandros
The linked technique can be automated even further by generating a Custom Output Format to create the CLI commands for the pages’ generation and then execute it as part of the website build.
With the above said:
rwslippey:
Next question is, am I barking up the wrong tree with trying to use Hugo for a larger ecommerce site with many products, categories, manufacturers, etc. The site is for a distributor of large equipment so freight shipping in snipcart or whatever backend we use will also be an issue.
It all depends on the complexity of the ecommerce site. Someone else made a Hugo ecommerce site by using Stripe as the backend because they found Snipcart to be expensive.
One can easily create a Google Merchant XML Feed per product with Hugo and populate a Stripe backend or even a Facebook Product Catalogue or whatever…
I am in the process of doing all of the above myself. I still have a few blanks that I need to work out like connecting Stripe payments with a JS cart for the Hugo site… but we’ll see…
There is no tutorial out there about creating a Hugo ecommerce website step-by-step.
3 Likes