Enable putting data, assets, and layouts together

(I made an issue for this before knowing that discussion should start here, so copying it here now.)

Hugo should let users group content/, layouts/, and static/ files under content/.

It’s much simpler for beginners, when starting fresh, and when porting an existing static site incrementally.

E.g.

mysite/
  content/
    _baseof.html
    _list.html
    _single.html
    about.md
    me.jpg
    style.css
    blog/ # inherits _baseof.html from content/
      _list.html
      _single.html
      foo.md
      articles/ # inherits _baseof.html from content/, and _list.html from blog/
        _single.html
        bar.md

Then layouts/ can be for when you want to factor out patterns into common templates:

mysite/
  layouts/
    _baseof.html
    _list.html
    _single.html
    blog/
      _single.html
      _list.html
      articles/
        _single.html

I have thought about and sometimes wish for a way to group files in a more functional way. Not to make the life for new users easier, but rather to make the organisation of bigger sites easier to manage.

But having these files living side by side in /content with a “_” prefix is filled with ambiguous situations. We have examples already that show that that’s not a great idea.

What I have been toying with would be something like:

mysite
├── assets
│   └── _h
└── content
    ├── _h
    │   ├── assets
    │   │   └── logo.png
    │   └── layouts
    │       └── single.html
    └── _index.md

The above would, as one example, enable deeply nested image galleries on branch pages, but it would obviously also create some challenges, e.g. what’s the scope of the single.html template?

The current structure makes sense to me.
When you have a process of content → layouts → output then it doesn’t make sense to have layouts in content.
Some of your content might not be in a folder on disk, it could be remote data.

If the layouts get mixed with content, then if you change theme you’d have to go through your content to change all the layouts? How to backup template (or export theme) without having to try and filter out a few GB of images and markdown.

Logical to have all the templates together with partials, shortcodes and render hooks.

The lookups from content to layout can be a little confusing to start with but it isn’t because they are in separate folders. A well documented default starter with content and layouts might be a nice to have.
A .lookup or .usingTemplate method or some sort of way to help me debug why my content wasn’t using the layout I expected would have been useful for me at the start. The layout system is really powerful but that can make it a little tricky at first.
I don’t think adding another “template in content” lookup possibility makes it less complex. It might give people an easy way of getting started but IMO they would be getting started with bad practice.

The abstraction of content from layout is used in everything else I’ve tried
e.g. Gatsby - components / templates / pages or Grav - pages folder / system & theme templates

But having these files living side by side in /content with a “_” prefix is filled with ambiguous situations. We have examples already that show that that’s not a great idea.

@bep How would an underscore prefix cause a problem? Or a double underscore prefix? Can you point to the examples?

You could have underscore “escaping”, where leading single underscores are reserved for Hugo purposes (e.g. _single.html), and leading double underscores are replaced with single underscores in the output (e.g. __single.html becomes _single.html). That way people can still have files with leading underscores (for some reason; is this actually a concern?) in the output if they need to, without conflicting with Hugo reserved syntax/names.

The above would, as one example, enable deeply nested image galleries on branch pages, but it would obviously also create some challenges, e.g. what’s the scope of the single.html template?

You could have it scoped to the corresponding content/:

mysite/
  assets/
    logo.png # corresponds to /logo.png
  content/
    _index.md # corresponds to /index.html
    foo/
      _assets/
        thing.png # corresponds to /foo/thing.png
      _content/
        _index.md # corresponds to /foo/index.html
      _layouts/
        single.html # only applies within content/foo/

So the whole top-level directory structure could be nest-able, and would only apply to the corresponding nested content/. Similar idea to the OP, except I would also want the option to not have the separate structure, i.e. _single.html.

When you have a process of content → layouts → output then it doesn’t make sense to have layouts in content. Some of your content might not be in a folder on disk, it could be remote data.

@andrewd72 My point wasn’t that all sites should put everything together; rather, it was that it should be possible, with the ability to separate things out as we do currently as a more powerful option. This would especially be useful to people who are just getting started with Hugo, or porting a static site from another static site generator that doesn’t have Hugo’s current organization. It would have helped me. Would I have stuck with that long-term? No, but it would have been easier to get started with.

The current directory organization could be an abstraction of this more simpler organization. Once you understand the simpler model (“this thing only affects things in this directory and subdirectories”), it’s easier to learn the more complex model (“this thing only affects things in the matching directory and subdirectories”).

If the layouts get mixed with content, then if you change theme you’d have to go through your content to change all the layouts? How to backup template (or export theme) without having to try and filter out a few GB of images and markdown.

Right, this is a trade-off, and one reason why you would want to use the layouts directory abstraction for larger sites, or more complex sites, or sites that need to use themes. For simple sites that have a single directory of content and a simple presentation design, like say a documentation site for a code project, it seems over-engineered to have half a dozen directories with different kinds of files all over the place.

Your way you’d have the exact same number of directories but mixed kinds of files in one of them.

I have assets/scss and assets/js and assets/images. I could put them all in assets. Separating makes them easier to find and use.

If you understand how static and assets are different and how content should be portable and so separate from layout, then the four make a lot of sense. One for themes so you can override rather than have to overwrite and a public folder for the output. That is your half a dozen.
How can you take any one of those away?

1 Like

Your way you’d have the exact same number of directories but mixed kinds of files in one of them.

There would be one directory:

content/
  _baseof.html
  index.md
  a.md
  b.md
  c.md
  d.jpg
  e.jpg
  f.jpg
  scripts.js
  styles.css

If you understand how static and assets are different and how content should be portable and so separate from layout, then the four make a lot of sense. One for themes so you can override rather than have to overwrite and a public folder for the output. That is your half a dozen.
How can you take any one of those away?

Perhaps my ignorance is showing. Based on the docs, the only difference I see between assets/ and static/ is that resources.Get and friends work for the former and not the latter. I’m not clear on why everything in static/ can’t be in assets/.

In the case above, all jpg, js, and css files would be static by default.

Edit: Word. Newline.

The content of static is copied as it is into the public folder of the website.

1 Like

It is a bit like not putting cooked and raw food on the same shelf.

Static, used as they are, already cooked.
e.g.
.htaccess file, not referred to but my site uses it
fonts
google verification file
site manifests and favicons
php contact form script
(you can use put styles.css + morestyles.css and link to them but you can’t do stuff to them like concat or easily inline them)
logo image used as is*

Assets, raw ingredients, usually need cooking.
e.g.
scss files, processed to css, inline critical css and/or can concat multiple sheets
*logo image → processed to mylogo.png, mylogo@2x.png (nicer)
sites opengraph image → processed to resize to whatever size is currently recommend e.g. 1200x
images are a bit like cauliflower, you could eat them raw but they are better cooked to be responsive

Let’s say I use your SSG with one content folder, I have 10000 4K images and I put them all in one content folder, without the distinction of static and assets they would all need to be copied to my webserver because there would be no way of easily knowing which was which*
More bandwidth is used, upload is long and more server SSD space is used.

*without running some scripts to read the html to detect the links to them, then whitelist all the things that are required even though not linked. If only we could have them in separate folders…

Thanks for the explanations. Makes sense.

The non-md/html files in my example would be static. If you have 10GB of pictures that need to be resized, then yeah, that’s not going to work well, but plenty of sites have simple needs that require no file processing at all. Everything is preprocessed. Maybe a small logo, some arrow images, a flowery quotation mark image to put behind some text. That’s it. Mostly text.