Roadmap to Hugo v1.0

@voxadam thanks for your feedback.

Search

I agree, that not all users require a search for their website. But larger sites like docs (e.g. those of Hugo) or other content-heavy ones would benefit from this feature. This way users don’t need to run a task-runner like Gulp or Grunt.

Plugins

Creating a larger ecosystem with plugins would be a great opportunity to create extensions for common usecases. Users would just download the plugins that they need while Hugo should remain lightweight.

Migration

Having the migrator tools as external tools would make them independet from the Hugo release cycle, but it would also mean that we would have to maintain them as another project. My concern is that those tools we will pay just little attention over the time. Having them directly integrated would force us to keep them up to date. But it would also mean that we would have to exclude the Jekyll importer that is already implemented, if we want to the treat the migrators as external tools.

Related post

It might be right that is a nice enhancement and not a number one priority. Some people requested such functionality and we could implement in on the go while implement the other milestones. I just wanted to give it some place in the spotlight.

Documentation

Improving the documentation would be really awesome if you could contribute in this department. Especially beginners should find a better way to get started with Hugo.

@rdwatters mentioned already some great ideas in this direction:

Include both templating and output HTML in all samples. This is great for beginners

I agree, that not all users require a search for their website. But larger sites like docs (e.g. those of Hugo) or other content-heavy ones would benefit from this feature. This way users don’t need to run a task-runner like Gulp or Grunt.

@digitalcraftsman, I absolutely agree that search needs to be on the docket as a planned feature, it’s a really important feature that people already want. I’m only questioning its being a requirement for v1.0. Is there any reason that a proper integrated search solution couldn’t be targeted for say, v1.3 (I totally pulled that number out of thin air)? Integrated search is an important and rather complex feature. I want to ensure that a search less than fully baked search functionality isn’t shoehorned in just to get v1.0 out the door. As a corollary, I’m not entirely convinced that v1.0 should be held back for search.

I’m okay with being wrong on this one; it’s just a thought.

@voxadam v1.0 is a big number and a lot of work has to be done until we reach it. In my opinion, the integration of the plugin system is the most time consuming and hardest milestone because it effects so much parts of the codebase.

Integrating the search seems to be relatively a relatively smaller task that could be done in that period of time, I think. But it should block the final release of v1.0


After starting this discussion it would be interesting what the members of the ‘core-team’ of Hugo think. Where do you see Hugo in the future? What are your prior milestones that are necessary to call a release v1.0?

/cc @spf13, @bep, @anthonyfok and others (it’s hard to define ‘core-team’)

My 50 cents is to be a little bit pragmatic about this, at least with the current “work force”.

Me myself, I only implement according to my own road map, that is: What I currently need or find interesting/challenging.

There are many nice to haves, but the only two things (that I can think of) that I really need now is:

  1. A proper way of handling the Nodes (front matter etc.). This is a problem I have thought long enough about and should be ready to implement. Not too hard, me think.
  2. A related posts feature. I kind of agree that it’s search related … but it really isn’t the same-same, in my head. This is a harder problem to solve in a good way. I have started on a prototype a couple of times …

As to search, I think Hugo should support that (as in: provide a way to extract the data in a index-friendly format), but it should live outside Hugo.

1 Like

I’d like to clarify my previous comments and say I’m more interested in the ability to just write to JSON. I see Hugo as a single tool for managing structured content, and I see JSON as content/data, not logic or presentation.

I’m not a go programmer, but it seems like a simple enough feature and extends the functionality of Hugo considerably, but not for just search. It also adds opportunities for an SPA CMS layer (see my comment above), more async options for improved UX, etc. I would agree that a full-tilt search solution might protract a V1.0 release unnecessarily.

1 Like

I love this thread. It’s great to see the passion that everyone has for this project.

I think about 1.0 a bit differently. For me 1.0 is the milestone when we have a stable foundation. This means that we are happy with our data model and interface. I think far too often people release 1.0 long before things are stable. It took us thousands of users to recognize that our node implementation isn’t as flexible as we need it to be.

In my eyes, as soon as we have this stable foundation we can release 1.0 and then a lot of awesome things can be layered on top of it with point releases.

As I’m thinking about it today there are four things that we need to improve to get to a stable foundation.

1. Nodes.

Initially as I designed Hugo I thought that a lot of the way other CMS / SSGs designed their content data model was backwards or broken. Wordpress has everything as a post… then non post things are oddly hacked in. Jekyll is similar but has posts and then dedicated static pages. These make sense given their blog specific roots, but are not a solid foundation for a full content CMS. Drupal has dynamic (almost object oriented) approach to their content model where the base is a Node and Pages and other content extend the base. I took inspiration from this approach and used similar terminology. In Hugo content would be created by the author and nodes would be created by Hugo. I also made the decision that content should live where it wants to live… meaning /content/post/foo.md becomes /post/foo.html. This proved to be a very friendly and intuitive interface for creating content. It was the right interface because it catered to the creator of the content, not the generator.

Now we have learned that this foundation has a few holes in it. People want to be able to have some content on their site that’s hidden from lists. People want to be able to not only define the layout of their taxonomy and other node pages, but provide content for them as well. In essence a lot of the functionality that was in the content layer needs to move down into the node layer. The technical challenges aren’t too hard, but the interface design is tougher. I’m still not sure how to make it so that a given file translates into a node and not content. I’ve thought of a few things here and would love to brainstorm with others on it so we come up with the right solution.

2. Localized content & assets

If I was going to design Hugo today I would do it a bit differently. I would have a post be represented by a directory and a set of files in that directory. Front matter is a novel idea, but it really stems from the concept that a single file represents the entirety of content. In practice a post is often made up of metadata, content (which sometimes spans multiple pages), and media. While the current approach of separating /static and /content makes a lot of sense in a general way… for things like sitewide assets like CSS & JS, it becomes a bit laborous to manage post assets independently of the content.

It would be really nice to be able to have something like

/content/post/foo/
   ->       content.md
   ->       metadata.yml
   ->       headerimage.png
   ->       picture.png

This is a very flexible approach that opens a lot of possibilities (multiple pages of content, multiple languages, etc).
The challenge is how do we unify this approach with the current approach of a file representing the entirety of the post.

3. Better asset handling

I’m not sure what this looks like. Fundamentally it’s a framework which allows for things like image processing, css preprocessing, etc.
I don’t want to rebuild gulp or similar systems, but I’d like to be able to allow for these kind of operations to happen within the context of Hugo.

4. Partial building.

This has been a long asked for feature. It’s hard to do and to date we’ve focused on making Hugo fast enough that it’s often not an issue. It becomes more of an issue when Hugo starts leaning on 3rd party applications to do things more as these tend to be significantly slower than Hugo itself.

I’ve given this a lot of thought and I believe I have a good model for how this can work and in a pretty simple and straightforward way. I believe that the approach that others have taken of trying to discover what needs to be rendered by examining a rendered site is fundamentally broken and at our current speed would actually be slower than Hugo is today. I think by leveraging the watch system we can implement a partial render in a very efficient way by tracking each file that changes and marking all associated content as dirty. Then in the render stage we can limit the processing to only dirty content.

For strictly static content this complexity isn’t needed and there’s already a very straightforward PR on improving the static file copy to only examine the changed file when watching. https://github.com/spf13/hugo/pull/1671

I believe this is it from me.

Reply to other suggestions

To respond to other suggestions in this thread.

native i18n and l10n support is very important. I don’t understand it well enough to suggest I can really qualify what needs to happen here. Luckily Hugo has a lot of bi-lingual developers who this affects significantly. I believe that my #2 point could be a part of this.

Could happen before or after 1.0:

  • easier deployment
  • related posts
  • easier migration
  • write JSON
  • Go implementation of AsciiDoc, ReST, etc
  • template functions
  • revamped docs / website

I hope they all happen ASAP, but we are a community of volunteers each having their own motivation and focus so they will happen when someone feels motivated to do it. I don’t see any of these being predicated on 1.0 or blocking the 1.0 release as none would have breaking changes to the interface or foundation.

8 Likes

Partial rebuild is an important addition to the list.

There is one thing I now find myself thinking about. How would partial rebuild interact with a related pages/posts feature? It may be obvious but I’m running on a suboptimal amount of sleep at the moment.


I really like the idea of a directory per post layout. It feels cleaner/neater to me.

Steve,
I love pretty much everything you’ve said and agree whole heartly, esp. on #1 & #2 (and I’m curious how #2 would affect #1). One thing you didn’t mention that I would add to your 1.0 list of foundation items:

Plugins

Plugins in Go are an intriguing problem. It’s probably beyond my skill set, but I’ve been contemplating how that would work. What types of plugins do we allow? Where do they plug into the build chain? What types do we pass them and what do they return?

In looking into all of this and trying to familiarize myself with the whole build process, I get the sense that APIs defined within Hugo may need to be altered once plugins enter the scene. I think that’s part of laying a solid foundation for the future.

One other side benefit I hope to see from opening Hugo up to plugins is that we become much better about adding godoc comments to all of our exported types and functions.

Postscript

@digitalcraftsman, let’s get one this straight: any definition of the core team should not include me. :smiley:

1 Like

If it should live outsite of Hugo it sounds like an interface that could be used by a plugin. Or did I misunderstood something?

What functionality should exactly be supported by a SPA CMS that’s designed for Hugo? Goes it beyound basic CRUD operations with content files?

In my opinion plugins are a nice way of implemented custom functionality without bloating the codebase of Hugo.

As you mentioned, we need a define an interface for plugins to allow them to interact with Hugo. But implementig such an interface should be carefully planned since it effects a lot of parts of the codebase. To quote Steve:

Therefore we would have to ask ourself this question before releasing v1.0. On Github there was already a little discussion of possible approaches but it didn’t got much attention.

Yes, this is still something in toying with. I’m taking a mobile JS app course this semester. As part of my application, I needed to put out a final project idea and an SPA CMS on top of Hugo was one of them. I’d want for users to be able to create complex relationships between pieces of content (or content types) from within the CMS.

That sounds very Interesting. It’s somehow like rango bit with a more advanced feature-set.

Currently, it’s only possible to work with the file’s content and metadata itself to build functions for a SPA CMS. Without an API (which is still just an idea) you’ve to write many things yourself by creating connections between the content and metadata (e.g. posts in the same category).

Let’s say you would write an SPA CMS as mobile app. Where exactly resides Hugo in your stack?

The ability to auto-publish future posts on the given date and time without using complicated third party tools or scripts would be awesome.

Scheduling posts in a simple way is essential for most people including me.

Also an import script from Tumblr to Hugo would be really cool.

You could achieve this with the publishdate variable in the frontmatter. According to the docs:

publishdate If in the future, content will not be rendered unless hugo is called with --buildFuture

Combining this with deployment tools like Wercker would give you exactly what you want. The remaining question is, how do you define complicated third-party tools? Would Wercker be one of them?


Also an import script from Tumblr to Hugo would be really cool.

I agree that it would be awesome to offer more support for migration tools. They’re already on the wishlist. But until know didn’t find a third-party tool/script on Github.

Preferebly those people, who migrate(d) from other platforms should write this migration function. Most of them are more familiar with the structure of those platforms. A good example would be the migration command for Jekyll that was added with the latest release.

I’ve updated the summary in the post according to the latest posts.

After defining what bullet points should be considered as a milestone and how they should look like (roughly) we should split this thread into seperate ones. Those should be specific for each milestone. In the near future it could be hard to follow a single discussion.

I am not particularly confident about using Wercker or Caddy Server for that matter.

I would prefer a solution like having Hugo running locally as a process that can --watch a directory and check post dates so that it can auto-publish on the right time. Spf13 mentioned something about partial/incremental building earlier.

As for Tumblr there is an API for retrieving stuff from their server https://www.tumblr.com/docs/en/api/v2
(I am unable to write a script since I am not a programmer)

With auto-publishing, do you mean the deployment of the generated content to a server?

Partial or imcemental builds only rerenders pages, whose content change, and those that are associated with them . At the moment, Hugo would generate the whole site even if you changed only a single line in a post. For optimization purposes we would only need to regenerate the page of the post and associated pages like the homepage that shows the latest posts. But you would still need a trigger for generating the site.

Incremental builds is an incredible hard problem to tackle – it will happen, eventually, but …

“Incremental publishing” on the other hand is easier, even when the timestamps change on every build. Just keep track of the MD5 hashes (or similar) of all the generated files.

@digitalcraftsman
Yes that is what I mean by partial or incremental build. But @bep has grasped what I am talking about. Even though I’ve no clue whatsoever about tracking MD5 hashes. LOL!

Thanks for your answers.