Future posts in variable within a regular build

feature
1

Improvement for new global variable future posts

Hugo already recognizes the future content. During a regular build, it reads the files, parses front-matter, but currently just discards it. What if instead of discarding, Hugo could build a separate list of .Pages.Future or something along that? Maybe do the same with drafts?

I am seeking for a way to do exactly timed scheduled builds based on next expiry/future post dates.

Theme developers/site admins can use that variable to their liking, it can provide more possibilities than just scheduled build.

1 Like
2

Let me explain myself ahead of time. I’ve been thinking of this problem for some time and here’s how far I’ve come.

How to schedule posting with publish/end date aka automated publishing.

Consider all this for already a little established sites with continuous deployment, CDN, large theme, many images, CSS/JS/HTML processing etc. etc.

Solution #1 - a regular rebuild task

The most common suggested solution is a scheduled task that triggers build and done by

  • build environment itself (i.e. Travis-ci, BitBucket Pipelines etc.)
  • external build trigger (i.e. Netlify POST build trigger)

The site is rebuilt enough often so that posts appear/are hidden when required.

Summary for this approach

Pros:

  • Hugo is amazingly fast to build on even large sites
  • Easy to implement on almost any CI/git/hosting scenario
  • Running often will ensure always up-to-date content

Cons:

  • Build is long on large sites considering other tasks like SASS/LESS, image minify, critical CSS, CSS/JS minify any other pre/post-processing module you might have
  • Each build purges CDN (i.e. forcing AWS CloudFront purge too often adds costs)
  • The first reload on CDN region after purge is slow
  • Build notifications not distinguishable from rebuilds vs real changes
  • What if rebuild and content commit triggers to come in at the same moment/in parallel?

Doing this at short intervals is not a sane approach to increase timing precision - daily, hourly, every 10 minutes? Again, consider increased CPU/billing costs (even using “free” Netlify - let’s be nice to them). What if everyone starts to do this (from forums/blogs I see a lot of doing like it)?

Final score: doable, but costly

Solution #2 (not possible) - trigger build at the exact time

Only thing, we need to know WHEN.

When can be defined as:
closest datetime from a merged array of - all published posts expiry dates + all future posts publish dates.

If this WHEN is written in a public location, then remote trigger could GET it, compare times and if required, do POST trigger for a build.

Getting soonest expiry date is kinda easy by doing range on all posts, some ifs, something like that. Now comes the real problem - getting to loop over future posts publish dates. During a normal build, Hugo excludes the future posts.

I have considered the following workarounds:

  1. One could enable --buildFuture, but then future posts should be hidden, so need to edit the whole theme to exclude them again from every location including sitemaps, tags etc. What if the theme is an external dependency?

  2. On CD/CI do a separate build step with --buildFuture on different destination directory for the sole purpose of getting the future posts publishing dates, output in file. Then copy this info to /public/

  3. On CD/CI perform command line hugo list future which outputs paths to .md files, then read/regex content for expirydate. Consider different front-matter syntaxes (yaml/toml/json). Again output results in /public/ location.

All of those are tedious, kind of fragile and hard hack-arounds. But if working, then external task could read those public dates (depending on output success/format) and do its logic.

Summary:

Pros:

  • Precision possible up to few minutes even with a static generated site
  • The low cost to execute
  • Can bring the solution on any site/theme without rewriting it

Cons:

  • Requires little of code setup for both site build and remote checker

Final score: almost there, if just had future on my hands

1 Like
3

Solution #3

Modify hugo list future to include date/time and make it machine reader friendly

Pros:

  • Very simple requirements (fits on one line)

Cons:

  • None
2 Likes
4

I like the idea.

Maybe would be possible also to add hugo list expiry which would return posts that have expiry date set and not reached yet?

Then the whole solution would be simply processing those both outputs.

2 Likes
5

Hello @bep,

I’m thinking over Solution #3. Although I’ve never written a Go code, I checked source code over the GH, understood what’s going on with list function and what comes together for it to work.

And… it’s really tempting to try do a PR for this to help out not just “request a feature” here.

So, here’s few questions raised while reading code where I would like to have input:

  1. Output format:
    1.1 CSV seems most sane as output is written during loop
    1.2 Thus pattern could be <date>, <path> giving predictable pattern for parsing
    1.3 Should <date> be Unix timestamp or formatted year-month-date time Z?
    1.4 Should anything else be added from post meta?
  2. How to implement
    2.1 Whether output is changed right away or some backwards compatibility should be provided?
    2.2 If so, how to implement/handle that? (more complicated)
    2.3 I think that all hugo list sub-commands should be affected by format change to have same pattern
    2.4 If above true, then for future posts date is publishDate, expired = expiryDate, what date drafts output should have?
  3. New command hugo list dueexpiry
    3.1 Having similar commands “expired” and “expiry” would be confusing and can cause errors due typos (is there better name?)
    3.2 Built by similar pattern to rest of list
    3.3 On range it outputs posts that have publishDate < now < expiryDate

P.S. For sake of feature completness, should hugo list all be added as well?