Whenever I run Hugo it seems to update all my posts, even the ones that haven’t changed. This in turn causes my SFTP based file synchronization to upload all my posts, rather than just the changed ones.
It there a way to get Hugo to not update outputs if the markdown file is older than the existing output (or maybe do a comparison before updating the output files)?
EDIT: I have a very hacky solution. I have a script that runs hugo, then goes through all files and computes a hash, then copies them to an “upload” directory only if the hash doesn’t match what’s already in there, then uploads the upload directory (with the timestamp based syncing enabled). That means that the timestamps in my upload directory only changes if the content has changed. The problem now is that Hugo seems to generate different files when run multiple times if I delete the “public” directory before running it. That means I can’t delete the public folder before running hugo, which means I run the risk of retaining stale files.
Well, either by just regenerating the file and then checking if it ended up the same before actually overwriting the existing file in the output directory (note: for this to work hugo has to be 100% deterministic in its outputs… that doesn’t appear to be the case right now)?
Or you could hash the “current” template (one hash for the whole template folder) and comparing it to the one that was used to build last time? Basically, hash all the inputs to the build process, cache what inputs were used last time, and then when building you check if either the hashes don’t match, or if the file isn’t in the public dir (someone could’ve deleted it). I guess you’d really like to cache the hash of the outputs too so you can detect if someone went in and modified the file in the output dir (if so, regenerate it from sources).
EDIT: I found out what part of Hugo seems to be non-deterministic (so that files get generated slightly differently when I run hugo multiple times). It has to do with the “social-buttons” in the red lounge theme. These are specified in the config.toml file, and for some reason they get output in a different order when you re-run hugo. It’s kind of freaky to me that the buttons move around in the HTML file! I guess the TOML parsing doesn’t guarantee the order for arrays of tables (specified by just entering several [[menu.social]] tables)?
Hugo is perfectly deterministic if the templates are. Of course, adding a .Now.Unix to the header partial kills this, or a list with a non-deterministic order.
Ah, thanks! That makes sense. TBH though I would expect it to be deterministic - I mean even if it’s arbitrary it should at least give the same result each time you run it IMO.
If I’m not mistaken, the menu entries are internally built up in Hugo as Go maps (see here). Per the Go spec, “the iteration order over maps is not specified and is not guaranteed to be the same from one iteration to the next.” Since your menu items have no defined weight, I’m assuming that ranging over the menu items will use the default sorting order, which ends up being the Go map iteration order. I could be totally off-base here, so maybe someone will correct me if I’m wrong!
Having said all of that, however, I agree with you in the sense that Hugo could itself track and preserve the insertion/iteration order. Since this thread veered off-topic from your original request, you may want to start a separate thread.
It sorts by Weight, then by Name. All menu entries will have a Name, so assuming different names, the iteration order should be deterministic. I haven’t looked, but I suspect the issue here is a theme NOT using the built-in menus. And it’s hard to blame Hugo for that. Take that with the Go std lib guys.
In that case, isn’t the issue really that TOML arrays get parsed into maps rather than arrays? I mean, it’s just an array of stuff so I’m not sure why on earth it would be represented as a map, surely the TOML parser should just stick the elements into an array in the order they’re encountered?
Since all of the nodes have the same Weight (0) and Name (""), the default Sort method always returns false (no swap). So, we must go deeper to find the initial order…
Hugo is retrieving the menu items from the config like this:
if menus := viper.GetStringMap("menu"); menus != nil {
I assume that the TOML/YAML package is maintaining the order that’s defined in the file, but Hugo asks for a map from Viper. We already covered maps and their random iteration order.
