Last-modified time changes but not the content of the file

support
1

Summary

I push through rsync my website to the remote server.

I observe that HTML files are pushing even if their content isn’t modified because the last-modified time has changed.

Description

After executing hugo build, The last-modified time of HTML files is modified but not their content.

This is a quiet annoying, because rsync pushes extra files that haven’t been modified

Rsync finds files that need to be transferred using a “quick check” algorithm (by default) that looks for files that have changed in size or in last-modified time.

I can add --checksum to compare the remote content with the local content, but it takes more compute resource.

Expected

IMO, if the file isn’t modified the last-modified time shouldn’t be modified.

What version of Hugo are you using (hugo version)?

$ hugo version
hugo v0.166.0+extended+withdeploy linux/amd64 BuildDate=unknown

Does this issue reproduce with the latest release?

This script checks if the last-modified time has been modified and not the index file.

#!/bin/bash

INDEX="public/index.html"
INDEX_COPY="index.html.copy"

# Get the last modification time of index.html
TIME_1="$(stat ${INDEX} -c %y)"

# Keep a copy of the file to compare with the new file
cp ${INDEX} ${INDEX_COPY}

# Update the website
hugo build

# Check if the file has been modified
if ! diff "${INDEX}" "${INDEX_COPY}"; then
  echo "${INDEX} has been modified"
  exit 1
fi

# Get the last modification time of index.html
TIME_2="$(stat ${INDEX} -c %y)"

# Compare the modified time
if [ "${TIME_1}" != "${TIME_2}" ]; then
  echo "the modification time has been updated"
fi
2

Sure, but it’s noise. I would rather incur this small cost during transfer rather than during each build.

See https://gohugo.io/host-and-deploy/deploy-with-rsync/.

Using the Hugo documentation site as an example, its public directory contains 2456 files for a total of 246 MB. After the initial transfer, subsequent transfers with rsync --checksum are very, very fast:

time rsync --checksum --chmod=D755,F644 --compress --delete --verbose --recursive public/ jdoe@20.242.251.118:/var/www/html/
sending incremental file list
functions/cast/index.html
functions/cast/tofloat/index.html
functions/cast/toint/index.html
functions/cast/tostring/index.html
opengraph/gohugoio-card-base-1_hu_524356e958a99f13.png
quick-reference/functions/index.html

sent 392,003 bytes  received 4,479 bytes  113,280.57 bytes/sec
total size is 248,620,335  speedup is 627.07

real    0m2.495s
user    0m0.078s
sys     0m0.051s
3

Thank Joe for sharing your point of view.

Yes the symptom isn’t a big deal, but it could if rsync sums the SHA of large files.
I think hugo does wrong to update last-modified time for files that aren’t modified.
Perhaps it would take time to change this important minor detail.

I don’t know how hugo works, but maybe hugo could skip the "recreation" of HTML where it isn’t necessary to recreate them and save build time and gain in performance.

4

When you build a project, Hugo builds the entire site, overwriting existing files in the publishDir. To do as you suggest, we would have to perform a checksum comparison for every file we would overwrite, every time we build a site. That is a waste of resources given that most sites in the wild are built in a CI/CD workflow where the publishDir is emphemeral.

The right place to handle this is, as needed, in the transport layer.

5

It’s make sense to not care if we are recreating file already existing if the publish directory is ephemeral.
But developer doesn’t compile every source files every time they edit a source file.

If I take how Makefile works: to know if it needs to recreate a binary, it looks the last-modified time of the source file and doesn’t sum the hash.
In the Makefile example, this is simple because the dependencies of a binary is explicitly written in the Makefile.

HTML files depend on Markdown files, layouts and probably other files I am missing.
And it looks like complicated to detect dependencies of files to create.

Now I understand why the right place to handle this is in the transport layer, and not during the build.