I think my issue is related to this. I think as a project grows, the issue becomes more prominent. So, I guess I will have to dedicate time to strip down the site (and we are talking tens of hours) and test one by one.
(I have diagnosed everything, from cache busters, to disabling fast render…but the issue still happens. So, I would not know what to point at as the culprit).
EDIT: Hugo v0.128.0 addresses this issue with defer.