The above highlighted line fixes it, but as it is not obvious reading the Netlify docs (they have promised to improve this), I thought I should post this.
DEPLOY_PRIME_URL contains the correct URL in pull request previews, but to make sure Hugo uses it for a deployment target, you need to pass it in as a argument to
hugo exactly like this:
[context.deploy-preview] command = "hugo -b $DEPLOY_PRIME_URL"
I have put exacty like this in bold, because I fumbled around with
context.deploy-preview.environment before Netlify told me it needed to be placed below