The above highlighted line fixes it, but as it is not obvious reading the Netlify docs (they have promised to improve this), I thought I should post this.
The DEPLOY_PRIME_URL contains the correct URL in pull request previews, but to make sure Hugo uses it for a deployment target, you need to pass it in as a argument to hugoexactly like this:
I have put exacty like this in bold, because I fumbled around with context.deploy-preview.build and context.deploy-preview.environment before Netlify told me it needed to be placed below context.deploy-preview.
Let me put it this way: I spent a fair amount of time yesterday, some of it talking to Netlify on a chat, trying out different variations. So when I say exactly this then I really mean it.
Spent some time trying to debug why absolute links on Netlify aren’t generated. As it turned out, if you have baseURL = "" inside config.toml, then you also need to have these lines inside netlify.toml: