Noticing that hugo server does not seem to recognize the layouts/404.html file itself, although it does look like it compiles it, which works on GitHub Pages, etc.
Any desire to have hugo server actually use 404.html? If so I could add an issue for a PR eventually.
Pretty sure such an issue exists. I think it was harder to fix than the value. Hugo server is a development server; if you want to see your 404 you can navigate to it.