Server-side or CLI rendering with KaTeX produces HTML, not an image, so you would still need to include CSS on the client side. If this is acceptable, roll your own local server, then use Hugo’s resources.GetRemote to send the tex and retrieve the HTML. The results are cached (getresource).
You cannot produce an SVG with KaTex, but you can with MathJax.
See https://discourse.gohugo.io/t/show-hugo-compile-time-latex-svg/15770.
Although the hosted app is no longer active, you can clone the repository and run it locally.