Add a way to import NPM modules as Hugo modules

feature
1

Currently, if you want to use Katex, you either have to check in a copy of it, or create a repo that contains a distribution of it, then import that. The former is unfortunate because the size of your module increases, and the latter is unfortunate because now you have to maintain that repo, checking for updates to Katex, and committing them to your Katex repo, and bump the Katex repo version in your module.

It would be great if we could “import” NPM modules as Hugo modules, where Hugo would download/install them at build time and make them available as mounts like Hugo modules. This would be the best of both worlds: your module doesn’t bloat from NPM dependencies, and you only have to specify the NPM package version in your module.

I don’t know if Go modules could be hacked to make this work, but in the worst case we could have:

[[module.npm.packages]]
path = "katex"
version = "0.16.4"

This would require npm to be installed. hugo env could print the npm version it found, etc.

2

I had a working prototype of this working which is why you get a HTTP 200 (OK) when going to https://gohugo.io/npmjs/katex

The idea would be that NPM packages would work exactly the same as packages on GitHub etc., but the path would be a path to a Go Module proxy server (which is a spec):

[[module.imports]]
path = "gohugo.io/npmjs/katex"

You would then control version upgrades the same way you’d do with other modules.

If I remember correctly, the one blocker I found with this is that it doesn’t seem to be a way to do this with a localhost proxy (a server started by the hugo command).

We’re not currently scaled to set up such a thing on a gohugo.io server – and that will bring all kinds of trust issues into the mix.

This may be worth revisiting, as the concept is rather simple. But note that I’m not prepared to build something entirely special for NPM (e.g. module.npm.packages) as there are so many aspects of this (version selection, vendoring, transitive dependencies, version graph, go workspaces, replace) – as that would mean a … ton of work.

Note to self to track down my work on this:

1 Like
3

It seems like too much work to have to deploy a special production Go proxy. Also, having to run that locally would be a hassle and error-prone.

By “import as Hugo modules,” I meant make it work like Hugo modules (as in making their files available for mounting), but they don’t have to necessarily be Hugo modules. One problem is that NPM package versions don’t have a “v” prefix. They also don’t have a go.mod. They might have other NPM deps that also have to be installed.

I think it would be easier to have a module specify the NPM packages it needs in the site working tree at build time, and Hugo puts them in /node_modules. Since NPM package versions can conflict between Hugo modules, make the NPM packages installed per Hugo module private to only that Hugo module. So, each Hugo module has its own version of /node_modules.

Edit: Anything a Hugo module mounts from its private /node_modules is visible in the shared/layered working tree.

4

Running a local proxy as part of the Hugo build isn’t hard (if it had worked) – it would be transparent for the end user.

5

I was actually wrong about the “local proxy server” not working.

I just tested this OK:

~/d/s/h/testgdeluxe ❯❯❯ GOPROXY=http://localhost:8072 hugo mod get gohugo.io/npmjs/katex        ✘ 130 main ✱ ◼
go: downloading gohugo.io/npmjs/katex v0.16.4
go: added gohugo.io/npmjs/katex v0.16.4

… but it’s still some missing pieces.

6

It would also need to avoid the “vendor” directory issue that Hugo modules has, e.g. the problem with bootstrap/scss/vendor.