Gitlab CI + Hugo module : error

Bertbk · October 21, 2020, 2:56pm

Hello,

My website is built and deployed on a (private) Gitlab server. Until now, I did not need Hugo module, only git submodule and everything were fine.

Since couple of weeks, I need to use Hugo module and since them, the Gitlab CI is broken. I get the same error as is I use Hugo from snap on my (linux) computer:

Error: we found a go.mod file in your project, but you need to install Go to use it. See https://golang.org/dl/.: module "github.com/wowchemy/wowchemy-hugo-modules/netlify-cms-academic" not found; either add it as a Hugo Module or store it in "/builds/project-0/themes".: module does not exist

Locally, I “fix” this by directly downloading the Hugo binary (instead of Snap). But on my Gitlab-CI, I do not know what to do. I try different Docker images but did not find a “good” one. Does anyone have an idea?

Here is my .gitlab-ci (note: I need both hugo module and git submodule). I used the monachus/hugo:latest but it (now) seems to fail.

variables:
  GIT_SUBMODULE_STRATEGY: recursive
  GIT_SSL_NO_VERIFY: "true"

pages:
  image: monachus/hugo:latest
  before_script:
    - apt-get update -y -qq
    - apt-get install git -y -qq
    - git submodule update --init --recursive
  script:
    - hugo
  artifacts:
    paths:
      - public
  only:
    - master

And these are the docker image I tried (maybe I did something wrong!)

Klakegg/alpine
tarampampam/hugo

Thanks in advance!

davidsneighbour · October 21, 2020, 2:58pm

Isn’t that explanation enough? A quick google of Gitlab and GoLang brought up this guide:

Bertbk · October 21, 2020, 3:18pm

Thanks for your answer. I tried to install go during the CI process but it failed (not enought space). I will try what you propose, even though it makes the process more complex than simply “download hugo docker image and launch it”.

Two remarks.

I followed the Hugo Documentation
A quick google search brought up that Hugo Snap binary is broken when using hugo module… and installing go lang does not, surprisingly, fix the problem, only changing the binary. I assume that installing Go during the CI process should not help, but I’ll give it a try anyway

davidsneighbour · October 21, 2020, 3:23pm

Did you check on the wowchemy forum about this issue? Maybe it came up there already.

It sounds like the “go missing” message might be mis-leading. What happens if you type in your local repo hugo mod vendor and add the newly added _vendor directory to your repo? That basically loads all modules into your repo, so Hugo does not need to load modules when it runs. Not sure though, because your description is a subjective description that might miss things that happen before that specific error comes up.

Put your repo public and people will have a look.

Bertbk · October 21, 2020, 3:56pm

What happens if you type in your local repo hugo mod vendor and add the newly added _vendor directory to your repo?

This makes everything work, indeed. I tried to add hugo mod get -u to download the modules before building the website, it did not work. The hugo mod get -u command seems to be launched.

This is what gitlab-runner shows:

$ hugo version
Hugo Static Site Generator v0.75.1-A4A7BAB7/extended linux/amd64 BuildDate: 2020-09-15T06:57:20Z
$ hugo mod get -u ./...
Update module in /builds/project-0
$ hugo
Error: we found a go.mod file in your project, but you need to install Go to use it. See https://golang.org/dl/.: module "github.com/wowchemy/wowchemy-hugo-modules/netlify-cms-academic" not found; either add it as a Hugo Module or store it in "/builds/project-0/themes".: module does not exist

I also tried hugo mod vendor but this fail with the go missing error.

Put your repo public and people will have a look.

Good idea. Currently this repo is private (and I want it to still be private), I will create a simple public repo with the problem and put the link here (probably tomorrow).

Thanks again.

davidsneighbour · October 21, 2020, 4:08pm

Add a script to your local repository with the following contents:

!#/bin/bash

hugo mod get -u ./...
hugo mod vendor
git add _vendor
git add go.*
git commit -m "chore: vendoring modules"

You could add this as push hook to your github repo (add it to .git/hooks/pre-push and chmod +x git/hooks/pre-push). This way you will always have updated modules in your repo.

Everything else… I am fighting it too, see the following post

https://discourse.gohugo.io/t/modular-hugo-on-netlify/28939/2

Bertbk · October 23, 2020, 2:16pm

EDIT: well there seems to be a fix!
EDIT 2 : The fix (=particular docker image) works, thanks for your help!

Thanks a lot, I’m gonna try. I’m on Gitlab and not Github and I’m not used to “push hook”… This is a huge problem if it doesn’t even work with Netlify!

Before I try this, here is a simple Gitlab Repo, which crashes, with the following errors. The repo is just a fork of wowchemy starter one plus a gitlab-ci.yml file

Running with gitlab-runner 13.5.0-rc2 (71c90c86)
  on docker-auto-scale 0277ea0f
Resolving secrets
00:00
Preparing the "docker+machine" executor
Using Docker executor with image monachus/hugo:latest ...
Pulling docker image monachus/hugo:latest ...
Using docker image sha256:dcd115f9613844746e5715e71aa24c2b88c2c18b518d469c07fb941ef9f03883 for monachus/hugo:latest with digest monachus/hugo@sha256:60ef3e3e2b9c650c6cf9988adfc06008edab8bf5d901e88c4216183a4f072b90 ...
Preparing environment
00:04
Running on runner-0277ea0f-project-21977324-concurrent-0 via runner-0277ea0f-srm-1603462103-055ce102...
Getting source from Git repository
00:02
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/bthierry/hugo-module/.git/
Created fresh repository.
Checking out fdefbc7f as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:01
$ hugo
Total in 4 ms
Error: we found a go.mod file in your project, but you need to install Go to use it. See https://golang.org/dl/.: module "github.com/wowchemy/wowchemy-hugo-modules/netlify-cms-academic" not found; either add it as a Hugo Module or store it in "/builds/bthierry/hugo-module/themes".: module does not exist
Cleaning up file based variables
00:00
ERROR: Job failed: exit code 1

bep · October 23, 2020, 2:41pm

Check this Docker image:

Maintained by @klakegg – and should have Go ready to use.

Bertbk · October 23, 2020, 3:09pm

Indeed, this docker image klakegg/hugo:ext-alpine seems to work fine. I tried on of Klakegg’s docker image previously but I failed. I guess I did not used it correctly. I posted bellow my gitlab-ci.yml file (An entrypoint must be set in the .gitlab-ci, though).

Maybe the hugo documentation should advice to use this image? Currently, it’s monachus that is adviced.

variables:
  GIT_SUBMODULE_STRATEGY: recursive

pages:
  image:
    name: klakegg/hugo:ext-alpine
    entrypoint : ['']
  before_script:
    - apk add --no-cache git
    - git submodule update --init --recursive
  script:
   - hugo
  artifacts:
    paths:
    - public
  only:
  - master

davidsneighbour · October 23, 2020, 3:44pm

It’s a Git feature, not Github or Gitlab related and worth a look for future problems.

https://git-scm.com/book/en/v2/Customizing-Git-Git-Hooks

klakegg · October 23, 2020, 3:48pm

The klakegg/hugo:ext-alpine image contains git - no need to reinstall it.

abhishek-kakkar · October 25, 2020, 7:57am

This is my working gitlab-ci.yml. I suppose the HUGO_ENV - production is required because it’s set to DEV by default.

image:
    name: klakegg/hugo:ext-alpine
    entrypoint : ['']

variables:
  GIT_SUBMODULE_STRATEGY: recursive
  HUGO_ENV: "production"

test:
  script:
  - hugo
  except:
  - master

pages:
  script:
  - hugo
  artifacts:
    paths:
    - public
  only:
  - master

I upgraded Wowchemy (formerly Academic) and this was what broke things. The GitLab CI file I linked above fixes it.

Bertbk · October 26, 2020, 10:00am

Thank you for the precision (and thanks for the great Docker image!).

~~I still meet a problem on my Gitlab self-hosted server (not on gitlab.com where everything works fine), the docker image cannot be downloaded:~~

Pulling docker image klakegg/hugo:ext-alpine ...
ERROR: Preparation failed: Error: No such image: klakegg/hugo:ext-alpine (executor_docker.go:182:0s)

~~I guess it’s due to the server options (allowed docker image) or because the docker image is too large (due to go ?)… Going to investigage. Thanks again!~~

~~Edit : (Self-hosted) Gitlab can download klakegg/hugo but not klakegg/hugo:ext-alpine docker image~~

Edit final: everything works fine, thanks!

Bertbk · October 26, 2020, 10:01am

Thanks! This indeed works ~~but not on my self-hosted Gitlab (at work). My problem is that gitlab does not succeed to download the docker image…~~

Bertbk · October 26, 2020, 10:02am

Thanks, I’m going to check!

system · October 28, 2020, 10:02am

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
How to deploy on Gitlab/ install hugo modules in Hugo docker image? support	3	1821	November 5, 2020
Hugo modules working locally, but failng in Gitlab CI support modules	4	324	December 27, 2023
Gitlab pages 404 - help? support	6	3175	September 4, 2018
Problems deploying Hugo site to Gitlab support	4	1189	December 5, 2017
Use a CentOS Docker Image to Build and Deploy a Hugo (Modules) Site to GitLab Pages tips & tricks	5	1386	December 23, 2020

Gitlab CI + Hugo module : error

Related Topics