Building hugo for purposes of contribution doesn't work (deps not installed?)

I wrote an update to the youtube shortcode (trivial, but useful).

I’d like to contribute it. I overrode the shortcode in my own local shortcode definitions and it Does the Thing.

I’d like to include a test to prove that The Thing is Done.

Granted:

$ env |grep GO
GOPATH=/Users/stevenharms/golang
GOROOT=/Users/stevenharms/go

I run: CGO_ENABLED=1 go install --tags extended github.com/gohugoio/hugo@latest

And I see things being downloaded (yay) but then it appears taht transitive deps are not loaded. To wit:

Starts off promising!

CGO_ENABLED=1 go install --tags extended gith
ub.com/gohugoio/hugo@latest
go: downloading github.com/gohugoio/hugo v0.108.0
go: downloading github.com/alecthomas/chroma/v2 v2.4.0
go: downloading github.com/bep/clock v0.3.0
...
...
../../golang/pkg/mod/github.com/dlclark/regexp2@v1.7.0/syntax/charclass.go:9:2: package unicode/utf8 is not in GOROOT (/Users/stevenharms/go/src/unicode/utf8)
../../golang/pkg/mod/golang.org/x/sys@v0.2.0/unix/dirent.go:10:8: package unsafe is not in GOROOT (/Users/stevenharms/go/src/unsafe)

Ker-splat. Unsurprisingly, nothing is installed in ~/go/bin

I’m unfamiliar with Golang, so maybe this has an obvious and brief fix?

Addendum:

go version
go version go1.19.4 darwin/amd64

What is the result of which go?

Have your tried clearing the Go module cache?

go clean -modcache
➜  hugo git:(add-youtube-shortcode-start-offset) ✗ which go
/usr/local/go/bin/go
➜  hugo git:(add-youtube-shortcode-start-offset) ✗ go version
go version go1.19.4 darwin/amd64
➜  hugo git:(add-youtube-shortcode-start-offset) ✗ go clean --modcache

I tried the same install chant after this, same error. Thanks for the suggestion though!

II would say that after Go Modules was introduced, building Hugo from source has mostly been simple.

I see 2 things to try:

While go install ... github.com/... should work, if you want to modify a shortcode in the Hugo source, I suspect the easies way is to start with cloning the repo:

git clone https://github.com/gohugoio/hugo.git
cd hugo
go build

Also, the -tags extended code is just needed to build a small part of Hugo and to get started (and it complicates the build setup as you need a C/C++ compiler installed). I would just run go build, go test. etc. without any additional flags.

When you run which go it shows that you have installed Go in /usr/local/go/bin/go.

But your GOROOT is /Users/stevenharms/go.

Maybe I’m missing something, but that doesn’t make sense to me.

When I perform a clean install of Go:

  • GOROOT points to the installation location.
  • GOPATH points to the go directory in my home directory. This is where downloaded packages are stored (~/go/pkg) and installed (~/go/bin).
  • CGO_ENABLED is set to 1, so you don’t need to set this when compiling.

You definitely got me around this. Thanks! I have a PR i want to get through and I want to write a test to prove the code. The only thing I don’t quite get is with a fresh clone of the hugo source, if I cd in and run the test suite, it says there aren’t any tests? What am I missing here?

➜  h git:(master) git log |head -1
commit 17055d1fa76ca23a408a2e6d4d67cc8023f4d2ed
➜  h git:(master) go build
➜  h git:(master) ls -l $GOPATH/bin/hugo
-rwxr-xr-x  1 stevenharms  staff  82524776 Dec 14 15:41 /Users/stevenharms/go/bin/hugo
➜  h git:(master) go test
?   	github.com/gohugoio/hugo	[no test files]
➜  h git:(master)

I am having a difficult time following what are you doing.

Assuming your GOPATH is $HOME/go, the process should be:

  1. Fork the Hugo repository
  2. Create a directory outside of the GOPATH ($HOME/code)
  3. cd into the directory you just created
  4. Clone your fork
  5. cd into the source directory ($HOME/code/hugo)
  6. Create a new branch with a meaningful name (git checkout -b fix/correct-typo)
  7. Make your changes.
  8. Compile (go install -tags extended)
  9. Test (mage -v test)
  10. Commit your changes locally
  11. Push the branch to your repository
  12. Head over to the Hugo repository to create the pull request

As ever, I’m moving closer, thanks to your help.

  1. Your list here should really form the basis of CONTRIBUTING.md! While I can edit templates and HTML, the golang toolchain and its attendant env vars make casual contribution onerous. Your summary is very helpful.
  2. For me it’s steps 7-9 in your list that are troublesome
➜  h git:(null-featureadd) ✗ go install -tags extended
# github.com/bep/golibsass/internal/libsass
In file included from json.cpp:2:
../../go/pkg/mod/github.com/bep/golibsass@v1.1.0/internal/libsass/../../libsass_src/src/json.cpp:1289:3: warning: 'sprintf' is deprecated: This function is provided for compatibility reasons only.  Due to security concerns inherent in the design of sprintf(3), it is highly recommended that you use snprintf(3) instead. [-Wdeprecated-declarations]
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/stdio.h:188:1: note: 'sprintf' has been explicitly marked deprecated here
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:215:48: note: expanded from macro '__deprecated_msg'

Chatty, but no error status. Continuing:

$ $GOPATH/bin/mage -v test 2>&1 |tee log
$ cat log
Running target: Test
exec: go "test" "./..." "-tags" "none"
?     github.com/gohugoio/hugo  [no test files]
ok    github.com/gohugoio/hugo/bufferpool (cached)
ok    github.com/gohugoio/hugo/cache/filecache  (cached)
ok    github.com/gohugoio/hugo/cache/namedmemcache  (cached)
--- FAIL: TestMethods (0.00s)
    --- FAIL: TestMethods/MethodsFromTypes (0.00s)
panic: dir must be set to the Hugo root [recovered]
  panic: dir must be set to the Hugo root
...
...[SNIP]

AFAICT, I’m in the hugo root. In the same dir as main.go. If I can get these tools running, I promise I can get steps 10-12 on my own. I do it all day every day. :slight_smile:

Addendum

I went back to the build steps suggested in CONTRIBUTING.md and found that the preferred test path is to run mage -v check. That also hit the same hugo root error as above.

You need to tell Go which packages to test, e.g. “all of them”:

go test ./...

OK @bep, I’m closer. I can confirm that go test ./... runs all the tests :+1:

So if I’m going to make a change to the youtube shortcode, I’ve isolated these two files to be the ones of interest:

  4  h   "tpl/tplimpl/embedded/templates/shortcodes/youtube.html" line 3
  5 %a   "hugolib/embedded_shortcodes_test.go" line 175

My theory is, make changes in buffer 4, above, test them in buffer 5. Great.

So…

➜  h git:(null-featureadd) go test hugolib/embedded_shortcodes_test.go
# command-line-arguments [command-line-arguments.test]
hugolib/embedded_shortcodes_test.go:45:13: undefined: newTestCfg
hugolib/embedded_shortcodes_test.go:65:2: undefined: writeSource
hugolib/embedded_shortcodes_test.go:65:38: undefined: simplePageWithURL
hugolib/embedded_shortcodes_test.go:69:7: undefined: buildSingleSite
hugolib/embedded_shortcodes_test.go:69:58: undefined: BuildCfg
hugolib/embedded_shortcodes_test.go:103:14: undefined: newTestCfg
hugolib/embedded_shortcodes_test.go:104:14: undefined: newTestHelper
hugolib/embedded_shortcodes_test.go:110:3: undefined: writeSource
hugolib/embedded_shortcodes_test.go:114:3: undefined: writeSource
hugolib/embedded_shortcodes_test.go:116:3: undefined: buildSingleSite
hugolib/embedded_shortcodes_test.go:116:3: too many errors
FAIL	command-line-arguments [build failed]
FAIL

Do i have to bootstrap some other test harness to test things in hugolib?

You need to use go test -run TestName ./thepackage

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.