Fix tests for Windows

dev
1

Now that some work is being done to get Hugo running correctly on Windows, the tests need to be updated.

@tatsushid did a test integration with appveyor, https://github.com/spf13/hugo/issues/380, which is an excellent starting point.

I have started updating the tests for helpers/path_test.go and this post is specific to those tests. A new discussion should be started for things not related to helpers’ tests.

Output at start:

PS C:\code\go\src\github.com\spf13\hugo\helpers> go test
--- FAIL: TestReplaceExtension (0.00 seconds)
        path_test.go:124: Test 7 failed. Expected ".ext" got "mydir.ext"
        path_test.go:124: Test 8 failed. Expected ".ext" got "mydir.ext"
--- FAIL: TestDirExists (0.00 seconds)
        path_test.go:154: Test 6 failed. Expected true got false
        path_test.go:154: Test 7 failed. Expected true got false
--- FAIL: TestAbsPathify (0.00 seconds)
        path_test.go:376: Test 1 failed. Expected "/dir" but go "\\dir"
--- FAIL: TestFilename (0.00 seconds)
        path_test.go:405: Test 8 failed. Expected "" got "directoy"
--- FAIL: TestFileAndExt (0.00 seconds)
        path_test.go:434: Test 8 failed. Expected filename "" got "directoy".
--- FAIL: TestUrlPrep (0.00 seconds)
        url_test.go:65: Test #0 failed. Expected "/section/name/" got "%5Csection%5Cname%5Cindex.html/"
--- FAIL: TestPretty (0.00 seconds)
        Location:       url_test.go:72
        Error:          Not equal: "\\section\\name\\index.html" (expected)
                                != "/section/name/index.html" (actual)

        Location:       url_test.go:73
        Error:          Not equal: "\\section\\sub\\name\\index.html" (expected)
                                != "/section/sub/name/index.html" (actual)

        Location:       url_test.go:74
        Error:          Not equal: "\\section\\name\\index.html" (expected)
                                != "/section/name/index.html" (actual)

        Location:       url_test.go:75
        Error:          Not equal: "\\section\\name\\index.html" (expected)
                                != "/section/name/index.html" (actual)

        Location:       url_test.go:76
        Error:          Not equal: "\\index.html" (expected)
                                != "/index.html" (actual)

        Location:       url_test.go:77
        Error:          Not equal: "\\\\name\\index.xml" (expected)
                                != "/name/index.xml" (actual)

        Location:       url_test.go:80
        Error:          Not equal: "\\section\\name\\index.html" (expected)
                                != "/section/name" (actual)

        Location:       url_test.go:81
        Error:          Not equal: "\\section\\sub\\name\\index.html" (expected)
                                != "/section/sub/name" (actual)

        Location:       url_test.go:82
        Error:          Not equal: "\\section\\name\\index.html" (expected)
                                != "/section/name" (actual)

        Location:       url_test.go:83
        Error:          Not equal: "\\section\\name\\index.html" (expected)
                                != "/section/name" (actual)

        Location:       url_test.go:84
        Error:          Not equal: "\\index.html" (expected)
                                != "/" (actual)

        Location:       url_test.go:85
        Error:          Not equal: "\\\\name\\index.xml" (expected)
                                != "/name/index.xml" (actual)

FAIL
exit status 1
FAIL    github.com/spf13/hugo/helpers   0.173s

I have fixed a couple of tests, but most of the remaining errors involve either the FileAndExt function or URLs.

FileAndExt:

--- FAIL: TestReplaceExtension (0.00 seconds)
        path_test.go:125: Test 7 failed. Expected ".ext" got "mydir.ext"
        path_test.go:125: Test 8 failed. Expected ".ext" got "mydir.ext"
--- FAIL: TestFilename (0.00 seconds)
        path_test.go:406: Test 8 failed. Expected "" got "directoy"
--- FAIL: TestFileAndExt (0.00 seconds)
        path_test.go:435: Test 8 failed. Expected filename "" got "directoy".

The URL errors are because path functions, which should use OS specific separators are also used for URLs. I propose that the URLs get their own functions wich are specific to URLs since the separators are not OS dependent.

In addition to the above issues, there are two tests that should be removed from the TestDirExists test table: the \tmp and \tmp\ tests since those aren’t valid for Windows.

@spf13 any guidance on this?

2

We should really separate out paths & urls.

I’ve started a lot of work around this. Source & Destination files should use paths, while pages should use Urls. There’s some overall work to refactor into this, but it’s clearly the right approach to take.

3

@spf13

  1. sounds like I shouldn’t touch separating out path and url separator handling since you have work in progress that would be affected by it.
  2. Is it ok to elide the /tmp and /tmp/ path tests since they aren’t valid for Windows? (I’m asking because I don’t like deleting tests to get a pass, but I don’t see a way around this without over-complicating things.)
  3. path.FileAndExt() causes the other test failures. All this function needes to do is accept a path and return that path’s file and ext, if they exist, or the portions that do exist. This must work consistently with Windows and follow the rules in the comments for what constitutes a ‘a no file name case.’

Is my description of the intent of path.FileAndExt() correct? I just want to clarify before I set about getting its behavior consistent in both Windows and *nix.

4

  1. Initially hugo had a few different concepts of a source.file and a destination.file. I’ve been working to consolidate them into a single type (or one for source and one for dest). I haven’t touched the url / path handling yet, and all my work is committed. So have at it.

  2. I’m not sure what to do here. afero is failing test for the same reason. I really hope there’s a better solution than deleting tests.

  3. I think you have it correct.

5
  1. One option would be to get the system’s temp directory and use that, if you wanted a /tmp specific solution . On Windows, that directory should not exist unless the user specifically creates it.

Since the path would commonly map to C;\, or whatever the system drive is, this would probably cause an error since permissions outside of the ‘users’ directory need elevated access, iirc.

I’ll tackle 3 first then.

I’ve been meaning to take a more detailed look at URL handling in Hugo since there seems to be some room for improvement; I was waiting for your refactor to drop first. I’ll probably start a separate topic on that after I’ve spent some time focusing on it.

6

Oh and for comment 1, I’d be interested in learning more. I’ve been thinking a lot about source and destination files because of the clean public directory issue and my desire to enable archiving of files in public that are affected by the synch for easy rollback of changes.

Normally, rollbacks of pushes should not be needed, but I think we’ve all been in situations where a push in production needed to be rolled back ASAP.

My thinking was that the build process could leverage afero and the final sync would push from a virtual filesystem to the physical; with an option to handle large media files differently as one may not want to have them in a virtualFS unless Hugo was running RAM resident; flushing the built content to disk as the persitent store.

7

Good news is the FileAndExt() function works properly. When I looked at the code, it looked like it should work as documented.

I swapped out FileAndExt() with a function I wrote that did essentially the same thing and the test results were exactly the same, even with filepath functions for creating paths.

Using filepath.FromSlash() in the tests results in the tests working in both *nix and Windows.

Once I can make sure that the changes work in both environments I’ll submit the patch.

The current blocker is getting master fixed so that Hugo will compile. I had already rebased when I discovered the compile issue.

8

A pull request was made for the changes to testing to get the path tests to work in an OS independent manner.

The remaining open issue is URL handling, which I might spin off into its own topic as there are a lot of URL related issues which should be considered when creating the URL specific functions for paths.

@spf13 thoughts?