# Read media and JSON content outside hugo's work directory

**URL:** <https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815>\
**Category:** support\
**Created:** [March 4, 2020, 1:00pm UTC](https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815 "2020-03-04T13:00:01Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![gosho](https://avatars.discourse-cdn.com/v4/letter/g/e0b2c6/32.png) [@gosho](https://discourse.gohugo.io/u/gosho)\
**Post date:** [March 4, 2020, 1:00pm UTC](https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815/1 "2020-03-04T13:00:01Z")

</div>

I have the following setup:

- a repo/directory (website-generator) for the hugo generator containing theme, layouts and all core site building assets.
- a repo/directory (documentation) for the site content - markdowns, media, git history etc.
- a repo/directory (website) for the build output

[https://github.com/gardener](https://github.com/gardener) see repos website-generator, documentation, website  
(because I am a new user and I can post up to 2 links !??)

The site build is triggered by a shell script. It **copies** the content from documentation into hugo as _content_ folder.

Next, there is a pre-build step in which a nodejs pulls the commits for each markdown and serializes them as json side-by-side with the corresponding markdowns.

The site-building then involves git-related partials that make use of `.GetJSON` to read the jsons and present git-related information on the relevant pages.

Another use of resources that I have (still not published in the repos above) is `.readDir` which I use to read a set of images form a provided directory and create carousel out of them dynamically.

All this worked, until I decided to skip the copy content step and instead use the `--contentDir` flag and point hugo directly to documentation. As I read the documentation and the discussions here, both `.GetJSON` and `.readDir` are not intended to be used outside hugo’s work directory:

- `.GetJSON` docs: “the source files must reside within Hugo’s working directory”
- `.readDir` docs: “Gets a directory listing from a directory relative to the current working directory.”

I can understand if `.GetJSON` is dedicated to non-content data files that need not be published and therefore not necessarily part of _content_ directory itself. Although, as you see it can have its uses too.  
I’m not sure about the rationale behind restricting `.readDir` though, but I assume it’s the same.

There was a proposal to use

```auto
{{ with .Page.Resources.GetMatch $jsonfile }}
{{ .Content | transform.Unmarshal }}
{{ end }}

```

, which seems to be able to take it over for `.GetJSON` for non-hugo resident _content_ directory. I want to confirm first with you if that is the case?  
But even if it is, I am not sure how to construct the path for `$jsonfile` here. In my case it’s not going to be starting with _content_ and I don’t want to hardcode paths.

For the use case with loading image files I guess I can get away with the same approach and using `.Match` with wildcards (`**.jpg`)?( Provided that there is a way to resolve the path construction issues above.)

If Hugo is in no position to provide functions to read files from content directory regardless of its location, then I suppose I need to create the commits jsons in hugo’s data directory, maintain the same directory structure and map them to their markdown counterparts based on it. Unless there is a better way.  
For the carousel use case using `.readDir` however, I have no good ideas because the images are content and need to stay with the content directory.

I am aware that I can create a symlink from hugo’s work directory to the documentation (content) directory and solve the issue with copying content and preserve current behavior. I am asking for an advise how to approach these problems without creating additional resources to circumvent limitations?

---

<div class="post-metadata">

**Author:** ![maiki](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/maiki/32/16384_2.png) [@maiki](https://discourse.gohugo.io/u/maiki)\
**Post date:** [March 4, 2020, 5:15pm UTC](https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815/2 "2020-03-04T17:15:46Z")

</div>

What does npm do for you? I looked at one of the “remote pages” in your docs repo, and couldn’t figure out what is happening. What is an example of the JSON you are reading?

---

<div class="post-metadata">

**Author:** ![gosho](https://avatars.discourse-cdn.com/v4/letter/g/e0b2c6/32.png) [@gosho](https://discourse.gohugo.io/u/gosho)\
**Post date:** [March 4, 2020, 6:29pm UTC](https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815/3 "2020-03-04T18:29:49Z")

</div>

Npm does a couple of things:

1. fetches commits history. `.GetJSON` has limited authentication options and doesn’t work for me.
2. fetches content from remote locations (public/private repos, wikis…). Remote pages are just placeholders with metadata, including the remote content location.
3. rewrites remote links to changed location

Here’s a [sample json](https://github.com/gardener/documentation/blob/master/website/documentation/050-tutorials/content/howto/access_pod_from_local/_index.md.json) for the [corresponding markdown](https://github.com/gardener/documentation/blob/master/website/documentation/050-tutorials/content/howto/access_pod_from_local/_index.md).

---

<div class="post-metadata">

**Author:** ![maiki](https://yyz2.discourse-cdn.com/flex036/user_avatar/discourse.gohugo.io/maiki/32/16384_2.png) [@maiki](https://discourse.gohugo.io/u/maiki)\
**Post date:** [March 4, 2020, 7:17pm UTC](https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815/4 "2020-03-04T19:17:56Z")

</div>

I think I’m missing something cognitively: you can show the commits in a public repo, but you can _not_ make the commits available over a public API, to be consumed by your site at build time?

---

<div class="post-metadata">

**Author:** ![gosho](https://avatars.discourse-cdn.com/v4/letter/g/e0b2c6/32.png) [@gosho](https://discourse.gohugo.io/u/gosho)\
**Post date:** [March 4, 2020, 10:14pm UTC](https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815/5 "2020-03-04T22:14:51Z")

</div>

If by public repo you refer to the `website` [repo](https://github.com/gardener/website) for the hugo build output, yes. Although that is not intended or pursued in any way and in fact good housekeeping should have taken care of it. As for the latter part, once they are fetched by npm, no I can use no API that I am aware of.

A mirror of the repos I showed above works also in a private network, gathering far more internal material, generally protected in ways that require more advanced authentication than `.GetJSON` supports. That’s also not visible in the npm we discuss on the public repo above because the public site building variant does not need it. Bottom line is, it does not qualify as public API, so I cannot use `.GetJSON` directly to get the commits from our internal githubs during site building.

But anyway, the key question here is, what can I use to read from file system locations content that is not inside hugo’s directory. Consider also the other use case I outlined, where I need to read a bunch of media files that are part of the content and that won’t help me there, when content does not reside in hugo. Page resources seems a close match to what perhaps could be generally applicable approach for the problem. Provided that there is a solution how to avoid hardcoding the path.

---

<div class="post-metadata">

**Author:** ![gosho](https://avatars.discourse-cdn.com/v4/letter/g/e0b2c6/32.png) [@gosho](https://discourse.gohugo.io/u/gosho)\
**Post date:** [May 26, 2020, 3:35pm UTC](https://discourse.gohugo.io/t/read-media-and-json-content-outside-hugos-work-directory/23815/6 "2020-05-26T15:35:22Z")

</div>

Recapping what worked for me if anyone with similar problem hits that.

In a distributed setup where content is another git repo, distinct from your hugo website builder repo, what worked for me best with `.GetJSON` and `.readDir ` was to symlink the source repo as `content` in the hugo repo. The only thing I’ve noticed you don’t get in this way is `.GitInfo` (but this might be related to other issues as well, I haven’t debugged that particular problem as `.GitInfo` was too limited for my case anyway). However, if the jsons you read with `.GetJSON` are useful only for the site building purposes and do not need to be served as content, you are much better off moving them to hugo’s data folder. There you could also employ them in `which` expressions and read them even without explicit `.GetJSON` with `$.Site.Data.<json-file>`.  
Having a well-known content folder path solves my issues with `.readDir` too.

I don’t have an answer how to supply dynamically the `contentDir` site configuration parameter (or as `--contentDir`) to form valid path e.g. for `.readDir` (imagine this setting as an environment variable specific to a particular deployment). It seems that the two just don’t play well together.

HTH