I have a remote media server which hosts various files like images, movies, tv shows, etc. and I want to organize them on a site build with Hugo. With rclone, I mounted the media content directory from the remote server to the assets directory of my Hugo project on my local machine. In my templates, I use resources.Match to retrieve all the paths to my media content and get their metadata (i.e. width and height of images, which are used in some templates) and to organize them on my pages.
With this setup, I am unable to build my project, because Hugo is trying to load hunderds of gigabytes of media data from the remote server. Why does Hugo need the complete movies and how to stop this behavior?
The best suggestion that I have is to build a JSON/YAML/TOML file containing the necessary meta data, then use resources.Get (local data) or resources.GetRemote in conjunction with transform.Unmarshal to access the data.
When you use the Get/GetMatch/Match/ByType methods you are “getting” the resource.
Thank you for your quick response!
Are you suggesting that I should build the metadata file by hand or has Hugo a built-in way to do this (the former is not really an option)?
I also plan to use hugo server to serve the site on my local network as a kind of media player, so fundamental changes to my setup are not really possible. I don’t think it is necessary that Hugo “getting a resource” / “matching filepaths” entails downloading the entire file. I run ffmpeg bash scripts on my local machine to generate movie thumbnails and this works perfectly, because only the data for one frame of a video is actually pulled from the remote media server. I am happy to look into this myself, but I need some directions in which parts of the source code to look.
No.
No. Find another way.
Hugo doesn’t work that way. When you say “Get” it gets.
No clue.
After some hours of stepping through the code I finally figured it out. The problem was my incomplete understanding of what “publishing a resource” (i.e. via .RelPermalink) does. The functions resources.[Get/GetMatch/Match/ByType] do not load complete files into memory as there is no way Hugo would be as fast as it is on big projects if that were the case.
Instead, “publishing a resource” means copying the file to the “public” directory, i.e. loading the complete file from disk. I did not notice this because I was using hugo server, so there were no files written to disk but only to Hugos vfs. It would be great if there was something like a --pass-through flag that bypasses the vfs and enables the use of .RelPermalink, etc. for projects like this.
FYI, with Hugo v0.123.0 and later hugo server renders to disk.