Hello
I’m a big fan of Hugo, but I stumbled on some efficiency issue. If you put data in the /static directory and you generate your website, these files are copied to the /public folder. This is neglectable with small or little files on space issues, but can be an issue if you have many files and/or other big media files like movies and pdf’s. Should Hugo not implement a
–forceLinkStatic to link to the data, so requirements get less and processing gets even more quickly? This can be implemented on all OS’s where file linking is available (I guess all where Hugo is running) and there could be a difference for hard-
or softlinking (–forceSoftLinkStatic) when the OS has the system call.
Thank you for any replies, (counter)ideas or workarounds!
Regards
Philippe
Hi Alexandros
Thank you for the reply, but as I see it, assets are more for handling CSS and js documents to be filtered or extracted somehow.
What I meant is when you have a reference to media files like .jpg, .mp4, or .avi.
e.g. in your content you can write:
! [image] ( /image.jpg ) or
{{< video src="/my_video.mp4" controls=“yes” >}}
So you place this file into your /static folder to be referenced, but Hugo makes a copy to /public first. That gives double space requirements. It’s ok for a couple of files, but what if you want to make a gallery? If a link to that reference would be possible instead of a whole copy, then all this double space is not wasted and the result is the same. Also, all this overhead of copying is left out, so Hugo can process everything more quickly.
Hi Alexandros
I give an example:
You want to show a picture on a webpage, so you write in some file.md:
! [image] ( /image.jpg )
You place the file “image.jpg” into your ./static directory, then you process your site with hugo.
If you look in ./public, you will find ./public/image.jpg. The file is copied by Hugo from ./static/image.jpg to ./public. That means you need the double space.
If you have a gallery of pictures you need double space for every file.
But if linking would be available, then you keep only 1 file and some bytes from the link in your local or server disk.
I don’t see how you can do that with Page bundles?
No. The contents of /public/ is what is published on the web when a Hugo project is deployed.
If you use Git the contents of /static/ Headless Bundles and Page Bundles will also occupy space since they are in version control.
But if you simply host to a server you only need to push the /public/ folder.
To ignore a file from Git Version control you need to use a .gitignore file. Use it to exclude the /public/ folder so that it does not occupy redundant space in your repo.
By the way the Hugo Docs have an excellent section about using GitHub and other hosting providers.
Ok. You can deploy only the content of /public/ to the server, but it happens to be that I implemtented the git repository and the hugo site on the same server. I cannot gitignore the /static/, because if someone clones the repository to his local disk he would be left out with an empty /static/.
So the user can clone the repository, make changes and push the content back to the git server, where a local repository and hugo site is active (the git push calls a git post-receive to update the /public/ and the documentroot of the webserver is pointing to this /public/ directory). It all works flawless, except the server is stuck with all these copied media files. That’s why links would come in handy.
I could process the post-receive to checksum all media files and link them afterwards, but I was thinking why Hugo is copying all these files in the first place instead of linking?
A lot of folks generate their site locally, then upload the files to a web server somewhere. If Hugo did the file linking as you mentioned, this would break things in those scenarios.
Hi Zachary!
Not at all if you use hardlinks! Unix has it and NTFS. It doesn’t make any difference if you upload with hardlinks to your webserver. Plus, linking is much faster than copying and as I said you save on local or server diskspace.
That’s why I suggested to get a --forceLinkStatic switch for the hugo renderer.
Yes, indeed. I could use deduplication on the server itself. That would solve this copying problem Anyway, I’m quiet satisfied with the combined git and production setup, it’s all selfcontained. At the meantime I setup a storage outside the documentroot, but inside the production container to store all media. The user can then URL to this location without all the copying hastle. So, I force the users not to use this /static/ anymore.
Anyway, I’m quiet satisfied with the combined git and production setup, it’s all selfcontained. At the meantime I setup a storage outside the documentroot, but inside the production container to store all media. The user can then URL to this location without all the copying hastle. So, I force the users not to use this /static/ anymore.
He even suggested some hardlink options, same like I suggested.