Still reaching file limit error on OpenBSD

joeborg · December 26, 2020, 2:58pm

trying to explore the hard limits of my configuration and I’m reaching it with the attached tree. Raised maxproc and maxfiles in the kernel configuration as well as open-files in .login. Any pointers would be appreciated. At this point, just adding a directory in assets or layouts raises an error when using hugo serve. ‘content’ can still have folders and files added but not by a lot.

happy Xmass and New Year to all.

.
|-- _serve.sh
|-- archetypes
|-- assets
|   `-- css
|       |-- _debug.scss
|       |-- _index.scss
|       |-- article
|       |   |-- _article copy 2.md
|       |   |-- _article copy 3.md
|       |   |-- _article copy 4.md
|       |   |-- _article copy 5.md
|       |   |-- _article copy 6.md
|       |   |-- _article copy 7.md
|       |   |-- _article copy 8.md
|       |   |-- _article copy 9.md
|       |   |-- _article copy.md
|       |   |-- _article.md
|       |   |-- _article.scss
|       |   |-- _cols.md
|       |   |-- _cols.scss
|       |   |-- _hero.scss
|       |   |-- _index.scss
|       |   |-- _section.md
|       |   |-- _sections.scss
|       |   |-- _slider.scss
|       |   `-- section.html
|       |-- blockquote
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- cards
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- collections
|       |   |-- _index.scss
|       |   |-- _tiles.scss
|       |   `-- _tiles~.scss
|       |-- compounds
|       |   `-- _index.scss
|       |-- figsec
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- figure
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- fonts
|       |   |-- Alegreya sans.css
|       |   |-- _index.scss
|       |   |-- fira-condensed.css
|       |   |-- font-awesome.css
|       |   `-- yanone kafesatz.css
|       |-- master
|       |-- singlets
|       |   |-- _button.md
|       |   |-- _button.scss
|       |   |-- _image.md
|       |   |-- _image.scss
|       |   |-- _index.scss
|       |   |-- button.html
|       |   |-- image.html
|       |   `-- images.html
|       |-- style.scss
|       `-- stylesheets
|           |-- _reset.scss
|           |-- _typography.scss
|           |-- tools
|           |   |-- _index.scss
|           |   |-- _media.scss
|           |   |-- index~.scss
|           |   `-- media~.scss
|           |-- untitled folder
|           `-- untitled folder 2
|-- config.toml
|-- content
|   |-- _index.md
|   |-- about
|   |   |-- _assets
|   |   |   `-- chaplin.jpg
|   |   |-- _index.md
|   |   |-- affiliations
|   |   |   `-- index.md
|   |   |-- chaplin.jpg
|   |   |-- lead
|   |   |   `-- index.md
|   |   `-- practice
|   |       `-- index.md
|   |-- contact-us
|   |   `-- index.md
|-- data
|-- layouts
|   |-- _default
|   |   |-- baseof.html
|   |   |-- list.html
|   |   |-- single.html
|   |   |-- taxonomy.html
|   |   `-- untitled.txt
|   |-- index.html
|   |-- partials
|   |   |-- footer
|   |   |   `-- contact.html
|   |   |-- footer.html
|   |   |-- head.html
|   |   |-- header.html
|   |   |-- main.html
|   |   `-- menus
|   |       |-- _impressum.md
|   |       |-- _impressum.scss
|   |       |-- _social.md
|   |       |-- _social.scss
|   |       |-- impressum.html
|   |       `-- social.html
|   `-- shortcodes
|       |-- _index.scss
|       |-- article
|       |   |-- _article.md
|       |   |-- _article.scss
|       |   |-- _cols.md
|       |   |-- _cols.scss
|       |   |-- _hero.scss
|       |   |-- _index.scss
|       |   |-- _section.md
|       |   |-- _sections.scss
|       |   |-- _slider.scss
|       |   `-- section.html
|       |-- blockquote
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- cards
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- collections
|       |   |-- _index.scss
|       |   |-- _tiles.scss
|       |   |-- _tiles~.scss
|       |   `-- cards
|       |       |-- _index.scss
|       |       |-- c.html
|       |       `-- doc.md
|       |-- compounds
|       |   |-- _index.scss
|       |   |-- blockquote
|       |   |   |-- _index.scss
|       |   |   |-- c.html
|       |   |   `-- doc.md
|       |   `-- figsec
|       |       |-- _index.scss
|       |       |-- c.html
|       |       `-- doc.md
|       |-- figsec
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- figure
|       |   |-- _index.scss
|       |   |-- code.html
|       |   `-- doc.md
|       |-- master
|       `-- singlets
|           |-- _button.md
|           |-- _button.scss
|           |-- _image.md
|           |-- _image.scss
|           |-- _index.scss
|           |-- button.html
|           |-- figure
|           |   |-- _index.scss
|           |   |-- c.html
|           |   `-- doc.md
|           |-- image.html
|           `-- images.html
|-- resources
|-- static
|   |-- brand
|   |   |-- DC A.svg
|   |   |-- DC B.svg
|   |   |-- DC C.svg
|   |   |-- DC D.svg
|   |   |-- DC E.svg
|   |   |-- DCA black.svg
|   |   |-- DCA colour.svg
|   |   |-- DCA reversed black.svg
|   |   `-- DCA reversed red.svg
|   |-- fonts
|   |   |-- Alegreya
|   |   |   |-- Alegreya-Black.ttf
|   |   |   |-- Alegreya-BlackItalic.ttf
|   |   |   |-- Alegreya-Bold.ttf
|   |   |   |-- Alegreya-BoldItalic.ttf
|   |   |   |-- Alegreya-ExtraBold.ttf
|   |   |   |-- Alegreya-ExtraBoldItalic.ttf
|   |   |   |-- Alegreya-Italic.ttf
|   |   |   |-- Alegreya-Medium.ttf
|   |   |   |-- Alegreya-MediumItalic.ttf
|   |   |   |-- Alegreya-Regular.ttf
|   |   |   `-- OFL.txt
|   |   |-- AlegreyaSansLightItalic.woff2
|   |   |-- Alegreya_SC
|   |   |   |-- AlegreyaSC-Black.ttf
|   |   |   |-- AlegreyaSC-BlackItalic.ttf
|   |   |   |-- AlegreyaSC-Bold.ttf
|   |   |   |-- AlegreyaSC-BoldItalic.ttf
|   |   |   |-- AlegreyaSC-ExtraBold.ttf
|   |   |   |-- AlegreyaSC-ExtraBoldItalic.ttf
|   |   |   |-- AlegreyaSC-Italic.ttf
|   |   |   |-- AlegreyaSC-Medium.ttf
|   |   |   |-- AlegreyaSC-MediumItalic.ttf
|   |   |   |-- AlegreyaSC-Regular.ttf
|   |   |   `-- OFL.txt
|   |   |-- Alegreya_Sans
|   |   |   |-- AlegreyaSans-Black.ttf
|   |   |   |-- AlegreyaSans-BlackItalic.ttf
|   |   |   |-- AlegreyaSans-Bold.ttf
|   |   |   |-- AlegreyaSans-BoldItalic.ttf
|   |   |   |-- AlegreyaSans-ExtraBold.ttf
|   |   |   |-- AlegreyaSans-ExtraBoldItalic.ttf
|   |   |   |-- AlegreyaSans-Italic.ttf
|   |   |   |-- AlegreyaSans-Light.ttf
|   |   |   |-- AlegreyaSans-LightItalic.ttf
|   |   |   |-- AlegreyaSans-Medium.ttf
|   |   |   |-- AlegreyaSans-MediumItalic.ttf
|   |   |   |-- AlegreyaSans-Regular.ttf
|   |   |   |-- AlegreyaSans-Thin.ttf
|   |   |   |-- AlegreyaSans-ThinItalic.ttf
|   |   |   `-- OFL.txt
|   |   |-- Fira
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100.woff
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100.woff2
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100italic.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100italic.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100italic.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100italic.woff
|   |   |   |-- fira-sans-extra-condensed-v2-latin-100italic.woff2
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200.woff
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200.woff2
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200italic.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200italic.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200italic.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200italic.woff
|   |   |   |-- fira-sans-extra-condensed-v2-latin-200italic.woff2
|   |   |   |-- fira-sans-extra-condensed-v2-latin-600.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-600.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-600.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-600.woff
|   |   |   |-- fira-sans-extra-condensed-v2-latin-600.woff2
|   |   |   |-- fira-sans-extra-condensed-v2-latin-900.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-900.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-900.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-900.woff
|   |   |   |-- fira-sans-extra-condensed-v2-latin-900.woff2
|   |   |   |-- fira-sans-extra-condensed-v2-latin-italic.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-italic.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-italic.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-italic.woff
|   |   |   |-- fira-sans-extra-condensed-v2-latin-italic.woff2
|   |   |   |-- fira-sans-extra-condensed-v2-latin-regular.eot
|   |   |   |-- fira-sans-extra-condensed-v2-latin-regular.svg
|   |   |   |-- fira-sans-extra-condensed-v2-latin-regular.ttf
|   |   |   |-- fira-sans-extra-condensed-v2-latin-regular.woff
|   |   |   `-- fira-sans-extra-condensed-v2-latin-regular.woff2
|   |   |-- Yanone
|   |   |   |-- yanone-kaffeesatz-v9-latin-200.eot
|   |   |   |-- yanone-kaffeesatz-v9-latin-200.svg
|   |   |   |-- yanone-kaffeesatz-v9-latin-200.ttf
|   |   |   |-- yanone-kaffeesatz-v9-latin-200.woff
|   |   |   |-- yanone-kaffeesatz-v9-latin-200.woff2
|   |   |   |-- yanone-kaffeesatz-v9-latin-300.eot
|   |   |   |-- yanone-kaffeesatz-v9-latin-300.svg
|   |   |   |-- yanone-kaffeesatz-v9-latin-300.ttf
|   |   |   |-- yanone-kaffeesatz-v9-latin-300.woff
|   |   |   |-- yanone-kaffeesatz-v9-latin-300.woff2
|   |   |   |-- yanone-kaffeesatz-v9-latin-700.eot
|   |   |   |-- yanone-kaffeesatz-v9-latin-700.svg
|   |   |   |-- yanone-kaffeesatz-v9-latin-700.ttf
|   |   |   |-- yanone-kaffeesatz-v9-latin-700.woff
|   |   |   |-- yanone-kaffeesatz-v9-latin-700.woff2
|   |   |   |-- yanone-kaffeesatz-v9-latin-regular.eot
|   |   |   |-- yanone-kaffeesatz-v9-latin-regular.svg
|   |   |   |-- yanone-kaffeesatz-v9-latin-regular.ttf
|   |   |   |-- yanone-kaffeesatz-v9-latin-regular.woff
|   |   |   `-- yanone-kaffeesatz-v9-latin-regular.woff2
|   |   |-- alegreya-sans
|   |   |   |-- AlegreyaSans-Black.otf
|   |   |   |-- AlegreyaSans-BlackItalic.otf
|   |   |   |-- AlegreyaSans-Bold.otf
|   |   |   |-- AlegreyaSans-BoldItalic.otf
|   |   |   |-- AlegreyaSans-ExtraBold.otf
|   |   |   |-- AlegreyaSans-ExtraBoldItalic.otf
|   |   |   |-- AlegreyaSans-Italic.otf
|   |   |   |-- AlegreyaSans-Light.otf
|   |   |   |-- AlegreyaSans-LightItalic.otf
|   |   |   |-- AlegreyaSans-Medium.otf
|   |   |   |-- AlegreyaSans-MediumItalic.otf
|   |   |   |-- AlegreyaSans-Regular.otf
|   |   |   |-- AlegreyaSans-Thin.otf
|   |   |   |-- AlegreyaSans-ThinItalic.otf
|   |   |   |-- AlegreyaSansSC-Black.otf
|   |   |   |-- AlegreyaSansSC-BlackItalic.otf
|   |   |   |-- AlegreyaSansSC-Bold.otf
|   |   |   |-- AlegreyaSansSC-BoldItalic.otf
|   |   |   |-- AlegreyaSansSC-ExtraBold.otf
|   |   |   |-- AlegreyaSansSC-ExtraBoldItalic.otf
|   |   |   |-- AlegreyaSansSC-Italic.otf
|   |   |   |-- AlegreyaSansSC-Light.otf
|   |   |   |-- AlegreyaSansSC-LightItalic.otf
|   |   |   |-- AlegreyaSansSC-Medium.otf
|   |   |   |-- AlegreyaSansSC-MediumItalic.otf
|   |   |   |-- AlegreyaSansSC-Regular.otf
|   |   |   |-- AlegreyaSansSC-Thin.otf
|   |   |   |-- AlegreyaSansSC-ThinItalic.otf
|   |   |   `-- SIL Open Font License.txt
|   |   |-- awesome
|   |   |   |-- FontAwesome.otf
|   |   |   |-- fontawesome-webfont.eot
|   |   |   |-- fontawesome-webfont.svg
|   |   |   |-- fontawesome-webfont.ttf
|   |   |   `-- fontawesome-webfont.woff
|   |   `-- default
|   |       |-- alegreya-sans-v8-latin-100.eot
|   |       |-- alegreya-sans-v8-latin-100.svg
|   |       |-- alegreya-sans-v8-latin-100.ttf
|   |       |-- alegreya-sans-v8-latin-100.woff
|   |       |-- alegreya-sans-v8-latin-100.woff2
|   |       |-- alegreya-sans-v8-latin-100italic.eot
|   |       |-- alegreya-sans-v8-latin-100italic.svg
|   |       |-- alegreya-sans-v8-latin-100italic.ttf
|   |       |-- alegreya-sans-v8-latin-100italic.woff
|   |       |-- alegreya-sans-v8-latin-100italic.woff2
|   |       |-- alegreya-sans-v8-latin-300.eot
|   |       |-- alegreya-sans-v8-latin-300.svg
|   |       |-- alegreya-sans-v8-latin-300.ttf
|   |       |-- alegreya-sans-v8-latin-300.woff
|   |       |-- alegreya-sans-v8-latin-300.woff2
|   |       |-- alegreya-sans-v8-latin-300italic.eot
|   |       |-- alegreya-sans-v8-latin-300italic.svg
|   |       |-- alegreya-sans-v8-latin-300italic.ttf
|   |       |-- alegreya-sans-v8-latin-300italic.woff
|   |       |-- alegreya-sans-v8-latin-300italic.woff2
|   |       |-- alegreya-sans-v8-latin-500.eot
|   |       |-- alegreya-sans-v8-latin-500.svg
|   |       |-- alegreya-sans-v8-latin-500.ttf
|   |       |-- alegreya-sans-v8-latin-500.woff
|   |       |-- alegreya-sans-v8-latin-500.woff2
|   |       |-- alegreya-sans-v8-latin-500italic.eot
|   |       |-- alegreya-sans-v8-latin-500italic.svg
|   |       |-- alegreya-sans-v8-latin-500italic.ttf
|   |       |-- alegreya-sans-v8-latin-500italic.woff
|   |       |-- alegreya-sans-v8-latin-500italic.woff2
|   |       |-- alegreya-sans-v8-latin-700.eot
|   |       |-- alegreya-sans-v8-latin-700.svg
|   |       |-- alegreya-sans-v8-latin-700.ttf
|   |       |-- alegreya-sans-v8-latin-700.woff
|   |       |-- alegreya-sans-v8-latin-700.woff2
|   |       |-- alegreya-sans-v8-latin-700italic.eot
|   |       |-- alegreya-sans-v8-latin-700italic.svg
|   |       |-- alegreya-sans-v8-latin-700italic.ttf
|   |       |-- alegreya-sans-v8-latin-700italic.woff
|   |       |-- alegreya-sans-v8-latin-700italic.woff2
|   |       |-- alegreya-sans-v8-latin-800.eot
|   |       |-- alegreya-sans-v8-latin-800.svg
|   |       |-- alegreya-sans-v8-latin-800.ttf
|   |       |-- alegreya-sans-v8-latin-800.woff
|   |       |-- alegreya-sans-v8-latin-800.woff2
|   |       |-- alegreya-sans-v8-latin-800italic.eot
|   |       |-- alegreya-sans-v8-latin-800italic.svg
|   |       |-- alegreya-sans-v8-latin-800italic.ttf
|   |       |-- alegreya-sans-v8-latin-800italic.woff
|   |       |-- alegreya-sans-v8-latin-800italic.woff2
|   |       |-- alegreya-sans-v8-latin-900.eot
|   |       |-- alegreya-sans-v8-latin-900.svg
|   |       |-- alegreya-sans-v8-latin-900.ttf
|   |       |-- alegreya-sans-v8-latin-900.woff
|   |       |-- alegreya-sans-v8-latin-900.woff2
|   |       |-- alegreya-sans-v8-latin-900italic.eot
|   |       |-- alegreya-sans-v8-latin-900italic.svg
|   |       |-- alegreya-sans-v8-latin-900italic.ttf
|   |       |-- alegreya-sans-v8-latin-900italic.woff
|   |       |-- alegreya-sans-v8-latin-900italic.woff2
|   |       |-- alegreya-sans-v8-latin-italic.eot
|   |       |-- alegreya-sans-v8-latin-italic.svg
|   |       |-- alegreya-sans-v8-latin-italic.ttf
|   |       |-- alegreya-sans-v8-latin-italic.woff
|   |       |-- alegreya-sans-v8-latin-italic.woff2
|   |       |-- alegreya-sans-v8-latin-regular.eot
|   |       |-- alegreya-sans-v8-latin-regular.svg
|   |       |-- alegreya-sans-v8-latin-regular.ttf
|   |       |-- alegreya-sans-v8-latin-regular.woff
|   |       `-- alegreya-sans-v8-latin-regular.woff2
|   |-- icons
|   |   |-- fa-brands.svg
|   |   |-- fa-light.svg
|   |   |-- fa-regular.svg
|   |   |-- fa-solid.svg
|   |   |-- fontawesome-webfont.svg
|   |   |-- menu button black.svg
|   |   |-- menu button.svg
|   |   |-- mxd-symbols.svg
|   |   |-- signin.svg
|   |   |-- social black.svg
|   |   |-- social rev.svg
|   |   `-- social white.svg
|   `-- imgs
|-- themes

78 directories, 370 files

moorereason · December 26, 2020, 5:59pm

What do the following commands show?

grep openfiles /etc/login.conf
grep kern.maxfiles /etc/sysctl.conf

joeborg · December 26, 2020, 6:11pm

$ sysctl kern.maxfiles
  kern.maxfiles=7030

.login: set openfiles=30000

restarted the computer. Problem persists even if I switched off smtpd and httpd. I’m using nfs to edit from another machine. The Openbsd computer is a Quad-Core Celeron® J3455 SoC with 8Gb ram and an old SSD. I use it for testing stuff mostly.

Just to figure out what was happening I created a new site and started moving in things until it started crashing.

When I do a static build it finishes without errors but all .RelPermalinks are transformed into #ZgotmplZ. Images and css seem to be processed properly. Links work when hugo server doesn’t fail with the above error.

moorereason · December 27, 2020, 2:08pm

That’s a special value that tells you you’re violating the template engines security model. Search for that string in the forums, and you’ll find plenty of threads like this one to help you out.

joeborg · December 28, 2020, 7:09pm

For posterity: the issues were duplex: I didn’t add the http protocol with the baseURL=url in config.taml. My mental map is tied with httpd configuration on these things and didn’t notice the http://protocol added. It’s hard to unlearn… Regarding #ZgotmplZ urls: seems like the problem resided between seat and keyboard.

The Error: listen tcp 192.168.13.7:1313: socket: too many open files still persists. If I find it’s another issue that warms the seat, I’ll report it here.

moorereason · December 29, 2020, 4:28am

I haven’t spent much time with OpenBSD, so I’m shooting from the hip here.

According to https://man.openbsd.org/errno, “Too many open files” (error 24) means:

The maximum number of file descriptors allowable for this process has been reached and a request for an open cannot be satisfied until at least one has been closed. getdtablesize(3) will obtain the current limit.

So, you may be hitting a per-process limit. Does OpenBSD have a kern.maxfilesperproc sysctl value?

joeborg · December 29, 2020, 9:48am

I’ll look into it. AFAIK I had upped that one too but my memory isn’t the best. Thanks for the tip.

joeborg · December 29, 2020, 3:17pm

for posterity: I updated the following in sysctl.conf:

kern.maxfiles=102400

…and this in /etc/login.conf:

staff:\
:datasize-cur=4096M:\
:datasize-max=infinity:\
:maxproc-max=512:\
:maxproc-cur=256:\
:openfiles-max=102400:\
:openfiles-cur=102400

hugo server seems to work. Thought this wasn’t necessary since sysctl can be invoked from the command line.

system · December 31, 2020, 3:17pm

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Too many files (only on non root user) support	3	888	November 30, 2021
Start building sites … hugo v0.104.2+extended linux/amd64 Error: Error building site: process: readAndProcessContent: walk: Readdir: decorate: lstat /home/drm/WEBSITE/public/docs: no such file or directorye support	3	432	October 26, 2022
Server.go error support	2	721	October 15, 2016
How to deal with large amount of resource files outside Hugo's dir sturcture support	1	540	December 13, 2019
MKDIR error support	0	488	January 15, 2017

Still reaching file limit error on OpenBSD

Related Topics