Ok, here is an idea: I tried to think about this in Support gallery alongside albums on list.html · Issue #12 · nicokaiser/hugo-theme-gallery · GitHub
In the current develop branch (which I might merge and release later) I separated the album and photo lists from the templates, so they can be re-used.
This way you could override the list.html template and add page_gallery.html to it:
{{ define "main" }}
{{- partial "page_header.html" . -}}
{{- partial "page_albums.html" . -}}
+ {{- partial "page_gallery.html" . -}}
{{ end }}
This way all albums also display all of their images. Now to hide images (e.g. which should only be used as album thumbnails), you can move them to a subfolder (like images/) and reference them like featured_image: images/thumbnail.jpg. As images/ does not contain index.md nor _index.md it is not seen as page by Hugo but as part of the current page bundle.
However now listing empty albums gets more tricky: we cannot list all pages (because then, also real “pages” like “About” or “Imprint” get listed as empty albums). I cannot think of an elegant way to solve this – maybe this is the point where it would make sense to start using sections, which I always wanted in order to avoid complexity…