I understand that you don’t want to duplicate functionality from build chains, but I think there is a missing development feature. Serving out of public makes perfect sense, but sometimes you need additional files served just during development. Source files and source maps are primary examples.
I really wouldn’t want to have those files in the public directory. Instead I would want to be able to map additional development time fallback sources for files not found in public.