SOLVED Unable to set robots.txt content

I’m trying to set the content of the robots.txt file in my Hugo site, but am unable to change it from just “User-agent: *” (I just want to add two lines after that).

I’ve got the content I want in static/robots.txt, but this seems to not be used or perhaps overwritten.

The site source is at

I’ve used grep and find to try and find where this other content might be coming from. I’m guessing it’s Hugo default.

What should I do to get custom content in robots.txt?

Figured it out: the robots.txt file should be at ‘layouts/_default/robots.txt’

@hughgrigg did you try just putting your robots.txt in the static folder? The rule for the static folder is that everything therein is copied over as is.

1 Like

Yes, but it is (sometimes) useful to build it from a template (iterate over all the pages etc.)