Multilingual site always get a noindex tag in root index.html


#1

I have made a multilingual site & I setup my config.toml (see below) to have "index,follow for robots.
And all works fine.

Except that the root index.html get an unexpected meta tag with noindex and this is badly evaluated by SEO tools.

And i just can’t get this to change, so any info/tip/link/idea welcome.

For the meta tag is works fine in my index.html in each language folder. I get

<meta name="robots" content="index,follow">

For Robots

For Sitemap

My robots.xml is also OK:

User-agent: *
Sitemap: https://camping-arolla-2018.netlify.com/sitemap.xml

This is my config.toml

enableRobotsTXT = true
defaultContentLanguageInSubdir = true
DefaultContentLanguage = "fr"

[params]
    [[Params.listeLangues]]
    code = "fr"
    [[Params.listeLangues]]
    code = "en"
    [[Params.listeLangues]]
    code = "nl"

[Languages]
    [Languages.fr]
        robots = "index,follow"
    [Languages.en]
        robots = "index,follow"
    [Languages.en]
        robots = "index,follow"

And this is the “faulty” meta tag inside my generated root index.html with noindex.

<head>
    <title>https://camping-arolla-2018.netlify.com/fr</title>
    <link rel="canonical" href="https://camping-arolla-2018.netlify.com/fr" />
    <meta name="robots" content="noindex">
    <meta charset="utf-8" />
    <meta http-equiv="refresh" content="0; url=https://camping-arolla-2018.netlify.com/fr" />
</head>

404.html not generated on root if defaultContentLanguageInSubdir true
#2

I’m using a conditional in my head for this:

… and setting “robotsdisallow” (not magic; could be any) in frontmatter as needed.


#3

Thanks a lot for the response @RickCogley !!

But I do the same thing …
It works fine on all the pages on each language, except on the root index.html (the index page outside language folders) which is accessed by default …

I guess this is somehow related to my directive defaultContentLanguageInSubdir = true.

<!DOCTYPE html>
<html class="no-js" lang="{{ with .Site.LanguageCode }}{{ . }}{{ else }}en-US{{ end }}">
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
        ...
        {{ if or (isset .Params "robots") (isset .Site.Params "robots") }}
           <meta name="robots" content="{{ with .Params.robots }}{{ . }}{{ else }}{{ .Site.Params.robots }}{{ end }}">
        {{ end }}
        ...

#4

I’m also wondering if this is related and share the same cause:

  • I get 404 pages correctly generated under my /root/lang/404.html (configured in /layout/404.html)
  • I can’t succeed to generate a /root/404.html

#5

I’m seeing the same thing in my site. I just opened: