I have made a multilingual site & I setup my config.toml (see below) to have "index,follow for robots.
Except that the root index.html get an unexpected meta tag with noindex and this is badly evaluated by SEO tools.
And i just can’t get this to change, so any info/tip/link/idea welcome.
For the meta tag is works fine in my index.html in each language folder. I get
<meta name="robots" content="index,follow">
For Robots
For Sitemap
My robots.xml is also OK:
User-agent: *
Sitemap: https://camping-arolla-2018.netlify.com/sitemap.xml
This is my config.toml
enableRobotsTXT = true
defaultContentLanguageInSubdir = true
DefaultContentLanguage = "fr"
[params]
[[Params.listeLangues]]
code = "fr"
[[Params.listeLangues]]
code = "en"
[[Params.listeLangues]]
code = "nl"
[Languages]
[Languages.fr]
robots = "index,follow"
[Languages.en]
robots = "index,follow"
[Languages.en]
robots = "index,follow"
And this is the “faulty” meta tag inside my generated root index.html with noindex .
<head>
<title>https://camping-arolla-2018.netlify.com/fr</title>
<link rel="canonical" href="https://camping-arolla-2018.netlify.com/fr" />
<meta name="robots" content="noindex">
<meta charset="utf-8" />
<meta http-equiv="refresh" content="0; url=https://camping-arolla-2018.netlify.com/fr" />
</head>
1 Like
I’m using a conditional in my head for this:
<!DOCTYPE html>
<html lang="{{ .Lang }}">
<head>
<meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
{{ with .Params.robotsdisallow }}<meta name="robots" content="noindex, nofollow, noarchive">{{ else }}<meta name="robots" content="index, follow, archive">{{ end }}
<title>{{ block "title" . }}{{ .Site.Title }}{{ end }}</title>
<meta name="description" content="{{ if .IsPage}}{{ .Description }}{{ else }}{{ if .IsHome }}{{ .Site.Params.description }}{{ else }}{{ if eq .Lang "en" }}Pages on '{{ .Title }}' from eSolia Inc., a Tokyo-based IT management and support company.{{ end }}{{ if eq .Lang "ja" }}'{{ .Title }}' に関するページ、東京をベースにしたITマネージメントやサポート会社、株式会社イソリアから。{{ end }}{{ end }}{{end}}">
<meta name="keywords" content="{{if .IsPage}}{{ range $i, $e := .Params.tags }}{{ if $i }}, {{ end }}{{ $e }}{{ end }}{{else}}{{ range $i, $e := .Site.Params.keywords }}{{ if $i }}, {{ end }}{{ $e }}{{ end }}{{end}}">
{{ if .Params.author }}<meta name="author" content="{{ .Params.author }}">{{ end }}
<link href="http://gmpg.org/xfn/11" rel="profile">
<meta name="generator" content="Hugo {{ .Hugo.Version}} {{ .Hugo.CommitHash }} {{ replace .Hugo.BuildDate "+0900" "JST" }}" />
<link rel="canonical" href="{{ .Permalink }}">
<script src="https://use.typekit.net/fwz4gtk.js"></script>
… and setting “robotsdisallow” (not magic; could be any) in frontmatter as needed.
Thanks a lot for the response @RickCogley !!
But I do the same thing …
I guess this is somehow related to my directive defaultContentLanguageInSubdir = true.
<!DOCTYPE html>
<html class="no-js" lang="{{ with .Site.LanguageCode }}{{ . }}{{ else }}en-US{{ end }}">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
...
{{ if or (isset .Params "robots") (isset .Site.Params "robots") }}
<meta name="robots" content="{{ with .Params.robots }}{{ . }}{{ else }}{{ .Site.Params.robots }}{{ end }}">
{{ end }}
...
I’m also wondering if this is related and share the same cause:
I get 404 pages correctly generated under my /root/lang/404.html (configured in /layout/404.html)
I can’t succeed to generate a /root/404.html
I’m seeing the same thing in my site. I just opened: