Indexable page not in sitemap

support
1

Below you can see my sitemap.xml template.

{{ printf "<?xml version=\"1.0\" encoding=\"utf-8\" standalone=\"yes\"?>" | safeHTML }}

<urlset
  xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
  xmlns:xhtml="http://www.w3.org/1999/xhtml">

  {{- range .Data.Pages -}}
    {{- if not (or
      (strings.Contains .RelPermalink "/tags/")
      (strings.Contains .RelPermalink "/posts/")
      (strings.Contains .RelPermalink "/categories/")
      ) -}}

      <url>

        <loc>{{ .Permalink }}</loc>

        {{- with .Lastmod -}}
          <lastmod>{{ .Format "2006-01-02T15:04:05-07:00" | safeHTML }}</lastmod>
        {{- end -}}
        
        {{- with .Sitemap.ChangeFreq -}}
          <changefreq>{{ . }}</changefreq>
        {{- end -}}

        {{- with .Sitemap.Priority -}}
          <priority>{{ . }}</priority>
        {{- end -}}

        {{- if .IsTranslated -}}
          {{- range .AllTranslations -}}
            <xhtml:link rel="alternate" hreflang="{{ .Lang }}" href="{{ .Permalink | safeHTML }}" />
          {{- end -}}
        {{- end -}}

      </url>

    {{- end -}}
  {{- end -}}
</urlset>

As you can see I have excluded pages containing links to all posts in my blog (like this one: All Posts - Den's Hub: Technology Solutions, Guides and Best Practices).

Now, Ahrefs warns me that these pages are indexable, but are not in my sitemap, obviously.

Question: What’s best practice - exclude such pages (with all posts) or include them into sitemap?

2

This is not really a GoHugo question or answer, but a general one: A lot of “audit tools” warn about things that are not really an issue. A lot of aHrefs warnings can be disabled once you know about it and ignore them.

In general, your sitemap is YOUR notice to search engines and indexes about pages YOU care about. If a page is indexable that just means that your markup has no meta tags or robots.txt rules that disallow indexing that specific page. Not having it in your sitemap leads to search engines finding that page slower, mostly by being linked somewhere else on your page.

Long story short: yes and no. You decide what’s important for you. I personally have an noindex, follow on my list pages, meaning robots can follow all links on those list pages and should not index the page itself. I am doing that because that page is a dynamic representation of “now” on my website and coming back in a week or a month might show something completely different.

I think, in regards to ahrefs, if you add a meta-robots tag with a noindex, follow to your list template ahrefs will stop “warning” you. By the way, those warnings have no influence on the audit score at ahrefs. Only errors have and those should be fixed before you think about the yellow dots :slight_smile:

So, long story short: If you want them indexed, add a meta tag with index, follow and add them to the sitemap. if you don’t want them indexed, add a meta tag with noindex,follow and don’t add them to the sitemap.

1 Like
3

Thank you, @davidsneighbour, for the very detailed and helpful answer! I will follow your advice and put noindex, follow meta on those pages.

And you are correct, post seems to be off-topic here, so moderators can delete it, if needed.

BTW, I’m sick of my own perfectionism and am tackling the only remaining blue warnings, since I don’t have any red or yellow ones anymore :crazy_face:

4

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.