Don't display categories that are attached to "unlisted" posts

So currently I am displaying categories on homepage this way:

    {{range ($.Site.GetPage "taxonomyTerm" "category").Pages }}
      <li class="item">
        <a href="{{ .Permalink }}" class="tag">{{.Title}}</a>
      </li>
    {{end}}

The problem with this code is that it will also display categories for the articles that are currently unlisted. I unlist them with this code in the Hugo:

---
unlisted: true
---

and then I prevent them from being displayed in the view with something like:

[....] (where (where .Site.RegularPages "Section" "blog") ".Params.unlisted" "!=" true)

So although the posts wont appear in category list page, the category alone will still show up because I don’t specify that I want to skip displaying categories that have unlisted posts.

How can I achieve it?