Simple search for Hugo (drop-in)

I’ve been trying to find a really simple search solution for Hugo for a long time, and I think I’ve finally cobbled something together that is genuinely drop-in and go.

It’s not super pretty, but that means you can style it to suit your own needs.

Like most of the other solutions I’ve seen out there, this requires a JSON index to be generated at build time, so the first thing to do is update your config.toml file so that Hugo outputs JSON:

[outputs]
  home = ["HTML", "RSS", "JSON"]

Next you’ll need a template file for building the JSON, nothing unusual here, just a file called index.json in your /layouts/_default directory. Its contents should look like this (the only thing you might not have is what I’ve called ‘abstract’, which gets written to the ‘summary’ key, but you could easily adjust this so that it picks up the first couple hundred words in your article, or whatever you like):

{{/* layouts/_default/index.json */}}
{{- $index := slice -}}
{{- range where .Site.RegularPages.ByDate.Reverse "Type" "not in"  (slice "page" "json") -}}
    {{- $index = $index | append (dict "title" ( .Title | plainify ) "permalink" .Permalink  "tags" ( .Params.tags ) "categories" ( .Params.categories ) "summary" ( .Params.abstract | markdownify | htmlUnescape | plainify) ) -}}
{{- end -}}
{{- $index | jsonify -}}

So far, so good, you have a JSON file!

Last bit is to drop-in the code for searching - if you want you can split this out into dedicated Javascript and CSS files, but the beauty is that you don’t need to if you don’t want to - this will work just as it is.

<style>
    .elbi-results {
        width: 100%;
    }
    .elbi-results-item {
        background-color: #FFFFFF;
        color: #000000;
        padding: 10px;
        border: 1px solid black;
    }
</style>

<div>
    <input type="text" id="elbi-input" placeholder="Search the site...">
    <ul id="elbi-results"></ul>
</div>

<script>
    const input = document.getElementById("elbi-input");
    const results = document.getElementById("elbi-results");
    const request = new Request("/index.json");

    fetch(request)
        .then(response => response.json())
        .then(data => {
            let pages = data;

            input.addEventListener("input",function(){
                let filteredPages = pages;
                results.innerHTML = "";

                if (input.value != ""){
                    // Normalize and replace diacritics so searching for 'ramayana' will return matches for 'Rāmāyaṇa' etc
                    let searchterms = input.value.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().split(" ");
                    
                    // Apply a filter to the array of pages for each search term
                    searchterms.forEach(function(term) {
                        filteredPages = filteredPages.filter(function(page) {
                            // The description is the full object, includes title, tags, categories, and summary text
                            // You could make this more specific by doing something like:
                            // let description = page.title;
                            let description = JSON.stringify(page);
                            return description.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().indexOf(term) !== -1;
                        });
                    }); // end of filter for loop

                    // For each of the pages in the final filtered list, insert into the results list
                    filteredPages.forEach(function(page) {
                        
                        results.insertAdjacentHTML("beforeend","<li class='elbi-results-item'><h2 style='font-size: 1.5rem;'><a href='" + page.permalink + "'>" + page.title + "</a></h2><p>" + page.summary + "</p><p style='margin-top: 5px'>Tagged: <strong>" + page.tags + "</strong></p></li>");
                    }); // end of page for loop

                }; // end of IF
                
            }); // end of event listener
        });
</script>

With any luck, that’s it! There’s practically no configuration required, although you’ll probably want to style it, so I’ve included some classes so that you can target it easily.

Hope it helps!

9 Likes

Is there a working demo you could share?

1 Like

Sure, I’m trialling it on the front page search of https://www.understandingreligion.org.uk :slight_smile:

2 Likes

Nice great job, really fast search just a feedback it is always nice to have x option to hide the sidebar and results for better user experience.

1 Like

Version 2!

Now with ranking / weighting for results (and a ‘clear’ button in the search bar).

You can edit the weightings if you wanted to put more emphasis on tags vs title etc. :smiley:

<style>
    .elbi-results {
        width: 100%;
    }
    .elbi-results-item {
        background-color: #FFFFFF;
        color: #000000;
        padding: 10px;
        border: 1px solid black;
    }
</style>

<div>
    <input type="search" class="input is-size-4" id="elbi-input" placeholder="Search the site...">
    <ul id="elbi-results"></ul>
</div>

<script>
    const input = document.getElementById("elbi-input");
    const results = document.getElementById("elbi-results");
    const request = new Request("/index.json");
    const display_score = false; // Set to true if you want to see the score in the results

    fetch(request)
        .then(response => response.json())
        .then(data => {
            let pages = data;

            input.addEventListener("input",function(){
                let filteredPages = pages;
                results.innerHTML = "";

                // If there is something in the search field
                if (input.value != ""){

                    // Reset the page score to zero
                    filteredPages.forEach(function(page) {
                        page.score = 0;
                    });

                    // Create array of search terms, split by space character
                    // Normalize and replace diacritics
                    let searchterms = input.value.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().split(" ");

                    // Apply a filter to the array of pages for each search term
                    searchterms.forEach(function(term) {
                        if (term != "") {
                            filteredPages = filteredPages.filter(function(page) {
                                // The description is the full object, includes title, tags, categories, and summary text
                                // You could make this more specific by doing something like:
                                // let description = page.title;
                                // or you could combine fields, for example page title and tags:
                                // let description = page.title + ' ' + JSON.stringify(page.tags)
                                let description = JSON.stringify(page);
                                return description.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().indexOf(term) !== -1;
                            });
                        }
                    }); // end of filter for loop

                    // Apply weighting to the results
                    searchterms.forEach(function(term) {
                        if (term != "") {
                            // Loop through each page in the array
                            filteredPages.forEach(function(page) {

                                // Assign 3 points for search term in title
                                if (page.title.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                                    page.score += 3
                                };

                                // Assign 2 points for search term in tags
                                if (JSON.stringify(page.tags).normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                                    page.score += 2
                                };

                                // Assign 1 point for search term in summary
                                if (page.summary.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                                    page.score += 1
                                };

                                // Assign 1 point for search term in the page categories
                                if (JSON.stringify(page.categories).normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                                    page.score += 1
                                };
                            })
                        };                                      
                    });

                    // Filter out any pages that don't have a score of at least 1
                    filteredPages = filteredPages.filter(function(page){
                        return page.score > 0;
                    })

                    // sort filtered results by title
                    // borrowed from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
                    filteredPages.sort(function(a, b) {
                        const titleA = a.title.toUpperCase(); // ignore upper and lowercase
                        const titleB = b.title.toUpperCase(); // ignore upper and lowercase
                        if (titleA < titleB) {
                            return -1;
                        }
                        if (titleA > titleB) {
                            return 1;
                        }
                        // titles must be equal
                        return 0;
                    });
                    
                    // then sort by page score
                    filteredPages.sort(function(a, b) {
                        return b.score - a.score;
                    });

                    // For each of the pages in the final filtered list, insert into the results list
                    filteredPages.forEach(function(page) {
                        
                        results.insertAdjacentHTML("beforeend","<li class='elbi-results-item'><h2 style='font-size: 1.5rem;'><a href='" + page.permalink + "'>" + page.title + "</a></h2><p>" + page.summary + "</p><p style='margin-top: 5px'>Tagged: <strong>" + page.tags + "</strong></p></li>");

                        if (display_score == true) {
                            results.insertAdjacentHTML("beforeend","<p>Result score: " + page.score + "</p>")
                        };

                    }); // end of page for loop

                }; // end of IF
                
            }); // end of event listener
        });
</script>
2 Likes

Works perfectly, and easy to modify to use other front matter!

But I did not manage make it international. it works for example.md - but not for example.fr.md

You know how I can do that?

Hm - sorry, not sure as I haven’t used translation in any of my sites.

I imagine those translated files aren’t being picked up by the JSON template - you would need to adjust the

range where .Site.RegularPages

bit to pick up translated pages, but I can’t help further I’m afraid!

Fair answer :wink:

So I am just talking to myself here… With your code without any change I get as expected
http://localhost:1313/index.json - but I also get (unexpected) http://localhost:1313/fr/index.json with all the correct data in it - but I can access it (wait…).

When I change this line:

const request = new Request("/index.json");

to

const request = new Request("/fr/index.json");

It works fine for the 2nd language, but obviously not for the core language.

BTW, I use your code as a /shortcode/search-jsn.html - that makes me think why not just make another one called /shortcode/search-jsn-fr.html with const request = new Request("/fr/index.json"); in it and use it as {{< search-jsn-fr >}} in the *.md

It feels very clumsy to me, but works perfectly.

If anybody has a better way let me know please.

hello i added this as a shortcode but the css styling is not working

No idea what went wrong, but I have it copy & paste as shortcode and the CSS works (although I don’t use that CSS as I use Tailwind)

You should be able to concat those two JSON arrays (the English one and the French one) to create one masterlist of all pages / languages, then the normal filter/search functions would work on the entire array.

I’m having a look into how to do this as I’m not sure - if I make any progress I’ll post it here!

okay, i just moved the css inside the div and it’s working fine, here is an example <input type="search" style="width: 90%; background: #fff;" class="input is-size-4" id="elbi-input" placeholder="Search the site...">

Just a heads up that the input and is-size-4 classes aren’t relevant to this code, I mistakenly included them from my own project, which uses Bulma as the CSS framework - not sure if that is confusing things.

Bit of a stab in the dark here, but… try switching out the old <script> block with this version.

<script>
  async function fetchPages() {
    const response_en = await fetch('/index.json');
    const response_fr = await fetch('/fr/index.json');

    const pages_en = await response_en.json();
    const pages_fr = await response_fr.json();
    const pages = pages_en.concat(pages_fr);
    
    return pages;
  }
  fetchPages()
    .then(pages => {
      input.addEventListener("input",function(){
        let filteredPages = pages;
        results.innerHTML = "";

        // If there is something in the search field
        if (input.value != ""){

            // Reset the page score to zero
            filteredPages.forEach(function(page) {
                page.score = 0;
            });

            // Create array of search terms, split by space character
            // Normalize and replace diacritics
            let searchterms = input.value.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().split(" ");

            // Apply a filter to the array of pages for each search term
            searchterms.forEach(function(term) {
                if (term != "") {
                    filteredPages = filteredPages.filter(function(page) {
                        // The description is the full object, includes title, tags, categories, and summary text
                        // You could make this more specific by doing something like:
                        // let description = page.title;
                        // or you could combine fields, for example page title and tags:
                        // let description = page.title + ' ' + JSON.stringify(page.tags)
                        let description = JSON.stringify(page);
                        return description.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().indexOf(term) !== -1;
                    });
                }
            }); // end of filter for loop

            // Apply weighting to the results
            searchterms.forEach(function(term) {
                if (term != "") {
                    // Loop through each page in the array
                    filteredPages.forEach(function(page) {

                        // Assign 3 points for search term in title
                        if (page.title.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                            page.score += 3
                        };

                        // Assign 2 points for search term in tags
                        if (JSON.stringify(page.tags).normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                            page.score += 2
                        };

                        // Assign 1 point for search term in summary
                        if (page.summary.normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                            page.score += 1
                        };

                        // Assign 1 point for search term in the page categories
                        if (JSON.stringify(page.categories).normalize("NFD").replace(/[\u0300-\u036f]/g,"").toLowerCase().includes(term)) {
                            page.score += 1
                        };
                    })
                };                                      
            });

            // Filter out any pages that don't have a score of at least 1
            filteredPages = filteredPages.filter(function(page){
                return page.score > 0;
            })

            // sort filtered results by title
            // borrowed from https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/sort
            filteredPages.sort(function(a, b) {
                const titleA = a.title.toUpperCase(); // ignore upper and lowercase
                const titleB = b.title.toUpperCase(); // ignore upper and lowercase
                if (titleA < titleB) {
                    return -1;
                }
                if (titleA > titleB) {
                    return 1;
                }
                // titles must be equal
                return 0;
            });
            
            // then sort by page score
            filteredPages.sort(function(a, b) {
                return b.score - a.score;
            });

            // For each of the pages in the final filtered list, insert into the results list
            filteredPages.forEach(function(page) {
                
                results.insertAdjacentHTML("beforeend","<li class='elbi-results-item'><h2 style='font-size: 1.5rem;'><a href='" + page.permalink + "'>" + page.title + "</a></h2><p>" + page.summary + "</p><p style='margin-top: 5px'>Tagged: <strong>" + page.tags + "</strong></p></li>");

                if (display_score == true) {
                    results.insertAdjacentHTML("beforeend","<p>Result score: " + page.score + "</p>")
                };

            }); // end of page for loop

        }; // end of IF
        
      }); // end of event listener
  });
</script>

I haven’t been able to test that, so if it works then I’ll be amazed(!)

I will stay with the different shortcodes. It allows also for placeholder="different placeholder..." and I can search/filter different parameters.

I think I am all good here and thanks for the great code!

2 Likes

@lb13 Thank you. This worked brilliantly!

I have a couple of questions, if you have time?

Currently when tags aren’t assigned to a post, I get tagged: null. Is it possible to have tags only show in the list when they are present and show categories when they’re present (I use one taxonomy on one type of post and another elsewhere).

Unfortunately, I’m not good with JS.

Incidentally :slight_smile: there was this announcement at the recent HugoConf…

This looks like a really powerful solution! I still prefer mine for its ability to be dropped in with minimal setup, but that one looks like it would cope much better with very large datasets, plus the fuzzy matching and word highlighting are nice touches.

1 Like

Is it possible to have tags only show in the list when they are present and show categories when they’re present

Yes mate, not a problem.

What you want to do is find the for loop that starts with this comment:

// For each of the pages in the final filtered list, insert into the results list

There’s a line in that loop that begins: results.insertAdjacentHTML("beforeend",...

And then it writes out a load of HTML, dropping in some values (like the tags) along the way. What you’ll need to do is use some javascript variables to build that HTML first, and include some logic where appropriate, so that the end result looks just how you want.

Here’s an example from one of my other sites:

// For each of the pages in the final filtered list, insert into the results list
filteredPages.forEach(function(page) {
    
    let tagsHTML = '';
    page.tags.forEach(function(tag){
    tagsHTML += "<a href=\"/tags/" + tag.replace(/ /g,'-') + "\" class=\"tag mr-2\">" + tag + "</a>";
})

let linkAddress = '';
if ( page.link == null ) { linkAddress = page.permalink } else { linkAddress = page.link };

let flagColour = '';
let flagBgColour = '';
if ( page.section == 'resources') {
    flagBgColour = '#FFC300'; flagColour = '#403100'
} else if ( page.section == 'posts' ) {
    flagBgColour = '#0A7E8C'; flagColour = '#FFFFFF'
};

let moreDetailsButtonHTML = '';
if ( page.section == 'resources' ) {
    moreDetailsButtonHTML = "<a class=\"button is-dark my-2\" href=\"" + page.permalink + "\">More Details</a>"
}
  
// Capitalise first letter of section name, depluralise
let buttonText = "Go to " + page.section.charAt(0).toUpperCase() + page.section.slice(1,page.section.length - 1);

let resultHTML = "<li class='elbi-results-item'><span style=\"display: inline-block; background-color: " + flagBgColour + "; color: " + flagColour + "; padding: 10px; margin-top: 5px; margin-bottom: 10px;\"><a style=\"color: inherit\" href=\"/" + page.section + "\">" + page.section + "</a></span><h2 style='font-size: 1.5rem; font-weight: 800;'>" + page.title + "</h2><p>" + page.description + "</p><a class=\"button my-2 mr-2\" href='" + linkAddress + "'>" + buttonText + "</a>" + moreDetailsButtonHTML + "<p style='margin-top: 5px'>" + tagsHTML + "</p></li>";
  
results.insertAdjacentHTML("beforeend",resultHTML);

if (display_score == true) {
    results.insertAdjacentHTML("beforeend","<p>Result score: " + page.score + "</p>")
};

Not all of that will be relevant to your use case, but hopefully illustrates the principle - you build up little blocks of HTML, and then put those blocks together in a variable called resultHTML, and then you drop that variable into the final insertAdjacentHTML function. You can use javascript IF statements to alter the HTML that you produce, depending on whatever you want to test.

You could pretty much just reuse my little tag loop that builds a list of tag links, and then adapt it for the categories as well. They don’t have to be anchors, they could be spans, paragraphs, whatever. If there aren’t any tags (or categories) then the variable is empty, and so nothing gets added to the end HTML (no ‘null’ messages).

You’ll want to have a bit of a play with this approach so that the end result suits your needs, but hopefully that gives you the right idea.

@lb13 Thank you for your response - much appreciated!

I attempted to adapt your code to fit my site, but unfortunately the search ceased to function.

I think I will need to learn JS.