How to add a search box backed by Elasticsearch


#1

I had no difficulty creating an Elasticsearch index on the Bonsai commercial service as documented in the links found on https://gohugo.io/tools/search/. I can query it using the curl CLI tool. What I can’t find are any straightforward directions for embedding a search box on my Hugo based blog that utilizes that index. This page explaining how to add a search box backed by Lunr is the sort of thing I’m looking for but for an Elasticsearch index. I’m not a frontend engineer. Despite a couple of hours googling and browsing sites like https://www.elastic.co/ I haven’t been able to find a simple “drop this into your Hugo site to enable Elasticsearch backed searching” solution.

Even though my blog is small (~400 articles over 5 years) I would prefer not to use a solution like Lunr which requires the client to load a large file since that doesn’t scale and can result in a poor user experience.


#2

I think we have all the ways listed in our forum: https://discourse.gohugo.io/search?q=lunr


#3

Thanks, @maiki, but I’m looking for equivalent resources for how to incorporate Elasticsearch into a Hugo site. It seems like the assumption is that anyone wanting to use Elasticsearch will be doing so in a commercial setting and thus will have the expertise and staff to wade through all the ES documentation and craft a solution from scratch.


#4

I’ll write tomorrow with code. Just finished doing this for my site at work. Borrowed it from Docdock. Depending on what you want, you may have to add a button and tweak CSS.


#5

@jmalin Dockdock uses Lunr.js though, which is not what OP asked for. OP doesn’t want to generate a large JSON file for search results.


#6

Oops. My mistake, I thought that krader1961 wanted to use commercial Elastic aka Swiftype. I can afford to do that, since my site is documentation for a software company. With the assistance of a former sales engineer from Lucidworks, I was able to generate JSON for searching, using only Hugo. I don’t know if I still have that around.


#7

There is no drop-in solution for Elasticsearch in order to get it set up for your website you will actually have to setup middleware to communicate with your database using one of Elasticsearch’s clients. (See here: https://www.elastic.co/guide/en/elasticsearch/client/index.html)

I’m no expert on the subject but I would set it up as follows.

  • Elastic Search Server
  • Middleware server running the Elasticsearch Javascript API (Since I like Node.js)*
  • Hugo Website

*The middleware can be on it’s own server or on the same server as Elasticsearch using Apache or something similar. Again I’m not an expert in this field so IDK if putting them on the same server would be good or bad in the long run.

I’d create a simple API on the middle-ware server based on my needs. This is good since none of your Elasticsearch credentials get exposed to the client. They remain on that middleware server.

So basically the search process would be like this:

  1. Visitor types something Client-side (in search bar on the Hugo website)
  2. Hugo website uses AJAX (https://developer.mozilla.org/en-US/docs/Web/Guide/AJAX) to send a request to middleware server
  3. Middleware server sends query based on the request to Elasticsearch
  4. Elasticsearch returns results to middleware server
  5. Middleware server spits out JSON based on the result and sends it back to the Hugo website
  6. Hugo website display results using the JSON.

Hopefully that helps you wrap your head around what would be involved.

Elasticsearch does have a javascript client for use in the browser, but it’s an experimental build. https://www.elastic.co/guide/en/elasticsearch/client/javascript-api/current/browser-builds.html. Also IDK if it’s meant for public facing sites. (Maybe if you can generate a read-only api key?)

Alternatives that don’t require users to download a large JSON file.

If this seems like too much work I recommend you check out these options.

Algolia

Forestry made a tutorial about how to sync up your Hugo site with Algolia. See here: https://forestry.io/blog/search-with-algolia-in-hugo/?q=&hPP=10&idx=blog&p=0

It involves generating a JSON file for search, but it’s not for users to download in the browser. It’s sent to Algolia to update the database so it scales much better than Lunr for large indexes. After you sync things up you can use Algolia’s instantsearch.js libary on your hugo website to search for results in your index. https://community.algolia.com/instantsearch.js/v2/getting-started.html

Algolia is a SaaS but it has a generous free tier

Swiftype

A commercial product by elastic. It allows you to add search to your site with by pasting a bit of code. It starts at $79/month though


#8

Oops I replied to the wrong person. See my other response.


#9

Ah, okay. From the original post, I thought you were asking how to point lunr.js to the index created by elasticsearch. The long answer from joshmossas is on point. :slight_smile:


#10

No worries, @jmalin. But I’m definitely not going to spend $80/month for the ability to search my personal blog :slightly_smiling_face: Even the $8/month of a commercial provisioner of the Elasticsearch service is hard to justify. But I figured that I could side-step even those minimal costs by running my own Elasticsearch instance.

Since it looks like there is no “do x, y, then z” to get even the most basic integration with a service like bonsai.com working I’ll probably just use Lunr and accept that the solution doesn’t scale.


#11

I recommend you check out Algolia. It scales really well and their free tier is pretty generous.


#12

Thanks for the detailed response, @joshmossas. I had already deduced that I would need some middleware to run alongside Caddy (the web server I use for providing access to my Hugo based blog). I was unable to find a “here’s how to do a basic implementation” article somewhere and was sad my google-fu didn’t find any relevant articles.

I had glanced at Algolia but didn’t proceed past their main web site page. I’ll take a closer look at that as a possible solution. Barring that I’ll fall back to Lunr.