Meilisearch: a blank canvas

Hey everybody! As some of you may know, I recently joined Meilisearch as a Developer Advocate. Part of my job is to help the community by answering questions, so my priority over the last few months has been to get to know Meilisearch. I believe the best way to learn new software is to use it just as any developer would: to build something. So, I set out to make a demo.

Meilisearch and MoMA

The first step was to find a nice dataset that I enjoyed working with. After several unsuccessful attempts to tame some unstructured datasets, I stumbled upon the repository of the Museum of Modern Art (MoMA). What a lucky find! They have two datasets: artists and artworks. I decided to go for the artworks dataset since it has all the essential information about each art piece: title, artist, dimensions, medium, even the artist’s biography. But the best part is that it’s available in JSON format! No conversion needed, no MongoDB dump to restore, no scraping: Meilisearch ready.

I used Meilisearch JavaScript to create the indexes, add the documents, and customize the settings. For the front-end, I decided to go for Instant Meilisearch combined with Vue InstantSearch as described in the meilisearch-vue repository. I’m a little lazy, so if I find a tool that provides nice results effortlessly, I will use it! Fortunately, InstantSearch provides all the front-end tools to customize the search environment at will, so you can experience the power of Meilisearch and its search-as-you-type feature with just a few lines of code. Finally, I tried to add some decent styling, but it is not my forte, so apologies in advance for the visual design.

You can see the result here.

Solving problems

So, I had found a dataset, indexed it in a Meilisearch instance, and set up a front-end for the search. All done, right? Not exactly; now came the time to iterate and make the search results feel good.

I was lucky to find a structured JSON dataset that worked effortlessly with Meilisearch. Nevertheless, I had to make some small changes to better display the results. Since Meilisearch’s automatic highlighting doesn’t work on query results that are stored in an array, I had to convert array values into strings before adding them.

I also wanted to allow users to sort the results by date. Since the date field didn’t have a standard format, I had to retrieve the year and store it in a new sortByDate field. Then there was another problem: at the time, Meilisearch did not support sorting at query time. To work around that, I created two more indexes with the custom ranking rules desc(DateToSortBy) and asc(DateToSortBy). Every time you changed the sorting option in the drop-down, Meilisearch was searching a completely different index, but it was fast enough that the switch was invisible.

Here you can see the sorting in action:

Search-time sorting was one of the most requested features on our public roadmap at the time, and it shipped in v0.22. The demo now uses a single index and a sortableAttributes setting instead of three indexes. You can read how that rework went in the art of sorting.

The most challenging part of creating this demo was working with an unknown dataset of 138,151 documents, because the large number of documents makes it difficult to know the possible values and length of the different fields. I first wanted the user to be able to filter results by medium (e.g. “photograph” or “paint”). That is, until I realized some of the artworks had medium fields that were many lines long! Take this one for example:

"H (vol. IX): Cover with lithographed manuscript text by Kliun on front; 1 lithographed illustration by Terent'ev; lithographed manuscript text. L (vol. XIII): Cover with lithographed illustration and manuscript design by Kirill Zdanevich on front; lithographed manuscript and typed text. M (vol. XIV): Cover with lithographed manuscript design and illustration on front by Kirill Zdanevich; lithographed manuscript and typed text. P (vol. XVII): Cover with lithographed manuscript design on front by Goncharova; and lithographed manuscript text by Mikhail Pustynin and Olga Olesha-Suok"

Can you imagine having this as a facet filter? Not very nice for the UI.

Close to perfect

The easiest part of making this demo was setting up Meilisearch. Once I decided which attributes I wanted to be displayed and searchable, the rest was a piece of cake. Aside from the highlighting and sorting issues mentioned above, the last thing I changed was the attribute ranking order. By default, the attribute ranking order is generated automatically based on the attributes' order of appearance in the first document indexed. In my case, the first attribute was title, which meant that artwork with matching query terms in the title field topped the results list. This delivered decent results, but I was pretty sure I could do better.

I not only had to consider which attributes are most important for this particular dataset; I also had to think about what types of queries are more likely to be made. If a user queries for a specific art piece using its title, he will get relevant results because the words used are unlikely to be found in lots of other fields. However, if a user queries the name of a famous artist, it is very possible to find that name in an artwork title, for example as a tribute. So I decided to place artist at the top of the searchable attributes list. Before I made the change, this is what you would find among the first results when searching for “Van Gogh” :

As you can see, it’s an art piece created by an artist named Willem Jacob Henri Berend Sandberg where the name “Van Gogh” appears in the title, but it’s not an actual Van Gogh.

Now, after changing the order of the searchable attributes in the settings, the first result is well and truly a piece of art created by Vincent Van Gogh:

Pretty amazing how a simple change like this can impact the search results. Meilisearch is ultra relevant out-of-the-box, but it lets you tune the relevancy of the results according to your needs, and that is essential to provide an awesome experience for the end user.

An open-source museum experience

The demo source code is available on GitHub. Feel free to play with it and show me what you can do.

How cool is it that we can access this extraordinary catalog? Thanks to MoMA’s generosity, you can explore the museum from anywhere in the world.

If you want to build something similar without provisioning a server first, Meilisearch Cloud gives you an instance to point the front end at, with a 14-day free trial and no credit card required.

Photo by Robert Bye on Unsplash

Frequently asked questions (FAQs)

What is the MoMA demo?

It is a search interface over the Museum of Modern Art's public artworks dataset, roughly 138,000 documents, built with Meilisearch on the back end and Vue InstantSearch on the front end. You can try it at moma.meilisearch.com.

Why does attribute order matter for relevancy?

Meilisearch's attribute ranking rule uses the order of the searchable attributes list to break ties. In this dataset, moving artist above title was the difference between a search for "Van Gogh" returning an actual Van Gogh and returning another artist's piece that merely mentions him in its title.

How do I sort search results by date in Meilisearch?

Add the field to sortableAttributes and pass a sort parameter at query time. This demo predates that feature and used three separate indexes with custom ranking rules instead, which is no longer necessary.

Why did the array values need converting to strings?

Meilisearch's automatic highlighting did not apply to values stored inside arrays at the time the demo was built, so array fields were flattened into strings before indexing to keep highlighting working in the results.