Searchkick [alpha]

:rocket: Search made easy

Searchkick provides sensible search defaults out of the box. It handles:

• stemming - tomatoes matches tomato
• special characters - jalapenos matches jalapeños
• extra whitespace - dishwasher matches dish washer
• misspellings - zuchini matches zucchini
• custom synonyms - qtip matches cotton swab

Powered by Elasticsearch

Battle-tested at Instacart

Get Started

Install Elasticsearch. For Homebrew, use:

brew install elasticsearch

Add this line to your application’s Gemfile:

gem "searchkick"

Add searchkick to models you want to search.

class Product < ActiveRecord::Base
  searchkick
end

Add data to the search index.

Product.reindex

And to query, use:

products = Product.search "2% Milk"
products.each do |product|
  puts product.name
  puts product._score # added by searchkick - between 0 and 1
end

Queries

Query like SQL

Product.search "2% Milk", where: {in_stock: true}, limit: 10, offset: 50

Search specific fields

fields: [:name, :brand]

Add conditions

where: {
  expires_at: {gt: Time.now}, # lt, gte, lte also available
  orders_count: 1..10,        # equivalent to {gte: 1, lte: 10}
  aisle_id: [25, 30],         # in
  store_id: {not: 2},         # not
  aisle_id: {not: [25, 30]},  # not in
  or: [
    [{in_stock: true}, {backordered: true}]
  ]
}

Order results

order: {_score: :desc} # most relevant first - default

Paginate

page: 3, per_page: 20 # works with will_paginate and kaminari
# or equivalently
limit: 20, offset: 40

Boost by a field

boost: "orders_count" # give popular documents a little boost

Partial Matches

By default, results must match all words in the query.

Product.search "fresh honey" # fresh AND honey

To change this, use:

Product.search "fresh honey", partial: true # fresh OR honey

Synonyms

class Product < ActiveRecord::Base
  searchkick synonyms: [["scallion", "green onion"], ["qtip", "cotton swab"]]
end

You must call Product.reindex after changing synonyms.

Indexing

Choose what data gets indexed.

class Product < ActiveRecord::Base
  def _source
    as_json only: [:name, :active], include: {brand: {only: [:city]}}
    # or equivalently
    {
      name: name,
      active: active,
      brand: {
        city: brand.city
      }
    }
  end
end

Searchkick uses find_in_batches to import documents. To eagar load associations, use the searchkick_import scope.

class Product < ActiveRecord::Base
  scope :searchkick_import, includes(:searches)
end

Get Better With Time

Improve results with analytics on conversions.

First, you must keep track of search conversions. The database works well for low volume, but feel free to use redis or another datastore.

class Search < ActiveRecord::Base
  belongs_to :product
  # fields: id, query, searched_at, converted_at, product_id
end

Add the conversions to the index.

class Product < ActiveRecord::Base
  has_many :searches

  def _source
    {
      name: name,
      conversions: searches.group("query").count
    }
  end
end

After the reindex is complete, tell the search method to use conversions.

Product.search "Fat Free Milk", conversions: true

Facets

search = Product.search "2% Milk", facets: [:store_id, :aisle_id]
search.facets.each do |facet|
  p facet
end

Advanced

Product.search "2% Milk", facets: {store_id: {where: {in_stock: true}}}

Deployment

Bonsai on Heroku

Install the add-on:

heroku addons:add bonsai

And create an initializer config/initializers/bonsai.rb with:

ENV["ELASTICSEARCH_URL"] = ENV["BONSAI_URL"]

Then deploy and reindex:

heroku run rake searchkick:reindex CLASS=Product

Migrating from Tire

1. Change search methods to tire.search and add index name in existing search calls

  Product.search "fruit"

should be replaced with

  Product.tire.search "fruit", index: "products"

1. Replace tire mapping w/ searchkick method

  searchkick index_name: "products_v2"

1. Deploy and reindex

  rake searchkick:reindex CLASS=Product # or Product.reindex in the console

1. Once it finishes, replace search calls w/ searchkick calls

Elasticsearch Gotchas

Inconsistent Scores

Due to the distributed nature of Elasticsearch, you can get incorrect results when the number of documents in the index is low. You can read more about it here. To fix this, do:

class Product < ActiveRecord::Base
  searchkick settings: {number_of_shards: 1}
end

Thanks

Thanks to Karel Minarik for Tire and Jaroslav Kalistsuk for zero downtime reindexing.

TODO

• Built-in synonyms from WordNet
• Autocomplete (partial word matching)
• Exact phrase matches (in order)
• Allow for "exact search" with quotes
• Test helpers - everyone should test their own search
• Did you mean?
• Make updates to old and new index while reindexing possibly with an another alias
• Dashboard w/ real-time analytics (separate gem)

Contributing

1. Fork it
2. Create your feature branch (git checkout -b my-new-feature)
3. Commit your changes (git commit -am 'Add some feature')
4. Push to the branch (git push origin my-new-feature)
5. Create new Pull Request