github / searchkick

Name Last Update
lib Loading commit data...
test Loading commit data...
.gitignore Loading commit data...
Gemfile Loading commit data...
LICENSE.txt Loading commit data...
README.md Loading commit data...
Rakefile Loading commit data...
searchkick.gemspec Loading commit data...

README.md

Searchkick

:rocket: Search made easy

Searchkick provides sensible search defaults. It handles:

  • stemming - tomatoes matches tomato
  • special characters - jalapeno matches jalapeño
  • extra whitespace - dishwasher matches dish washer
  • misspellings - zuchini matches zucchini
  • custom synonyms - qtip matches cotton swab

Plus:

  • query like SQL - no need to learn a new query language
  • reindex without downtime
  • continually improve results from conversions - pretty awesome

:tangerine: Battle-tested at Instacart

Get Started

Install Elasticsearch. For Homebrew, use:

brew install elasticsearch

Add this line to your application’s Gemfile:

gem "searchkick"

Add searchkick to models you want to search.

class Product < ActiveRecord::Base
  searchkick
end

Add data to the search index.

Product.reindex

And to query, use:

products = Product.search "2% Milk"
products.each do |product|
  puts product.name
  puts product._score # added by searchkick - between 0 and 1
end

Queries

Query like SQL

Product.search "2% Milk", where: {in_stock: true}, limit: 10, offset: 50

Search specific fields

fields: [:name, :brand]

Where

where: {
  expires_at: {gt: Time.now}, # lt, gte, lte also available
  orders_count: 1..10,        # equivalent to {gte: 1, lte: 10}
  aisle_id: [25, 30],         # in
  store_id: {not: 2},         # not
  aisle_id: {not: [25, 30]},  # not in
  or: [
    [{in_stock: true}, {backordered: true}]
  ]
}

Order

order: {_score: :desc} # most relevant first - default

Limit / offset

limit: 20, offset: 40

Boost by a field

boost: "orders_count" # give popular documents a little boost

Pagination

Plays nicely with kaminari and will_paginate.

# controller
@products = Product.search "milk", page: params[:page], per_page: 20

# view
<%= paginate @products %>

Partial Matches

By default, results must match all words in the query.

Product.search "fresh honey" # fresh AND honey

To change this, use:

Product.search "fresh honey", partial: true # fresh OR honey

Synonyms

class Product < ActiveRecord::Base
  searchkick synonyms: [["scallion", "green onion"], ["qtip", "cotton swab"]]
end

You must call Product.reindex after changing synonyms.

Indexing

Choose what data is indexed.

class Product < ActiveRecord::Base
  def _source
    as_json only: [:name, :active], include: {brand: {only: [:city]}}
    # or equivalently
    {
      name: name,
      active: active,
      brand: {
        city: brand.city
      }
    }
  end
end

Searchkick uses find_in_batches to import documents. To eager load associations, use the searchkick_import scope.

class Product < ActiveRecord::Base
  scope :searchkick_import, includes(:searches)
end

Continually Improve Results

Use data on search conversions to improve results. This can make a huge difference.

First, keep track of conversions. The database works well for low volume, but feel free to use Redis or another datastore.

class Search < ActiveRecord::Base
  belongs_to :product
  # fields: id, query, searched_at, converted_at, product_id
end

Add conversions to the index.

class Product < ActiveRecord::Base
  has_many :searches

  def _source
    {
      name: name,
      conversions: searches.group("query").count
    }
  end
end

Reindex and tell the search method to use conversions.

Product.search "Fat Free Milk", conversions: true

Facets

search = Product.search "2% Milk", facets: [:store_id, :aisle_id]
p search.facets

Advanced

Product.search "2% Milk", facets: {store_id: {where: {in_stock: true}}}

Deployment

Heroku

Choose an add-on: SearchBox, Bonsai, or Found.

# SearchBox
heroku addons:add searchbox:starter

# Bonsai
heroku addons:add bonsai

# Found
heroku addons:add foundelasticsearch

And create an initializer config/initializers/elasticsearch.rb with:

# SearchBox
ENV["ELASTICSEARCH_URL"] = ENV["SEARCHBOX_URL"]

# Bonsai
ENV["ELASTICSEARCH_URL"] = ENV["BONSAI_URL"]

# Found
ENV["ELASTICSEARCH_URL"] = ENV["FOUNDELASTICSEARCH_URL"]

Then deploy and reindex:

heroku run rake searchkick:reindex CLASS=Product

Reference

Reindex one record

product = Product.find 10
product.reindex

Use a different index name

class Product < ActiveRecord::Base
  searchkick index_name: "products_v2"
end

Eagar load associations

Product.search "milk", include: [:brand, :stores]

Do not load models

Product.search "milk", load: false

Migrating from Tire

  1. Change search methods to tire.search and add index name in existing search calls
  Product.search "fruit"

should be replaced with

  Product.tire.search "fruit", index: "products"
  1. Replace tire mapping w/ searchkick method
  searchkick index_name: "products_v2"
  1. Deploy and reindex
  rake searchkick:reindex CLASS=Product # or Product.reindex in the console
  1. Once it finishes, replace search calls w/ searchkick calls

Elasticsearch Gotchas

Inconsistent Scores

Due to the distributed nature of Elasticsearch, you can get incorrect results when the number of documents in the index is low. You can read more about it here. To fix this, do:

class Product < ActiveRecord::Base
  searchkick settings: {number_of_shards: 1}
end

Thanks

Thanks to Karel Minarik for Tire and Jaroslav Kalistsuk for zero downtime reindexing.

TODO

  • Built-in synonyms from WordNet

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request