github / searchkick

Download as
Browse Code »

Commit 12caa1e64fb234fb355324efb576925d605b7571

Authored by Andrew Kane
2 parents 5fc35e07 5797824c
Exists in master and in 21 other branches allow_missing, conversions_period, conversions_v2, eager_load, find_in_batches, highlighted, index_name, lazy_index_name, less_deps, nested, reindex_async_threads, reindex_refactor, relation, relation_search, routing_queue, scope_attributes, search_as_you_type, synonyms_ci, test_language, threads, wildcard

Merge branch 'master' into raise_dangerous_operations

Showing 51 changed files with 489 additions and 234 deletions   Show diff stats
.travis.yml
  Diff comments   View file @12caa1e
... ... @@ -4,7 +4,7 @@ services:
4 4 - elasticsearch
5 5 - mongodb
6 6 before_install:
7   - - ./ci/before_install.sh
  7 + - ./test/ci/before_install.sh
8 8 script: bundle exec rake test
9 9 before_script:
10 10 - psql -c 'create database searchkick_test;' -U postgres
... ... @@ -14,14 +14,14 @@ notifications:
14 14 on_failure: change
15 15 gemfile:
16 16 - Gemfile
17   - - gemfiles/activerecord41.gemfile
18   - - gemfiles/activerecord40.gemfile
19   - - gemfiles/activerecord32.gemfile
20   - - gemfiles/activerecord31.gemfile
21   - - gemfiles/mongoid2.gemfile
22   - - gemfiles/mongoid3.gemfile
23   - - gemfiles/mongoid4.gemfile
  17 + - test/gemfiles/activerecord41.gemfile
  18 + - test/gemfiles/activerecord40.gemfile
  19 + - test/gemfiles/activerecord32.gemfile
  20 + - test/gemfiles/activerecord31.gemfile
  21 + - test/gemfiles/mongoid2.gemfile
  22 + - test/gemfiles/mongoid3.gemfile
  23 + - test/gemfiles/mongoid4.gemfile
24 24 matrix:
25 25 include:
26   - - gemfile: gemfiles/nobrainer.gemfile
  26 + - gemfile: test/gemfiles/nobrainer.gemfile
27 27 env: NOBRAINER=true
... ...
CHANGELOG.md
  Diff comments   View file @12caa1e
1   -## 0.9.1 [unreleased]
  1 +## 0.9.2 [unreleased]
  2 +
  3 +- Added support for Elasticsearch 2.0
  4 +- Added support for aggregations
  5 +- Added ability to use misspellings for partial matches
  6 +- Added `fragment_size` option for highlight
  7 +- Added `took` method to results
  8 +
  9 +## 0.9.1
2 10  
3 11 - `and` now matches `&`
4 12 - Added `transpositions` option to misspellings
5 13 - Added `boost_mode` and `log` options to `boost_by`
  14 +- Added `prefix_length` option to `misspellings`
  15 +- Added ability to set env
6 16  
7 17 ## 0.9.0
8 18  
... ...
README.md
  Diff comments   View file @12caa1e
... ... @@ -45,7 +45,7 @@ Add this line to your application’s Gemfile:
45 45 gem 'searchkick'
46 46 ```
47 47  
48   -For Elasticsearch 0.90, use version `0.6.3` and [this readme](https://github.com/ankane/searchkick/blob/v0.6.3/README.md).
  48 +For Elasticsearch 2.0, use the `elasticsearch2` branch and [this readme](https://github.com/ankane/searchkick/blob/elasticsearch2/README.md). For Elasticsearch 0.90, use version `0.6.3` and [this readme](https://github.com/ankane/searchkick/blob/v0.6.3/README.md).
49 49  
50 50 Add searchkick to models you want to search.
51 51  
... ... @@ -117,6 +117,35 @@ Limit / offset
117 117 limit: 20, offset: 40
118 118 ```
119 119  
  120 +### Results
  121 +
  122 +Searches return a `Searchkick::Results` object. This responds like an array to most methods.
  123 +
  124 +```ruby
  125 +results = Product.search("milk")
  126 +results.size
  127 +results.any?
  128 +results.each { ... }
  129 +```
  130 +
  131 +Get total results
  132 +
  133 +```ruby
  134 +results.total_count
  135 +```
  136 +
  137 +Get the time the search took (in milliseconds) [master]
  138 +
  139 +```ruby
  140 +results.took
  141 +```
  142 +
  143 +Get the full response from Elasticsearch
  144 +
  145 +```ruby
  146 +results.response
  147 +```
  148 +
120 149 ### Boosting
121 150  
122 151 Boost important fields
... ... @@ -240,6 +269,8 @@ end
240 269 ```ruby
241 270 class Product < ActiveRecord::Base
242 271 searchkick synonyms: [["scallion", "green onion"], ["qtip", "cotton swab"]]
  272 + # or
  273 + # searchkick synonyms: Proc.new { CSV.read("/some/path/synonyms.csv") }
243 274 end
244 275 ```
245 276  
... ... @@ -282,7 +313,7 @@ Or turn off misspellings with:
282 313 Product.search "zuchini", misspellings: false # no zucchini
283 314 ```
284 315  
285   -Swapping two letters counts as two edits. To count the [transposition of two adjacent characters as a single edit](https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance), use: [master]
  316 +Swapping two letters counts as two edits. To count the [transposition of two adjacent characters as a single edit](https://en.wikipedia.org/wiki/Damerau%E2%80%93Levenshtein_distance), use:
286 317  
287 318 ```ruby
288 319 Product.search "mikl", misspellings: {transpositions: true} # milk
... ... @@ -706,6 +737,12 @@ Dog.search &quot;airbudd&quot;, suggest: true # suggestions for all animals
706 737  
707 738 ## Debugging Queries
708 739  
  740 +See how Elasticsearch scores your queries with:
  741 +
  742 +```ruby
  743 +Product.search("soap", explain: true)
  744 +```
  745 +
709 746 See how Elasticsearch tokenizes your queries with:
710 747  
711 748 ```ruby
... ... @@ -759,6 +796,28 @@ Then deploy and reindex:
759 796 heroku run rake searchkick:reindex CLASS=Product
760 797 ```
761 798  
  799 +### Amazon Elasticsearch Service
  800 +
  801 +You must use an [IP-based access policy](http://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-gsg-search.html) for Searchkick to work.
  802 +
  803 +Include `elasticsearch 1.0.14` or greater in your Gemfile.
  804 +
  805 +```ruby
  806 +gem "elasticsearch", ">= 1.0.14"
  807 +```
  808 +
  809 +Create an initializer `config/initializers/elasticsearch.rb` with:
  810 +
  811 +```ruby
  812 +ENV["ELASTICSEARCH_URL"] = "http://es-domain-1234.us-east-1.es.amazonaws.com"
  813 +```
  814 +
  815 +Then deploy and reindex:
  816 +
  817 +```sh
  818 +rake searchkick:reindex CLASS=Product
  819 +```
  820 +
762 821 ### Other
763 822  
764 823 Create an initializer `config/initializers/elasticsearch.rb` with:
... ... @@ -1009,6 +1068,18 @@ Reindex all models - Rails only
1009 1068 rake searchkick:reindex:all
1010 1069 ```
1011 1070  
  1071 +Turn on misspellings after a certain number of characters
  1072 +
  1073 +```ruby
  1074 +Product.search "api", misspellings: {prefix_length: 2} # api, apt, no ahi
  1075 +```
  1076 +
  1077 +**Note:** With this option, if the query length is the same as `prefix_length`, misspellings are turned off
  1078 +
  1079 +```ruby
  1080 +Product.search "ah", misspellings: {prefix_length: 2} # ah, no aha
  1081 +```
  1082 +
1012 1083 ## Large Data Sets
1013 1084  
1014 1085 For large data sets, check out [Keeping Elasticsearch in Sync](https://www.found.no/foundation/keeping-elasticsearch-in-sync/). Searchkick will make this easy in the future.
... ...
ci/before_install.sh
View file @5fc35e0
... ... @@ -1,14 +0,0 @@
1   -#!/usr/bin/env bash
2   -
3   -wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.3.2.deb
4   -sudo dpkg -i elasticsearch-1.3.2.deb
5   -sudo service elasticsearch restart
6   -
7   -if [ -n "$NOBRAINER" ]; then
8   - source /etc/lsb-release && echo "deb http://download.rethinkdb.com/apt $DISTRIB_CODENAME main" | sudo tee /etc/apt/sources.list.d/rethinkdb.list
9   - wget -qO- http://download.rethinkdb.com/apt/pubkey.gpg | sudo apt-key add -
10   - sudo apt-get update -q
11   - sudo apt-get install rethinkdb
12   - sudo cp /etc/rethinkdb/default.conf.sample /etc/rethinkdb/instances.d/instance1.conf
13   - sudo service rethinkdb restart
14   -fi
gemfiles/activerecord31.gemfile
View file @5fc35e0
... ... @@ -1,7 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "sqlite3"
7   -gem "activerecord", "~> 3.1.0"
gemfiles/activerecord32.gemfile
View file @5fc35e0
... ... @@ -1,7 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "sqlite3"
7   -gem "activerecord", "~> 3.2.0"
gemfiles/activerecord40.gemfile
View file @5fc35e0
... ... @@ -1,8 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "sqlite3"
7   -gem "activerecord", "~> 4.0.0"
8   -gem "activejob_backport"
gemfiles/activerecord41.gemfile
View file @5fc35e0
... ... @@ -1,8 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "sqlite3"
7   -gem "activerecord", "~> 4.1.0"
8   -gem "activejob_backport"
gemfiles/mongoid2.gemfile
View file @5fc35e0
... ... @@ -1,7 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "mongoid", "~> 2"
7   -gem "bson_ext"
gemfiles/mongoid3.gemfile
View file @5fc35e0
... ... @@ -1,6 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "mongoid", "~> 3.1.0"
gemfiles/mongoid4.gemfile
View file @5fc35e0
... ... @@ -1,7 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "mongoid", "~> 4.0.0"
7   -gem "activejob_backport"
gemfiles/nobrainer.gemfile
View file @5fc35e0
... ... @@ -1,6 +0,0 @@
1   -source 'https://rubygems.org'
2   -
3   -# Specify your gem's dependencies in searchkick.gemspec
4   -gemspec path: "../"
5   -
6   -gem "nobrainer", "0.27.0"
lib/searchkick.rb
  Diff comments   View file @12caa1e
... ... @@ -29,6 +29,7 @@ module Searchkick
29 29 attr_accessor :wordnet_path
30 30 attr_accessor :timeout
31 31 attr_accessor :models
  32 + attr_writer :env
32 33 end
33 34 self.search_method_name = :search
34 35 self.wordnet_path = "/var/lib/wn_s.pl"
... ...
lib/searchkick/index.rb
  Diff comments   View file @12caa1e
... ... @@ -100,7 +100,7 @@ module Searchkick
100 100  
101 101 def similar_record(record, options = {})
102 102 like_text = retrieve(record).to_hash
103   - .keep_if { |k, v| !options[:fields] || options[:fields].map(&:to_s).include?(k) }
  103 + .keep_if { |k, _| !options[:fields] || options[:fields].map(&:to_s).include?(k) }
104 104 .values.compact.join(" ")
105 105  
106 106 # TODO deep merge method
... ... @@ -118,9 +118,7 @@ module Searchkick
118 118  
119 119 def search_model(searchkick_klass, term = nil, options = {}, &block)
120 120 query = Searchkick::Query.new(searchkick_klass, term, options)
121   - if block
122   - block.call(query.body)
123   - end
  121 + block.call(query.body) if block
124 122 if options[:execute] == false
125 123 query
126 124 else
... ... @@ -317,7 +315,9 @@ module Searchkick
317 315 max_gram: 50
318 316 },
319 317 searchkick_stemmer: {
320   - type: "snowball",
  318 + # use stemmer if language is lowercase, snowball otherwise
  319 + # TODO deprecate language option in favor of stemmer
  320 + type: options[:language] == options[:language].to_s.downcase ? "stemmer" : "snowball",
321 321 language: options[:language] || "English"
322 322 }
323 323 },
... ... @@ -347,6 +347,9 @@ module Searchkick
347 347  
348 348 # synonyms
349 349 synonyms = options[:synonyms] || []
  350 +
  351 + synonyms = synonyms.call if synonyms.respond_to?(:call)
  352 +
350 353 if synonyms.any?
351 354 settings[:analysis][:filter][:searchkick_synonym] = {
352 355 type: "synonym",
... ... @@ -377,7 +380,7 @@ module Searchkick
377 380 end
378 381  
379 382 if options[:special_characters] == false
380   - settings[:analysis][:analyzer].each do |analyzer, analyzer_settings|
  383 + settings[:analysis][:analyzer].each do |_, analyzer_settings|
381 384 analyzer_settings[:filter].reject! { |f| f == "asciifolding" }
382 385 end
383 386 end
... ... @@ -385,8 +388,8 @@ module Searchkick
385 388 mapping = {}
386 389  
387 390 # conversions
388   - if options[:conversions]
389   - mapping[:conversions] = {
  391 + if (conversions_field = options[:conversions])
  392 + mapping[conversions_field] = {
390 393 type: "nested",
391 394 properties: {
392 395 query: {type: "string", analyzer: "searchkick_keyword"},
... ... @@ -559,6 +562,5 @@ module Searchkick
559 562 obj
560 563 end
561 564 end
562   -
563 565 end
564 566 end
... ...
lib/searchkick/logging.rb
  Diff comments   View file @12caa1e
... ... @@ -62,7 +62,8 @@ module Searchkick
62 62 end
63 63  
64 64 def self.reset_runtime
65   - rt, self.runtime = runtime, 0
  65 + rt = runtime
  66 + self.runtime = 0
66 67 rt
67 68 end
68 69  
... ... @@ -122,7 +123,8 @@ module Searchkick
122 123  
123 124 module ClassMethods
124 125 def log_process_action(payload)
125   - messages, runtime = super, payload[:searchkick_runtime]
  126 + messages = super
  127 + runtime = payload[:searchkick_runtime]
126 128 messages << ("Searchkick: %.1fms" % runtime.to_f) if runtime.to_f > 0
127 129 messages
128 130 end
... ...
lib/searchkick/model.rb
  Diff comments   View file @12caa1e
... ... @@ -2,7 +2,6 @@ module Searchkick
2 2 module Reindex; end # legacy for Searchjoy
3 3  
4 4 module Model
5   -
6 5 def searchkick(options = {})
7 6 raise "Only call searchkick once per model" if respond_to?(:searchkick_index)
8 7  
... ... @@ -18,12 +17,11 @@ module Searchkick
18 17 class_variable_set :@@searchkick_callbacks, callbacks
19 18 class_variable_set :@@searchkick_index, options[:index_name] || [options[:index_prefix], model_name.plural, Searchkick.env].compact.join("_")
20 19  
21   - define_singleton_method(Searchkick.search_method_name) do |term = nil, options = {}, &block|
22   - searchkick_index.search_model(self, term, options, &block)
23   - end
24   - extend Searchkick::Reindex # legacy for Searchjoy
25   -
26 20 class << self
  21 + def searchkick_search(term = nil, options = {}, &block)
  22 + searchkick_index.search_model(self, term, options, &block)
  23 + end
  24 + alias_method Searchkick.search_method_name, :searchkick_search
27 25  
28 26 def searchkick_index
29 27 index = class_variable_get :@@searchkick_index
... ... @@ -43,12 +41,13 @@ module Searchkick
43 41 class_variable_get(:@@searchkick_callbacks) && Searchkick.callbacks?
44 42 end
45 43  
46   - def reindex(options = {})
  44 + def searchkick_reindex(options = {})
47 45 if respond_to?(:current_scope) && current_scope && current_scope.to_sql != default_scoped.to_sql
48 46 raise Searchkick::DangerousOperation, "Only call reindex on models, not relations"
49 47 end
50 48 searchkick_index.reindex_scope(searchkick_klass, options)
51 49 end
  50 + alias_method :reindex, :searchkick_reindex unless method_defined?(:reindex)
52 51  
53 52 def clean_indices
54 53 searchkick_index.clean_indices
... ... @@ -65,8 +64,8 @@ module Searchkick
65 64 def searchkick_index_options
66 65 searchkick_index.index_options
67 66 end
68   -
69 67 end
  68 + extend Searchkick::Reindex # legacy for Searchjoy
70 69  
71 70 if callbacks
72 71 callback_name = callbacks == :async ? :reindex_async : :reindex
... ... @@ -97,9 +96,7 @@ module Searchkick
97 96 def should_index?
98 97 true
99 98 end unless method_defined?(:should_index?)
100   -
101 99 end
102 100 end
103   -
104 101 end
105 102 end
... ...
lib/searchkick/query.rb
  Diff comments   View file @12caa1e
... ... @@ -92,24 +92,41 @@ module Searchkick
92 92 shared_options = {
93 93 query: term,
94 94 operator: operator,
95   - boost: factor
  95 + boost: 10 * factor
96 96 }
97 97  
  98 + misspellings =
  99 + if options.key?(:misspellings)
  100 + options[:misspellings]
  101 + elsif options.key?(:mispellings)
  102 + options[:mispellings] # why not?
  103 + elsif field == "_all" || field.end_with?(".analyzed")
  104 + true
  105 + else
  106 + # TODO default to true and remove this
  107 + false
  108 + end
  109 +
  110 + if misspellings != false
  111 + edit_distance = (misspellings.is_a?(Hash) && (misspellings[:edit_distance] || misspellings[:distance])) || 1
  112 + transpositions =
  113 + if misspellings.is_a?(Hash) && misspellings[:transpositions] == true
  114 + {fuzzy_transpositions: true}
  115 + elsif below20?
  116 + {}
  117 + else
  118 + {fuzzy_transpositions: false}
  119 + end
  120 + prefix_length = (misspellings.is_a?(Hash) && misspellings[:prefix_length]) || 0
  121 + max_expansions = (misspellings.is_a?(Hash) && misspellings[:max_expansions]) || 3
  122 + end
  123 +
98 124 if field == "_all" || field.end_with?(".analyzed")
99   - shared_options[:cutoff_frequency] = 0.001 unless operator == "and"
  125 + shared_options[:cutoff_frequency] = 0.001 unless operator == "and" || misspellings == false
100 126 qs.concat [
101   - shared_options.merge(boost: 10 * factor, analyzer: "searchkick_search"),
102   - shared_options.merge(boost: 10 * factor, analyzer: "searchkick_search2")
  127 + shared_options.merge(analyzer: "searchkick_search"),
  128 + shared_options.merge(analyzer: "searchkick_search2")
103 129 ]
104   - misspellings = options.key?(:misspellings) ? options[:misspellings] : options[:mispellings] # why not?
105   - if misspellings != false
106   - edit_distance = (misspellings.is_a?(Hash) && (misspellings[:edit_distance] || misspellings[:distance])) || 1
107   - transpositions = (misspellings.is_a?(Hash) && misspellings[:transpositions] == true) ? {fuzzy_transpositions: true} : {}
108   - qs.concat [
109   - shared_options.merge(fuzziness: edit_distance, max_expansions: 3, analyzer: "searchkick_search").merge(transpositions),
110   - shared_options.merge(fuzziness: edit_distance, max_expansions: 3, analyzer: "searchkick_search2").merge(transpositions)
111   - ]
112   - end
113 130 elsif field.end_with?(".exact")
114 131 f = field.split(".")[0..-2].join(".")
115 132 queries << {match: {f => shared_options.merge(analyzer: "keyword")}}
... ... @@ -118,6 +135,10 @@ module Searchkick
118 135 qs << shared_options.merge(analyzer: analyzer)
119 136 end
120 137  
  138 + if misspellings != false
  139 + qs.concat qs.map { |q| q.except(:cutoff_frequency).merge(fuzziness: edit_distance, prefix_length: prefix_length, max_expansions: max_expansions, boost: factor).merge(transpositions) }
  140 + end
  141 +
121 142 queries.concat(qs.map { |q| {match: {field => q}} })
122 143 end
123 144  
... ... @@ -134,7 +155,7 @@ module Searchkick
134 155 if below12?
135 156 {script_score: {script: "doc['count'].value"}}
136 157 else
137   - {field_value_factor: {field: "count"}}
  158 + {field_value_factor: {field: "#{conversions_field}.count"}}
138 159 end
139 160  
140 161 payload = {
... ... @@ -143,13 +164,13 @@ module Searchkick
143 164 should: {
144 165 nested: {
145 166 path: conversions_field,
146   - score_mode: "total",
  167 + score_mode: "sum",
147 168 query: {
148 169 function_score: {
149 170 boost_mode: "replace",
150 171 query: {
151 172 match: {
152   - query: term
  173 + "#{conversions_field}.query" => term
153 174 }
154 175 }
155 176 }.merge(script_score)
... ... @@ -169,11 +190,9 @@ module Searchkick
169 190 if boost_by.is_a?(Array)
170 191 boost_by = Hash[boost_by.map { |f| [f, {factor: 1}] }]
171 192 elsif boost_by.is_a?(Hash)
172   - multiply_by, boost_by = boost_by.partition { |k,v| v[:boost_mode] == "multiply" }.map{ |i| Hash[i] }
173   - end
174   - if options[:boost]
175   - boost_by[options[:boost]] = {factor: 1}
  193 + multiply_by, boost_by = boost_by.partition { |_, v| v[:boost_mode] == "multiply" }.map { |i| Hash[i] }
176 194 end
  195 + boost_by[options[:boost]] = {factor: 1} if options[:boost]
177 196  
178 197 custom_filters.concat boost_filters(boost_by, log: true)
179 198 multiply_filters.concat boost_filters(multiply_by || {})
... ... @@ -188,12 +207,10 @@ module Searchkick
188 207 boost_where.each do |field, value|
189 208 if value.is_a?(Array) && value.first.is_a?(Hash)
190 209 value.each do |value_factor|
191   - value, factor = value_factor[:value], value_factor[:factor]
192   - custom_filters << custom_filter(field, value, factor)
  210 + custom_filters << custom_filter(field, value_factor[:value], value_factor[:factor])
193 211 end
194 212 elsif value.is_a?(Hash)
195   - value, factor = value[:value], value[:factor]
196   - custom_filters << custom_filter(field, value, factor)
  213 + custom_filters << custom_filter(field, value[:value], value[:factor])
197 214 else
198 215 factor = 1000
199 216 custom_filters << custom_filter(field, value, factor)
... ... @@ -206,7 +223,7 @@ module Searchkick
206 223 if !boost_by_distance[:field] || !boost_by_distance[:origin]
207 224 raise ArgumentError, "boost_by_distance requires :field and :origin"
208 225 end
209   - function_params = boost_by_distance.select { |k, v| [:origin, :scale, :offset, :decay].include?(k) }
  226 + function_params = boost_by_distance.select { |k, _| [:origin, :scale, :offset, :decay].include?(k) }
210 227 function_params[:origin] = function_params[:origin].reverse
211 228 custom_filters << {
212 229 boost_by_distance[:function] => {
... ... @@ -252,7 +269,7 @@ module Searchkick
252 269 # filters
253 270 filters = where_filters(options[:where])
254 271 if filters.any?
255   - if options[:facets]
  272 + if options[:facets] || options[:aggs]
256 273 payload[:filter] = {
257 274 and: filters
258 275 }
... ... @@ -272,9 +289,7 @@ module Searchkick
272 289 # facets
273 290 if options[:facets]
274 291 facets = options[:facets] || {}
275   - if facets.is_a?(Array) # convert to more advanced syntax
276   - facets = Hash[facets.map { |f| [f, {}] }]
277   - end
  292 + facets = Hash[facets.map { |f| [f, {}] }] if facets.is_a?(Array) # convert to more advanced syntax
278 293  
279 294 payload[:facets] = {}
280 295 facets.each do |field, facet_options|
... ... @@ -310,7 +325,7 @@ module Searchkick
310 325 # offset is not possible
311 326 # http://elasticsearch-users.115913.n3.nabble.com/Is-pagination-possible-in-termsStatsFacet-td3422943.html
312 327  
313   - facet_options.deep_merge!(where: options[:where].reject { |k| k == field }) if options[:smart_facets] == true
  328 + facet_options.deep_merge!(where: options.fetch(:where, {}).reject { |k| k == field }) if options[:smart_facets] == true
314 329 facet_filters = where_filters(facet_options[:where])
315 330 if facet_filters.any?
316 331 payload[:facets][field][:facet_filter] = {
... ... @@ -322,6 +337,41 @@ module Searchkick
322 337 end
323 338 end
324 339  
  340 + # aggregations
  341 + if options[:aggs]
  342 + aggs = options[:aggs]
  343 + payload[:aggs] = {}
  344 +
  345 + aggs = aggs.map { |f| [f, {}] }.to_h if aggs.is_a?(Array) # convert to more advanced syntax
  346 +
  347 + aggs.each do |field, agg_options|
  348 + size = agg_options[:limit] ? agg_options[:limit] : 100_000
  349 +
  350 + payload[:aggs][field] = {
  351 + terms: {
  352 + field: agg_options[:field] || field,
  353 + size: size
  354 + }
  355 + }
  356 +
  357 + agg_options.deep_merge!(where: options.fetch(:where, {}).reject { |k| k == field }) if options[:smart_aggs] == true
  358 + agg_filters = where_filters(agg_options[:where])
  359 +
  360 + if agg_filters.any?
  361 + payload[:aggs][field] = {
  362 + filter: {
  363 + bool: {
  364 + must: agg_filters
  365 + }
  366 + },
  367 + aggs: {
  368 + field => payload[:aggs][field]
  369 + }
  370 + }
  371 + end
  372 + end
  373 + end
  374 +
325 375 # suggestions
326 376 if options[:suggest]
327 377 suggest_fields = (searchkick_options[:suggest] || []).map(&:to_s)
... ... @@ -355,6 +405,10 @@ module Searchkick
355 405 payload[:highlight][:post_tags] = [tag.to_s.gsub(/\A</, "</")]
356 406 end
357 407  
  408 + if (fragment_size = options[:highlight][:fragment_size])
  409 + payload[:highlight][:fragment_size] = fragment_size
  410 + end
  411 +
358 412 highlight_fields = options[:highlight][:fields]
359 413 if highlight_fields
360 414 payload[:highlight][:fields] = {}
... ... @@ -379,9 +433,7 @@ module Searchkick
379 433 end
380 434  
381 435 # routing
382   - if options[:routing]
383   - @routing = options[:routing]
384   - end
  436 + @routing = options[:routing] if options[:routing]
385 437 end
386 438  
387 439 @body = payload
... ... @@ -477,9 +529,7 @@ module Searchkick
477 529 value = {gte: value.first, (value.exclude_end? ? :lt : :lte) => value.last}
478 530 end
479 531  
480   - if value.is_a?(Array)
481   - value = {in: value}
482   - end
  532 + value = {in: value} if value.is_a?(Array)
483 533  
484 534 if value.is_a?(Hash)
485 535 value.each do |op, op_value|
... ... @@ -505,9 +555,11 @@ module Searchkick
505 555 when :regexp # support for regexp queries without using a regexp ruby object
506 556 filters << {regexp: {field => {value: op_value}}}
507 557 when :not # not equal
508   - filters << {not: term_filters(field, op_value)}
  558 + filters << {not: {filter: term_filters(field, op_value)}}
509 559 when :all
510   - filters << {terms: {field => op_value, execution: "and"}}
  560 + op_value.each do |value|
  561 + filters << term_filters(field, value)
  562 + end
511 563 when :in
512 564 filters << term_filters(field, op_value)
513 565 else
... ... @@ -559,7 +611,7 @@ module Searchkick
559 611 def custom_filter(field, value, factor)
560 612 {
561 613 filter: {
562   - and: where_filters({field => value})
  614 + and: where_filters(field => value)
563 615 },
564 616 boost_factor: factor
565 617 }
... ... @@ -595,6 +647,10 @@ module Searchkick
595 647 below_version?("1.4.0")
596 648 end
597 649  
  650 + def below20?
  651 + below_version?("2.0.0")
  652 + end
  653 +
598 654 def below_version?(version)
599 655 Gem::Version.new(Searchkick.server_version) < Gem::Version.new(version)
600 656 end
... ...
lib/searchkick/reindex_job.rb
  Diff comments   View file @12caa1e
1 1 module Searchkick
2 2 class ReindexJob
3   -
4 3 def initialize(klass, id)
5 4 @klass = klass
6 5 @id = id
... ... @@ -23,6 +22,5 @@ module Searchkick
23 22 index.store record
24 23 end
25 24 end
26   -
27 25 end
28 26 end
... ...
lib/searchkick/reindex_v2_job.rb
  Diff comments   View file @12caa1e
... ... @@ -19,6 +19,5 @@ module Searchkick
19 19 index.store record
20 20 end
21 21 end
22   -
23 22 end
24 23 end
... ...
lib/searchkick/results.rb
  Diff comments   View file @12caa1e
... ... @@ -15,27 +15,24 @@ module Searchkick
15 15 @options = options
16 16 end
17 17  
  18 + # experimental: may not make next release
  19 + def records
  20 + @records ||= results_query(klass, hits)
  21 + end
  22 +
18 23 def results
19 24 @results ||= begin
20 25 if options[:load]
21 26 # results can have different types
22 27 results = {}
23 28  
24   - hits.group_by { |hit, i| hit["_type"] }.each do |type, grouped_hits|
25   - records = type.camelize.constantize
26   - if options[:includes]
27   - if defined?(NoBrainer::Document) && records < NoBrainer::Document
28   - records = records.preload(options[:includes])
29   - else
30   - records = records.includes(options[:includes])
31   - end
32   - end
33   - results[type] = results_query(records, grouped_hits)
  29 + hits.group_by { |hit, _| hit["_type"] }.each do |type, grouped_hits|
  30 + results[type] = results_query(type.camelize.constantize, grouped_hits).to_a.index_by { |r| r.id.to_s }
34 31 end
35 32  
36 33 # sort
37 34 hits.map do |hit|
38   - results[hit["_type"]].find { |r| r.id.to_s == hit["_id"].to_s }
  35 + results[hit["_type"]][hit["_id"].to_s]
39 36 end.compact
40 37 else
41 38 hits.map do |hit|
... ... @@ -78,6 +75,23 @@ module Searchkick
78 75 response["facets"]
79 76 end
80 77  
  78 + def aggs
  79 + @aggs ||= begin
  80 + response["aggregations"].dup.each do |field, filtered_agg|
  81 + buckets = filtered_agg[field]
  82 + # move the buckets one level above into the field hash
  83 + if buckets
  84 + filtered_agg.delete(field)
  85 + filtered_agg.merge!(buckets)
  86 + end
  87 + end
  88 + end
  89 + end
  90 +
  91 + def took
  92 + response["took"]
  93 + end
  94 +
81 95 def model_name
82 96 klass.model_name
83 97 end
... ... @@ -141,19 +155,30 @@ module Searchkick
141 155  
142 156 private
143 157  
144   - def results_query(records, grouped_hits)
  158 + def results_query(records, hits)
  159 + ids = hits.map { |hit| hit["_id"] }
  160 +
  161 + if options[:includes]
  162 + records =
  163 + if defined?(NoBrainer::Document) && records < NoBrainer::Document
  164 + records.preload(options[:includes])
  165 + else
  166 + records.includes(options[:includes])
  167 + end
  168 + end
  169 +
145 170 if records.respond_to?(:primary_key) && records.primary_key
146 171 # ActiveRecord
147   - records.where(records.primary_key => grouped_hits.map { |hit| hit["_id"] }).to_a
  172 + records.where(records.primary_key => ids)
148 173 elsif records.respond_to?(:all) && records.all.respond_to?(:for_ids)
149 174 # Mongoid 2
150   - records.all.for_ids(grouped_hits.map { |hit| hit["_id"] }).to_a
  175 + records.all.for_ids(ids)
151 176 elsif records.respond_to?(:queryable)
152 177 # Mongoid 3+
153   - records.queryable.for_ids(grouped_hits.map { |hit| hit["_id"] }).to_a
  178 + records.queryable.for_ids(ids)
154 179 elsif records.respond_to?(:unscoped) && records.all.respond_to?(:preload)
155 180 # Nobrainer
156   - records.unscoped.where(:id.in => grouped_hits.map { |hit| hit["_id"] }).to_a
  181 + records.unscoped.where(:id.in => ids)
157 182 else
158 183 raise "Not sure how to load records"
159 184 end
... ...
lib/searchkick/tasks.rb
  Diff comments   View file @12caa1e
1 1 require "rake"
2 2  
3 3 namespace :searchkick do
4   -
5 4 desc "reindex model"
6 5 task reindex: :environment do
7 6 if ENV["CLASS"]
... ... @@ -31,5 +30,4 @@ namespace :searchkick do
31 30 end
32 31  
33 32 end
34   -
35 33 end
... ...
lib/searchkick/version.rb
  Diff comments   View file @12caa1e
1 1 module Searchkick
2   - VERSION = "0.9.0"
  2 + VERSION = "0.9.1"
3 3 end
... ...
test/aggs_test.rb 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,81 @@
  1 +require_relative "test_helper"
  2 +
  3 +class AggsTest < Minitest::Test
  4 + def setup
  5 + super
  6 + store [
  7 + {name: "Product Show", latitude: 37.7833, longitude: 12.4167, store_id: 1, in_stock: true, color: "blue", price: 21, created_at: 2.days.ago},
  8 + {name: "Product Hide", latitude: 29.4167, longitude: -98.5000, store_id: 2, in_stock: false, color: "green", price: 25, created_at: 2.days.from_now},
  9 + {name: "Product B", latitude: 43.9333, longitude: -122.4667, store_id: 2, in_stock: false, color: "red", price: 5},
  10 + {name: "Foo", latitude: 43.9333, longitude: 12.4667, store_id: 3, in_stock: false, color: "yellow", price: 15}
  11 + ]
  12 + end
  13 +
  14 + def test_basic
  15 + assert_equal ({1 => 1, 2 => 2}), store_agg(aggs: [:store_id])
  16 + end
  17 +
  18 + def test_where
  19 + assert_equal ({1 => 1}), store_agg(aggs: {store_id: {where: {in_stock: true}}})
  20 + end
  21 +
  22 + def test_field
  23 + assert_equal ({1 => 1, 2 => 2}), store_agg(aggs: {store_id: {}})
  24 + assert_equal ({1 => 1, 2 => 2}), store_agg(aggs: {store_id: {field: "store_id"}})
  25 + assert_equal ({1 => 1, 2 => 2}), store_agg({aggs: {store_id_new: {field: "store_id"}}}, "store_id_new")
  26 + end
  27 +
  28 + def test_limit
  29 + agg = Product.search("Product", aggs: {store_id: {limit: 1}}).aggs["store_id"]
  30 + assert_equal 1, agg["buckets"].size
  31 + # assert_equal 3, agg["doc_count"]
  32 + assert_equal(1, agg["sum_other_doc_count"]) if Gem::Version.new(Searchkick.server_version) >= Gem::Version.new("1.4.0")
  33 + end
  34 +
  35 + def test_where_no_smart_aggs
  36 + assert_equal ({2 => 2}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false}}})
  37 + assert_equal ({2 => 2}), store_agg(where: {color: "blue"}, aggs: {store_id: {where: {in_stock: false}}})
  38 + end
  39 +
  40 + def test_smart_aggs
  41 + assert_equal ({1 => 1}), store_agg(where: {in_stock: true}, aggs: [:store_id], smart_aggs: true)
  42 + end
  43 +
  44 + def test_smart_aggs_where
  45 + assert_equal ({2 => 1}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false}}}, smart_aggs: true)
  46 + end
  47 +
  48 + def test_smart_aggs_where_override
  49 + assert_equal ({2 => 1}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false, color: "blue"}}}, smart_aggs: true)
  50 + assert_equal ({}), store_agg(where: {color: "blue"}, aggs: {store_id: {where: {in_stock: false, color: "red"}}}, smart_aggs: true)
  51 + end
  52 +
  53 + def test_smart_aggs_skip_agg
  54 + assert_equal ({1 => 1, 2 => 2}), store_agg(where: {store_id: 2}, aggs: [:store_id], smart_aggs: true)
  55 + end
  56 +
  57 + def test_smart_aggs_skip_agg_complex
  58 + assert_equal ({1 => 1, 2 => 1}), store_agg(where: {store_id: 2, price: {gt: 5}}, aggs: [:store_id], smart_aggs: true)
  59 + end
  60 +
  61 + def test_multiple_aggs
  62 + assert_equal ({"store_id" => {1 => 1, 2 => 2}, "color" => {"blue" => 1, "green" => 1, "red" => 1}}), store_multiple_aggs(aggs: [:store_id, :color])
  63 + end
  64 +
  65 + protected
  66 +
  67 + def buckets_as_hash(agg)
  68 + agg["buckets"].map { |v| [v["key"], v["doc_count"]] }.to_h
  69 + end
  70 +
  71 + def store_agg(options, agg_key = "store_id")
  72 + buckets = Product.search("Product", options).aggs[agg_key]
  73 + buckets_as_hash(buckets)
  74 + end
  75 +
  76 + def store_multiple_aggs(options)
  77 + Product.search("Product", options).aggs.map do |field, filtered_agg|
  78 + [field, buckets_as_hash(filtered_agg)]
  79 + end.to_h
  80 + end
  81 +end
... ...
test/autocomplete_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestAutocomplete < Minitest::Test
4   -
  3 +class AutocompleteTest < Minitest::Test
5 4 def test_autocomplete
6 5 store_names ["Hummus"]
7 6 assert_search "hum", ["Hummus"], autocomplete: true
... ... @@ -63,5 +62,4 @@ class TestAutocomplete &lt; Minitest::Test
63 62 store_names ["hi@example.org"]
64 63 assert_search "hi@example.org", ["hi@example.org"], fields: [{name: :exact}]
65 64 end
66   -
67 65 end
... ...
test/boost_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestBoost < Minitest::Test
4   -
  3 +class BoostTest < Minitest::Test
5 4 # conversions
6 5  
7 6 def test_conversions
... ... @@ -133,5 +132,4 @@ class TestBoost &lt; Minitest::Test
133 132 ]
134 133 assert_order "san", ["San Francisco", "San Antonio", "San Marino"], boost_by_distance: {field: :location, origin: [37, -122], scale: "1000mi"}
135 134 end
136   -
137 135 end
... ...
test/ci/before_install.sh 0 → 100755
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,15 @@
  1 +#!/usr/bin/env bash
  2 +
  3 +sudo apt-get purge elasticsearch
  4 +wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.7.3.deb
  5 +sudo dpkg -i elasticsearch-1.7.3.deb
  6 +sudo service elasticsearch start
  7 +
  8 +if [ -n "$NOBRAINER" ]; then
  9 + source /etc/lsb-release && echo "deb http://download.rethinkdb.com/apt $DISTRIB_CODENAME main" | sudo tee /etc/apt/sources.list.d/rethinkdb.list
  10 + wget -qO- http://download.rethinkdb.com/apt/pubkey.gpg | sudo apt-key add -
  11 + sudo apt-get update -q
  12 + sudo apt-get install rethinkdb
  13 + sudo cp /etc/rethinkdb/default.conf.sample /etc/rethinkdb/instances.d/instance1.conf
  14 + sudo service rethinkdb restart
  15 +fi
... ...
test/facets_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2   -require "active_support/core_ext"
3   -
4   -class TestFacets < Minitest::Test
5 2  
  3 +class FacetsTest < Minitest::Test
6 4 def setup
  5 + skip if elasticsearch2?
7 6 super
8 7 store [
9 8 {name: "Product Show", latitude: 37.7833, longitude: 12.4167, store_id: 1, in_stock: true, color: "blue", price: 21, created_at: 2.days.ago},
... ... @@ -79,14 +78,13 @@ class TestFacets &lt; Minitest::Test
79 78 skip if Gem::Version.new(Searchkick.server_version) >= Gem::Version.new("1.4.0")
80 79 options = {where: {store_id: 2}, facets: {store_id: {stats: true}}}
81 80 facets = Product.search("Product", options).facets["store_id"]["terms"]
82   - expected_facets_keys = %w[term count total_count min max total mean]
  81 + expected_facets_keys = %w(term count total_count min max total mean)
83 82 assert_equal expected_facets_keys, facets.first.keys
84 83 end
85 84  
86 85 protected
87 86  
88   - def store_facet(options, facet_key="store_id")
  87 + def store_facet(options, facet_key = "store_id")
89 88 Hash[Product.search("Product", options).facets[facet_key]["terms"].map { |v| [v["term"], v["count"]] }]
90 89 end
91   -
92 90 end
... ...
test/gemfiles/activerecord31.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,7 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "sqlite3"
  7 +gem "activerecord", "~> 3.1.0"
... ...
test/gemfiles/activerecord32.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,7 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "sqlite3"
  7 +gem "activerecord", "~> 3.2.0"
... ...
test/gemfiles/activerecord40.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,8 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "sqlite3"
  7 +gem "activerecord", "~> 4.0.0"
  8 +gem "activejob_backport"
... ...
test/gemfiles/activerecord41.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,8 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "sqlite3"
  7 +gem "activerecord", "~> 4.1.0"
  8 +gem "activejob_backport"
... ...
test/gemfiles/mongoid2.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,7 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "mongoid", "~> 2"
  7 +gem "bson_ext"
... ...
test/gemfiles/mongoid3.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,6 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "mongoid", "~> 3.1.0"
... ...
test/gemfiles/mongoid4.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,7 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "mongoid", "~> 4.0.0"
  7 +gem "activejob_backport"
... ...
test/gemfiles/nobrainer.gemfile 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,6 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "nobrainer", "0.27.0"
... ...
test/highlight_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestHighlight < Minitest::Test
4   -
  3 +class HighlightTest < Minitest::Test
5 4 def test_basic
6 5 store_names ["Two Door Cinema Club"]
7 6 assert_equal "Two Door <em>Cinema</em> Club", Product.search("cinema", fields: [:name], highlight: true).with_details.first[1][:highlight][:name]
... ... @@ -41,7 +40,7 @@ class TestHighlight &lt; Minitest::Test
41 40 json = {
42 41 query: {
43 42 match: {
44   - _all: "cinema"
  43 + "name.analyzed" => "cinema"
45 44 }
46 45 },
47 46 highlight: {
... ... @@ -54,5 +53,4 @@ class TestHighlight &lt; Minitest::Test
54 53 }
55 54 assert_equal "Two Door <strong>Cinema</strong> Club", Product.search(json: json).with_details.first[1][:highlight][:"name.analyzed"]
56 55 end
57   -
58 56 end
... ...
test/index_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestIndex < Minitest::Test
4   -
  3 +class IndexTest < Minitest::Test
5 4 def test_clean_indices
6 5 old_index = Searchkick::Index.new("products_test_20130801000000000")
7 6 different_index = Searchkick::Index.new("items_test_20130801000000000")
... ... @@ -93,14 +92,14 @@ class TestIndex &lt; Minitest::Test
93 92 end
94 93  
95 94 def test_unsupported_version
96   - raises_exception = ->(s) { raise Elasticsearch::Transport::Transport::Error.new("[500] No query registered for [multi_match]") }
  95 + raises_exception = ->(_) { raise Elasticsearch::Transport::Transport::Error.new("[500] No query registered for [multi_match]") }
97 96 Searchkick.client.stub :search, raises_exception do
98 97 assert_raises(Searchkick::UnsupportedVersionError) { Product.search("test") }
99 98 end
100 99 end
101 100  
102 101 def test_invalid_query
103   - assert_raises(Searchkick::InvalidQueryError) { Product.search(query: {}) }
  102 + assert_raises(Searchkick::InvalidQueryError) { Product.search(query: {boom: true}) }
104 103 end
105 104  
106 105 def test_reindex
... ... @@ -119,5 +118,4 @@ class TestIndex &lt; Minitest::Test
119 118 end
120 119  
121 120 end
122   -
123 121 end
... ...
test/inheritance_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestInheritance < Minitest::Test
4   -
  3 +class InheritanceTest < Minitest::Test
5 4 def test_child_reindex
6 5 store_names ["Max"], Cat
7 6 assert Dog.reindex
... ... @@ -76,5 +75,4 @@ class TestInheritance &lt; Minitest::Test
76 75 store_names ["Product B"], Animal
77 76 assert_search "product", ["Product A", "Product B"], index_name: [Product.searchkick_index.name, Animal.searchkick_index.name], conversions: false
78 77 end
79   -
80 78 end
... ...
test/match_test.rb
  Diff comments   View file @12caa1e
... ... @@ -2,8 +2,7 @@
2 2  
3 3 require_relative "test_helper"
4 4  
5   -class TestMatch < Minitest::Test
6   -
  5 +class MatchTest < Minitest::Test
7 6 # exact
8 7  
9 8 def test_match
... ... @@ -195,5 +194,4 @@ class TestMatch &lt; Minitest::Test
195 194 ]
196 195 assert_search "almond", []
197 196 end
198   -
199 197 end
... ...
test/model_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestModel < Minitest::Test
4   -
  3 +class ModelTest < Minitest::Test
5 4 def test_disable_callbacks_model
6 5 store_names ["product a"]
7 6  
... ... @@ -20,7 +19,7 @@ class TestModel &lt; Minitest::Test
20 19 def test_disable_callbacks_global
21 20 # make sure callbacks default to on
22 21 assert Searchkick.callbacks?
23   -
  22 +
24 23 store_names ["product a"]
25 24  
26 25 Searchkick.disable_callbacks
... ... @@ -34,5 +33,4 @@ class TestModel &lt; Minitest::Test
34 33  
35 34 assert_search "product", ["product a", "product b"]
36 35 end
37   -
38 36 end
... ...
test/query_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestQuery < Minitest::Test
4   -
  3 +class QueryTest < Minitest::Test
5 4 def test_basic
6 5 store_names ["Milk", "Apple"]
7 6 query = Product.search("milk", execute: false)
... ... @@ -10,5 +9,4 @@ class TestQuery &lt; Minitest::Test
10 9 query.body[:query] = {match_all: {}}
11 10 assert_equal ["Apple", "Milk"], query.execute.map(&:name).sort
12 11 end
13   -
14 12 end
... ...
test/records_test.rb 0 → 100644
  Diff comments   View file @12caa1e
... ... @@ -0,0 +1,8 @@
  1 +require_relative "test_helper"
  2 +
  3 +class RecordsTest < Minitest::Test
  4 + def test_records
  5 + store_names ["Milk", "Apple"]
  6 + assert_equal Product.search("milk").records.where(name: "Milk").count, 1
  7 + end
  8 +end
... ...
test/reindex_job_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestReindexJob < Minitest::Test
4   -
  3 +class ReindexJobTest < Minitest::Test
5 4 def setup
6 5 super
7 6 Searchkick.disable_callbacks
... ... @@ -29,5 +28,4 @@ class TestReindexJob &lt; Minitest::Test
29 28 Product.searchkick_index.refresh
30 29 assert_search "*", []
31 30 end
32   -
33 31 end
... ...
test/reindex_v2_job_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestReindexV2Job < Minitest::Test
4   -
  3 +class ReindexV2JobTest < Minitest::Test
5 4 def setup
6 5 skip unless defined?(ActiveJob)
7 6 super
... ... @@ -30,5 +29,4 @@ class TestReindexV2Job &lt; Minitest::Test
30 29 Product.searchkick_index.refresh
31 30 assert_search "*", []
32 31 end
33   -
34 32 end
... ...
test/routing_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestRouting < Minitest::Test
4   -
  3 +class RoutingTest < Minitest::Test
5 4 def test_routing_query
  5 + skip if elasticsearch2?
6 6 query = Store.search("Dollar Tree", routing: "Dollar Tree", execute: false)
7 7 assert_equal query.params[:routing], "Dollar Tree"
8 8 end
9 9  
10 10 def test_routing_mappings
  11 + skip if elasticsearch2?
11 12 index_options = Store.searchkick_index.index_options
12   - assert_equal index_options[:mappings][:_default_][:_routing], {required: true, path: "name"}
  13 + assert_equal index_options[:mappings][:_default_][:_routing], required: true, path: "name"
13 14 end
14 15 end
... ...
test/should_index_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestShouldIndex < Minitest::Test
4   -
  3 +class ShouldIndexTest < Minitest::Test
5 4 def test_basic
6 5 store_names ["INDEX", "DO NOT INDEX"]
7 6 assert_search "index", ["INDEX"]
... ... @@ -30,5 +29,4 @@ class TestShouldIndex &lt; Minitest::Test
30 29 Product.searchkick_index.refresh
31 30 assert_search "index", []
32 31 end
33   -
34 32 end
... ...
test/similar_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestSimilar < Minitest::Test
4   -
  3 +class SimilarTest < Minitest::Test
5 4 def test_similar
6 5 store_names ["Annie's Naturals Organic Shiitake & Sesame Dressing"]
7 6 assert_search "Annie's Naturals Shiitake & Sesame Vinaigrette", ["Annie's Naturals Organic Shiitake & Sesame Dressing"], similar: true
... ... @@ -16,5 +15,4 @@ class TestSimilar &lt; Minitest::Test
16 15 store_names ["Lucerne Milk Chocolate Fat Free", "Clover Fat Free Milk"]
17 16 assert_order "Lucerne Fat Free Chocolate Milk", ["Lucerne Milk Chocolate Fat Free", "Clover Fat Free Milk"], similar: true
18 17 end
19   -
20 18 end
... ...
test/sql_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestSql < Minitest::Test
4   -
  3 +class SqlTest < Minitest::Test
5 4 def test_limit
6 5 store_names ["Product A", "Product B", "Product C", "Product D"]
7 6 assert_order "product", ["Product A", "Product B"], order: {name: :asc}, limit: 2
... ... @@ -139,11 +138,11 @@ class TestSql &lt; Minitest::Test
139 138 # https://gist.github.com/jprante/7099463
140 139 def test_where_range_array
141 140 store [
142   - {name: "Product A", user_ids: [11, 23, 13, 16, 17, 23.6]},
143   - {name: "Product B", user_ids: [1, 2, 3, 4, 5, 6, 7, 8, 8.9, 9.1, 9.4]},
  141 + {name: "Product A", user_ids: [11, 23, 13, 16, 17, 23]},
  142 + {name: "Product B", user_ids: [1, 2, 3, 4, 5, 6, 7, 8, 9]},
144 143 {name: "Product C", user_ids: [101, 230, 150, 200]}
145 144 ]
146   - assert_search "product", ["Product A"], where: {user_ids: {gt: 10, lt: 23.9}}
  145 + assert_search "product", ["Product A"], where: {user_ids: {gt: 10, lt: 24}}
147 146 end
148 147  
149 148 def test_where_range_array_again
... ... @@ -244,6 +243,40 @@ class TestSql &lt; Minitest::Test
244 243 assert_search "aaaa", ["aabb"], misspellings: {distance: 2}
245 244 end
246 245  
  246 + def test_misspellings_prefix_length
  247 + store_names ["ap", "api", "apt", "any", "nap", "ah", "ahi"]
  248 + assert_search "ap", ["ap"], misspellings: {prefix_length: 2}
  249 + assert_search "api", ["ap", "api", "apt"], misspellings: {prefix_length: 2}
  250 + end
  251 +
  252 + def test_misspellings_prefix_length_operator
  253 + store_names ["ap", "api", "apt", "any", "nap", "ah", "aha"]
  254 + assert_search "ap ah", ["ap", "ah"], operator: "or", misspellings: {prefix_length: 2}
  255 + assert_search "api ahi", ["ap", "api", "apt", "ah", "aha"], operator: "or", misspellings: {prefix_length: 2}
  256 + end
  257 +
  258 + def test_misspellings_fields_operator
  259 + store [
  260 + {name: "red", color: "red"},
  261 + {name: "blue", color: "blue"},
  262 + {name: "cyan", color: "blue green"},
  263 + {name: "magenta", color: "red blue"},
  264 + {name: "green", color: "green"}
  265 + ]
  266 + assert_search "red blue", ["red", "blue", "cyan", "magenta"], operator: "or", fields: ["color"], misspellings: false
  267 + end
  268 +
  269 + def test_fields_operator
  270 + store [
  271 + {name: "red", color: "red"},
  272 + {name: "blue", color: "blue"},
  273 + {name: "cyan", color: "blue green"},
  274 + {name: "magenta", color: "red blue"},
  275 + {name: "green", color: "green"}
  276 + ]
  277 + assert_search "red blue", ["red", "blue", "cyan", "magenta"], operator: "or", fields: ["color"]
  278 + end
  279 +
247 280 def test_fields
248 281 store [
249 282 {name: "red", color: "light blue"},
... ... @@ -267,9 +300,9 @@ class TestSql &lt; Minitest::Test
267 300  
268 301 def test_big_decimal
269 302 store [
270   - {name: "Product", latitude: 100.0}
  303 + {name: "Product", latitude: 80.0}
271 304 ]
272   - assert_search "product", ["Product"], where: {latitude: {gt: 99}}
  305 + assert_search "product", ["Product"], where: {latitude: {gt: 79}}
273 306 end
274 307  
275 308 # load
... ... @@ -299,7 +332,7 @@ class TestSql &lt; Minitest::Test
299 332 def test_select
300 333 store [{name: "Product A", store_id: 1}]
301 334 result = Product.search("product", load: false, select: [:name, :store_id]).first
302   - assert_equal %w[id name store_id], result.keys.reject { |k| k.start_with?("_") }.sort
  335 + assert_equal %w(id name store_id), result.keys.reject { |k| k.start_with?("_") }.sort
303 336 assert_equal ["Product A"], result.name # this is not great
304 337 end
305 338  
... ... @@ -329,5 +362,4 @@ class TestSql &lt; Minitest::Test
329 362 assert Product.search("product", include: [:store]).first.association(:store).loaded?
330 363 end
331 364 end
332   -
333 365 end
... ...
test/suggest_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestSuggest < Minitest::Test
4   -
  3 +class SuggestTest < Minitest::Test
5 4 def test_basic
6 5 store_names ["Great White Shark", "Hammerhead Shark", "Tiger Shark"]
7 6 assert_suggest "How Big is a Tigre Shar", "how big is a tiger shark", fields: [:name]
... ... @@ -78,5 +77,4 @@ class TestSuggest &lt; Minitest::Test
78 77 def assert_suggest_all(term, expected, options = {})
79 78 assert_equal expected.sort, Product.search(term, options.merge(suggest: true)).suggestions.sort
80 79 end
81   -
82 80 end
... ...
test/synonyms_test.rb
  Diff comments   View file @12caa1e
1 1 require_relative "test_helper"
2 2  
3   -class TestSynonyms < Minitest::Test
4   -
  3 +class SynonymsTest < Minitest::Test
5 4 def test_bleach
6 5 store_names ["Clorox Bleach", "Kroger Bleach"]
7 6 assert_search "clorox", ["Clorox Bleach", "Kroger Bleach"]
... ... @@ -46,5 +45,4 @@ class TestSynonyms &lt; Minitest::Test
46 45 # store_names ["Creature", "Beast", "Dragon"], Animal
47 46 # assert_search "animal", ["Creature", "Beast"], {}, Animal
48 47 # end
49   -
50 48 end
... ...
test/test_helper.rb
  Diff comments   View file @12caa1e
... ... @@ -3,6 +3,7 @@ Bundler.require(:default)
3 3 require "minitest/autorun"
4 4 require "minitest/pride"
5 5 require "logger"
  6 +require "active_support/core_ext" if defined?(NoBrainer)
6 7  
7 8 ENV["RACK_ENV"] = "test"
8 9  
... ... @@ -17,6 +18,10 @@ I18n.config.enforce_available_locales = true
17 18  
18 19 ActiveJob::Base.logger = nil if defined?(ActiveJob)
19 20  
  21 +def elasticsearch2?
  22 + Searchkick.server_version.starts_with?("2.")
  23 +end
  24 +
20 25 if defined?(Mongoid)
21 26  
22 27 def mongoid2?
... ... @@ -96,7 +101,7 @@ elsif defined?(NoBrainer)
96 101 field :color, type: String
97 102 field :latitude
98 103 field :longitude
99   - field :description, type: String
  104 + field :description, type: String
100 105  
101 106 belongs_to :store, validates: false
102 107 end
... ... @@ -220,15 +225,15 @@ end
220 225  
221 226 class Store
222 227 searchkick \
223   - routing: :name,
  228 + routing: elasticsearch2? ? false : "name",
224 229 merge_mappings: true,
225 230 mappings: {
226 231 store: {
227 232 properties: {
228   - name: {type: "string", analyzer: "keyword"},
  233 + name: {type: "string", analyzer: "keyword"}
229 234 }
230 235 }
231   - }
  236 + }
232 237 end
233 238  
234 239 class Animal
... ... @@ -247,7 +252,6 @@ Store.reindex
247 252 Animal.reindex
248 253  
249 254 class Minitest::Test
250   -
251 255 def setup
252 256 Product.destroy_all
253 257 Store.destroy_all
... ... @@ -279,5 +283,4 @@ class Minitest::Test
279 283 def assert_first(term, expected, options = {}, klass = Product)
280 284 assert_equal expected, klass.search(term, options).map(&:name).first
281 285 end
282   -
283 286 end
... ...