github / searchkick

Download as
Browse Code »

Commit 29ab35b496ce3ec09572a600397de945492cceb6

Authored by Oskar Szrajer
2 parents 1569a146 71dac198
Exists in master and in 21 other branches allow_missing, conversions_period, conversions_v2, eager_load, find_in_batches, highlighted, index_name, lazy_index_name, less_deps, nested, reindex_async_threads, reindex_refactor, relation, relation_search, routing_queue, scope_attributes, search_as_you_type, synonyms_ci, test_language, threads, wildcard

Merge remote-tracking branch 'upstream/master' into adv_facets

Showing 18 changed files with 465 additions and 194 deletions   Show diff stats
CHANGELOG.md
  Diff comments   View file @29ab35b
  1 +## 0.7.1
  2 +
  3 +- Fixed huge issue w/ zero-downtime reindexing on 0.90
  4 +
  5 +## 0.7.0
  6 +
  7 +- Added support for Elasticsearch 1.1
  8 +- Dropped support for Elasticsearch below 0.90.4 (unfortunate side effect of above)
  9 +
  10 +## 0.6.3
  11 +
  12 +- Removed patron since no support for Windows
  13 +- Added error if `searchkick` is called multiple times
  14 +
  15 +## 0.6.2
  16 +
  17 +- Added logging
  18 +- Fixed index_name option
  19 +- Added ability to use proc as the index name
  20 +
  21 +## 0.6.1
  22 +
  23 +- Fixed huge issue w/ zero-downtime reindexing on 0.90 and elasticsearch-ruby 1.0
  24 +- Restore load: false behavior
  25 +- Restore total_entries method
  26 +
  27 +## 0.6.0
  28 +
  29 +- Moved to elasticsearch-ruby
  30 +- Added support for modifying the query and viewing the response
  31 +- Added support for page_entries_info method
  32 +
1 33 ## 0.5.3
2 34  
3 35 - Fixed bug w/ word_* queries
... ...
README.md
  Diff comments   View file @29ab35b
... ... @@ -147,7 +147,7 @@ Product.search "fresh honey" # fresh AND honey
147 147 To change this, use:
148 148  
149 149 ```ruby
150   -Product.search "fresh honey", partial: true # fresh OR honey
  150 +Product.search "fresh honey", operator: "or" # fresh OR honey
151 151 ```
152 152  
153 153 By default, results must match the entire word - `back` will not match `backpack`. You can change this behavior with:
... ... @@ -504,7 +504,7 @@ And to search, use:
504 504 ```ruby
505 505 Animal.search "*" # all animals
506 506 Dog.search "*" # just dogs
507   -Animal.search "*", type: [Dog, Cat] # just cats and dogs [master]
  507 +Animal.search "*", type: [Dog, Cat] # just cats and dogs
508 508 ```
509 509  
510 510 **Note:** The `suggest` option retrieves suggestions from the parent at the moment.
... ... @@ -576,7 +576,13 @@ end
576 576 And use the `query` option to search:
577 577  
578 578 ```ruby
579   -Product.search query: {match: {name: "milk"}}
  579 +products = Product.search query: {match: {name: "milk"}}
  580 +```
  581 +
  582 +View the response with:
  583 +
  584 +```ruby
  585 +products.response
580 586 ```
581 587  
582 588 To keep the mappings and settings generated by Searchkick, use:
... ... @@ -587,9 +593,7 @@ class Product < ActiveRecord::Base
587 593 end
588 594 ```
589 595  
590   -## Experimental [master]
591   -
592   -Modify the query generated by Searchkick.
  596 +To modify the query generated by Searchkick, use:
593 597  
594 598 ```ruby
595 599 query = Product.search "2% Milk", execute: false
... ... @@ -597,15 +601,9 @@ query.body[:query] = {match_all: {}}
597 601 products = query.execute
598 602 ```
599 603  
600   -View the response with:
601   -
602   -```ruby
603   -products.response
604   -```
605   -
606 604 ## Reference
607 605  
608   -Searchkick requires Elasticsearch `0.90.0` or higher.
  606 +Searchkick requires Elasticsearch `0.90.4` or higher.
609 607  
610 608 Reindex one record
611 609  
... ... @@ -754,6 +752,10 @@ rake searchkick:reindex:all
754 752  
755 753 4. Once it finishes, replace search calls w/ searchkick calls
756 754  
  755 +## Note about 0.6.0 and 0.7.0
  756 +
  757 +If running Searchkick `0.6.0` or `0.7.0` and Elasticsearch `0.90`, we recommend upgrading to Searchkick `0.6.1` or `0.7.1` to fix an issue that causes downtime when reindexing.
  758 +
757 759 ## Elasticsearch Gotchas
758 760  
759 761 ### Inconsistent Scores
... ... @@ -770,7 +772,7 @@ For convenience, this is set by default in the test environment.
770 772  
771 773 ## Thanks
772 774  
773   -Thanks to Karel Minarik for [Tire](https://github.com/karmi/tire), Jaroslav Kalistsuk for [zero downtime reindexing](https://gist.github.com/jarosan/3124884), and Alex Leschenko for [Elasticsearch autocomplete](https://github.com/leschenko/elasticsearch_autocomplete).
  775 +Thanks to Karel Minarik for [Elasticsearch Ruby](https://github.com/elasticsearch/elasticsearch-ruby) and [Tire](https://github.com/karmi/tire), Jaroslav Kalistsuk for [zero downtime reindexing](https://gist.github.com/jarosan/3124884), and Alex Leschenko for [Elasticsearch autocomplete](https://github.com/leschenko/elasticsearch_autocomplete).
774 776  
775 777 ## Roadmap
776 778  
... ... @@ -797,7 +799,5 @@ Everyone is encouraged to help improve this project. Here are a few ways you can
797 799 To get started with development and testing:
798 800  
799 801 1. Clone the repo
800   -2. Install PostgreSQL and create a database called `searchkick_test` (`psql -d postgres -c "create database searchkick_test"`)
801   -3. Install Elasticsearch
802   -4. `bundle`
803   -5. `rake test`
  802 +2. `bundle`
  803 +3. `rake test`
... ...
gemfiles/mongoid4.gemfile
  Diff comments   View file @29ab35b
... ... @@ -3,4 +3,4 @@ source 'https://rubygems.org'
3 3 # Specify your gem's dependencies in searchkick.gemspec
4 4 gemspec path: "../"
5 5  
6   -gem "mongoid", github: "mongoid/mongoid"
  6 +gem "mongoid", "4.0.0.beta1"
... ...
lib/searchkick.rb
  Diff comments   View file @29ab35b
1   -require "tire"
  1 +require "active_model"
  2 +require "elasticsearch"
  3 +require "hashie"
2 4 require "searchkick/version"
  5 +require "searchkick/index"
3 6 require "searchkick/reindex"
4 7 require "searchkick/results"
5 8 require "searchkick/query"
... ... @@ -7,9 +10,14 @@ require "searchkick/search"
7 10 require "searchkick/similar"
8 11 require "searchkick/model"
9 12 require "searchkick/tasks"
10   -require "searchkick/logger" if defined?(Rails)
  13 +require "searchkick/logging" if defined?(Rails)
11 14  
12 15 module Searchkick
  16 +
  17 + def self.client
  18 + @client ||= Elasticsearch::Client.new(url: ENV["ELASTICSEARCH_URL"])
  19 + end
  20 +
13 21 @callbacks = true
14 22  
15 23 def self.enable_callbacks
... ...
lib/searchkick/index.rb 0 → 100644
  Diff comments   View file @29ab35b
... ... @@ -0,0 +1,137 @@
  1 +module Searchkick
  2 + class Index
  3 + attr_reader :name
  4 +
  5 + def initialize(name)
  6 + @name = name
  7 + end
  8 +
  9 + def create(options = {})
  10 + client.indices.create index: name, body: options
  11 + end
  12 +
  13 + def delete
  14 + client.indices.delete index: name
  15 + end
  16 +
  17 + def exists?
  18 + client.indices.exists index: name
  19 + end
  20 +
  21 + def refresh
  22 + client.indices.refresh index: name
  23 + end
  24 +
  25 + def store(record)
  26 + client.index(
  27 + index: name,
  28 + type: document_type(record),
  29 + id: record.id,
  30 + body: search_data(record)
  31 + )
  32 + end
  33 +
  34 + def remove(record)
  35 + client.delete(
  36 + index: name,
  37 + type: document_type(record),
  38 + id: record.id
  39 + )
  40 + end
  41 +
  42 + def import(records)
  43 + if records.any?
  44 + client.bulk(
  45 + index: name,
  46 + type: document_type(records.first),
  47 + body: records.map{|r| data = search_data(r); {index: {_id: data["_id"] || data["id"] || r.id, data: data}} }
  48 + )
  49 + end
  50 + end
  51 +
  52 + def retrieve(record)
  53 + client.get(
  54 + index: name,
  55 + type: document_type(record),
  56 + id: record.id
  57 + )["_source"]
  58 + end
  59 +
  60 + def klass_document_type(klass)
  61 + klass.model_name.to_s.underscore
  62 + end
  63 +
  64 + protected
  65 +
  66 + def client
  67 + Searchkick.client
  68 + end
  69 +
  70 + def document_type(record)
  71 + klass_document_type(record.class)
  72 + end
  73 +
  74 + def search_data(record)
  75 + source = record.search_data
  76 +
  77 + # stringify fields
  78 + source = source.inject({}){|memo,(k,v)| memo[k.to_s] = v; memo}
  79 +
  80 + # Mongoid 4 hack
  81 + if defined?(BSON::ObjectId) and source["_id"].is_a?(BSON::ObjectId)
  82 + source["_id"] = source["_id"].to_s
  83 + end
  84 +
  85 + options = record.class.searchkick_options
  86 +
  87 + # conversions
  88 + conversions_field = options[:conversions]
  89 + if conversions_field and source[conversions_field]
  90 + source[conversions_field] = source[conversions_field].map{|k, v| {query: k, count: v} }
  91 + end
  92 +
  93 + # hack to prevent generator field doesn't exist error
  94 + (options[:suggest] || []).map(&:to_s).each do |field|
  95 + source[field] = nil if !source[field]
  96 + end
  97 +
  98 + # locations
  99 + (options[:locations] || []).map(&:to_s).each do |field|
  100 + if source[field]
  101 + if source[field].first.is_a?(Array) # array of arrays
  102 + source[field] = source[field].map{|a| a.map(&:to_f).reverse }
  103 + else
  104 + source[field] = source[field].map(&:to_f).reverse
  105 + end
  106 + end
  107 + end
  108 +
  109 + cast_big_decimal(source)
  110 +
  111 + # p search_data
  112 +
  113 + source.as_json
  114 + end
  115 +
  116 + # change all BigDecimal values to floats due to
  117 + # https://github.com/rails/rails/issues/6033
  118 + # possible loss of precision :/
  119 + def cast_big_decimal(obj)
  120 + case obj
  121 + when BigDecimal
  122 + obj.to_f
  123 + when Hash
  124 + obj.each do |k, v|
  125 + obj[k] = cast_big_decimal(v)
  126 + end
  127 + when Enumerable
  128 + obj.map do |v|
  129 + cast_big_decimal(v)
  130 + end
  131 + else
  132 + obj
  133 + end
  134 + end
  135 +
  136 + end
  137 +end
... ...
lib/searchkick/logger.rb
View file @1569a14
... ... @@ -1,19 +0,0 @@
1   -require "tire/rails/logger"
2   -require "tire/rails/logger/log_subscriber"
3   -
4   -class Tire::Rails::LogSubscriber
5   -
6   - # better output format
7   - def search(event)
8   - self.class.runtime += event.duration
9   - return unless logger.debug?
10   -
11   - payload = event.payload
12   -
13   - name = "%s (%.1fms)" % [payload[:name], event.duration]
14   - query = payload[:search].to_s
15   -
16   - debug " #{color(name, YELLOW, true)} #{query}"
17   - end
18   -
19   -end
lib/searchkick/logging.rb 0 → 100644
  Diff comments   View file @29ab35b
... ... @@ -0,0 +1,89 @@
  1 +# based on https://gist.github.com/mnutt/566725
  2 +
  3 +module Searchkick
  4 + class Query
  5 + def execute_with_instrumentation
  6 + event = {
  7 + name: "#{searchkick_klass.name} Search",
  8 + query: params
  9 + }
  10 + ActiveSupport::Notifications.instrument("search.searchkick", event) do
  11 + execute_without_instrumentation
  12 + end
  13 + end
  14 +
  15 + alias_method_chain :execute, :instrumentation
  16 + end
  17 +
  18 + # https://github.com/rails/rails/blob/master/activerecord/lib/active_record/log_subscriber.rb
  19 + class LogSubscriber < ActiveSupport::LogSubscriber
  20 + def self.runtime=(value)
  21 + Thread.current[:searchkick_runtime] = value
  22 + end
  23 +
  24 + def self.runtime
  25 + Thread.current[:searchkick_runtime] ||= 0
  26 + end
  27 +
  28 + def self.reset_runtime
  29 + rt, self.runtime = runtime, 0
  30 + rt
  31 + end
  32 +
  33 + def search(event)
  34 + self.class.runtime += event.duration
  35 + return unless logger.debug?
  36 +
  37 + payload = event.payload
  38 + name = "#{payload[:name]} (#{event.duration.round(1)}ms)"
  39 + type = payload[:query][:type]
  40 +
  41 + # no easy way to tell which host the client will use
  42 + host = Searchkick.client.transport.hosts.first
  43 + debug " #{color(name, YELLOW, true)} curl #{host[:protocol]}://#{host[:host]}:#{host[:port]}/#{CGI.escape(payload[:query][:index])}#{type ? "/#{type.map{|t| CGI.escape(t) }.join(",")}" : ""}/_search?pretty -d '#{payload[:query][:body].to_json}'"
  44 + end
  45 + end
  46 +
  47 + # https://github.com/rails/rails/blob/master/activerecord/lib/active_record/railties/controller_runtime.rb
  48 + module ControllerRuntime
  49 + extend ActiveSupport::Concern
  50 +
  51 + protected
  52 +
  53 + attr_internal :searchkick_runtime
  54 +
  55 + def process_action(action, *args)
  56 + # We also need to reset the runtime before each action
  57 + # because of queries in middleware or in cases we are streaming
  58 + # and it won't be cleaned up by the method below.
  59 + Searchkick::LogSubscriber.reset_runtime
  60 + super
  61 + end
  62 +
  63 + def cleanup_view_runtime
  64 + searchkick_rt_before_render = Searchkick::LogSubscriber.reset_runtime
  65 + runtime = super
  66 + searchkick_rt_after_render = Searchkick::LogSubscriber.reset_runtime
  67 + self.searchkick_runtime = searchkick_rt_before_render + searchkick_rt_after_render
  68 + runtime - searchkick_rt_after_render
  69 + end
  70 +
  71 + def append_info_to_payload(payload)
  72 + super
  73 + payload[:searchkick_runtime] = (searchkick_runtime || 0) + Searchkick::LogSubscriber.reset_runtime
  74 + end
  75 +
  76 + module ClassMethods
  77 + def log_process_action(payload)
  78 + messages, runtime = super, payload[:searchkick_runtime]
  79 + messages << ("Searchkick: %.1fms" % runtime.to_f) if runtime.to_f > 0
  80 + messages
  81 + end
  82 + end
  83 + end
  84 +end
  85 +
  86 +Searchkick::LogSubscriber.attach_to :searchkick
  87 +ActiveSupport.on_load(:action_controller) do
  88 + include Searchkick::ControllerRuntime
  89 +end
... ...
lib/searchkick/model.rb
  Diff comments   View file @29ab35b
... ... @@ -2,18 +2,22 @@ module Searchkick
2 2 module Model
3 3  
4 4 def searchkick(options = {})
  5 + raise "Only call searchkick once per model" if respond_to?(:searchkick_index)
  6 +
5 7 class_eval do
6   - cattr_reader :searchkick_options, :searchkick_env, :searchkick_klass, :searchkick_index
  8 + cattr_reader :searchkick_options, :searchkick_env, :searchkick_klass
7 9  
8 10 class_variable_set :@@searchkick_options, options.dup
9 11 class_variable_set :@@searchkick_env, ENV["RACK_ENV"] || ENV["RAILS_ENV"] || "development"
10 12 class_variable_set :@@searchkick_klass, self
11 13 class_variable_set :@@searchkick_callbacks, options[:callbacks] != false
  14 + class_variable_set :@@searchkick_index, options[:index_name] || [options[:index_prefix], model_name.plural, searchkick_env].compact.join("_")
12 15  
13   - # set index name
14   - # TODO support proc
15   - index_name = options[:index_name] || [options[:index_prefix], model_name.plural, searchkick_env].compact.join("_")
16   - class_variable_set :@@searchkick_index, Tire::Index.new(index_name)
  16 + def self.searchkick_index
  17 + index = class_variable_get :@@searchkick_index
  18 + index = index.call if index.respond_to? :call
  19 + Searchkick::Index.new(index)
  20 + end
17 21  
18 22 extend Searchkick::Search
19 23 extend Searchkick::Reindex
... ... @@ -45,7 +49,11 @@ module Searchkick
45 49 def reindex
46 50 index = self.class.searchkick_index
47 51 if destroyed? or !should_index?
48   - index.remove self
  52 + begin
  53 + index.remove self
  54 + rescue Elasticsearch::Transport::Transport::Errors::NotFound
  55 + # do nothing
  56 + end
49 57 else
50 58 index.store self
51 59 end
... ... @@ -55,79 +63,6 @@ module Searchkick
55 63 respond_to?(:to_hash) ? to_hash : serializable_hash
56 64 end
57 65  
58   - def to_indexed_json
59   - source = search_data
60   -
61   - # stringify fields
62   - source = source.inject({}){|memo,(k,v)| memo[k.to_s] = v; memo}
63   -
64   - # Mongoid 4 hack
65   - if defined?(BSON::ObjectId) and source["_id"].is_a?(BSON::ObjectId)
66   - source["_id"] = source["_id"].to_s
67   - end
68   -
69   - options = self.class.searchkick_options
70   -
71   - # conversions
72   - conversions_field = options[:conversions]
73   - if conversions_field and source[conversions_field]
74   - source[conversions_field] = source[conversions_field].map{|k, v| {query: k, count: v} }
75   - end
76   -
77   - # hack to prevent generator field doesn't exist error
78   - (options[:suggest] || []).map(&:to_s).each do |field|
79   - source[field] = nil if !source[field]
80   - end
81   -
82   - # locations
83   - (options[:locations] || []).map(&:to_s).each do |field|
84   - if source[field]
85   - if source[field].first.is_a?(Array) # array of arrays
86   - source[field] = source[field].map{|a| a.map(&:to_f).reverse }
87   - else
88   - source[field] = source[field].map(&:to_f).reverse
89   - end
90   - end
91   - end
92   -
93   - # change all BigDecimal values to floats due to
94   - # https://github.com/rails/rails/issues/6033
95   - # possible loss of precision :/
96   - cast_big_decimal =
97   - proc do |obj|
98   - case obj
99   - when BigDecimal
100   - obj.to_f
101   - when Hash
102   - obj.each do |k, v|
103   - obj[k] = cast_big_decimal.call(v)
104   - end
105   - when Enumerable
106   - obj.map! do |v|
107   - cast_big_decimal.call(v)
108   - end
109   - else
110   - obj
111   - end
112   - end
113   -
114   - cast_big_decimal.call(source)
115   -
116   - # p search_data
117   -
118   - source.to_json
119   - end
120   -
121   - # TODO remove
122   -
123   - def self.document_type
124   - model_name.to_s.underscore
125   - end
126   -
127   - def document_type
128   - self.class.document_type
129   - end
130   -
131 66 end
132 67 end
133 68  
... ...
lib/searchkick/query.rb
  Diff comments   View file @29ab35b
... ... @@ -35,15 +35,10 @@ module Searchkick
35 35  
36 36 operator = options[:operator] || (options[:partial] ? "or" : "and")
37 37  
38   - # model and eagar loading
39   - load = options[:load].nil? ? true : options[:load]
40   - load = (options[:include] ? {include: options[:include]} : true) if load
41   -
42 38 # pagination
43 39 page = [options[:page].to_i, 1].max
44 40 per_page = (options[:limit] || options[:per_page] || 100000).to_i
45 41 offset = options[:offset] || (page - 1) * per_page
46   - index_name = options[:index_name] || searchkick_index.name
47 42  
48 43 conversions_field = searchkick_options[:conversions]
49 44 personalize_field = searchkick_options[:personalize]
... ... @@ -83,9 +78,9 @@ module Searchkick
83 78 fields: [field],
84 79 query: term,
85 80 use_dis_max: false,
86   - operator: operator,
87   - cutoff_frequency: 0.001
  81 + operator: operator
88 82 }
  83 + shared_options[:cutoff_frequency] = 0.001 unless operator == "and"
89 84 queries.concat [
90 85 {multi_match: shared_options.merge(boost: 10, analyzer: "searchkick_search")},
91 86 {multi_match: shared_options.merge(boost: 10, analyzer: "searchkick_search2")}
... ... @@ -126,13 +121,16 @@ module Searchkick
126 121 path: conversions_field,
127 122 score_mode: "total",
128 123 query: {
129   - custom_score: {
  124 + function_score: {
  125 + boost_mode: "replace",
130 126 query: {
131 127 match: {
132 128 query: term
133 129 }
134 130 },
135   - script: "doc['count'].value"
  131 + script_score: {
  132 + script: "doc['count'].value"
  133 + }
136 134 }
137 135 }
138 136 }
... ... @@ -151,7 +149,9 @@ module Searchkick
151 149 field: options[:boost]
152 150 }
153 151 },
154   - script: "log(doc['#{options[:boost]}'].value + 2.718281828)"
  152 + script_score: {
  153 + script: "log(doc['#{options[:boost]}'].value + 2.718281828)"
  154 + }
155 155 }
156 156 end
157 157  
... ... @@ -162,7 +162,7 @@ module Searchkick
162 162 personalize_field => options[:user_id]
163 163 }
164 164 },
165   - boost: 100
  165 + boost_factor: 100
166 166 }
167 167 end
168 168  
... ... @@ -171,16 +171,16 @@ module Searchkick
171 171 filter: {
172 172 term: options[:personalize]
173 173 },
174   - boost: 100
  174 + boost_factor: 100
175 175 }
176 176 end
177 177  
178 178 if custom_filters.any?
179 179 payload = {
180   - custom_filters_score: {
  180 + function_score: {
  181 + functions: custom_filters,
181 182 query: payload,
182   - filters: custom_filters,
183   - score_mode: "total"
  183 + score_mode: "sum"
184 184 }
185 185 }
186 186 end
... ... @@ -279,18 +279,22 @@ module Searchkick
279 279 end
280 280 end
281 281  
  282 + # model and eagar loading
  283 + load = options[:load].nil? ? true : options[:load]
  284 +
282 285 # An empty array will cause only the _id and _type for each hit to be returned
283 286 # http://www.elasticsearch.org/guide/reference/api/search/fields/
284 287 payload[:fields] = [] if load
285 288  
286   - tire_options = {load: load, size: per_page, from: offset}
287 289 if options[:type] or klass != searchkick_klass
288   - tire_options[:type] = [options[:type] || klass].flatten.map(&:document_type)
  290 + @type = [options[:type] || klass].flatten.map{|v| searchkick_index.klass_document_type(v) }
289 291 end
290 292  
291   - @search = Tire::Search::Search.new(index_name, tire_options)
292 293 @body = payload
293 294 @facet_limits = facet_limits
  295 + @page = page
  296 + @per_page = per_page
  297 + @load = load
294 298 end
295 299  
296 300 def searchkick_index
... ... @@ -305,20 +309,30 @@ module Searchkick
305 309 klass.searchkick_klass
306 310 end
307 311  
308   - def document_type
309   - klass.document_type
  312 + def params
  313 + params = {
  314 + index: options[:index_name] || searchkick_index.name,
  315 + body: body
  316 + }
  317 + params.merge!(type: @type) if @type
  318 + params
310 319 end
311 320  
312 321 def execute
313   - @search.options[:payload] = body
314 322 begin
315   - response = @search.json
316   - rescue Tire::Search::SearchRequestFailed => e
  323 + response = Searchkick.client.search(params)
  324 + rescue => e # TODO rescue type
317 325 status_code = e.message[1..3].to_i
318 326 if status_code == 404
319 327 raise "Index missing - run #{searchkick_klass.name}.reindex"
320   - elsif status_code == 500 and (e.message.include?("IllegalArgumentException[minimumSimilarity >= 1]") or e.message.include?("No query registered for [multi_match]"))
321   - raise "Upgrade Elasticsearch to 0.90.0 or greater"
  328 + elsif status_code == 500 and (
  329 + e.message.include?("IllegalArgumentException[minimumSimilarity >= 1]") or
  330 + e.message.include?("No query registered for [multi_match]") or
  331 + e.message.include?("[match] query does not support [cutoff_frequency]]") or
  332 + e.message.include?("No query registered for [function_score]]")
  333 + )
  334 +
  335 + raise "This version of Searchkick requires Elasticsearch 0.90.4 or greater"
322 336 else
323 337 raise e
324 338 end
... ... @@ -333,7 +347,13 @@ module Searchkick
333 347 response["facets"][field]["other"] = facet["total"] - facet["terms"].sum{|term| term["count"] }
334 348 end
335 349  
336   - Searchkick::Results.new(response, @search.options.merge(term: term, model_name: searchkick_klass.model_name))
  350 + opts = {
  351 + page: @page,
  352 + per_page: @per_page,
  353 + load: @load,
  354 + includes: options[:include] || options[:includes]
  355 + }
  356 + Searchkick::Results.new(searchkick_klass, response, opts)
337 357 end
338 358  
339 359 private
... ...
lib/searchkick/reindex.rb
  Diff comments   View file @29ab35b
... ... @@ -2,36 +2,28 @@ module Searchkick
2 2 module Reindex
3 3  
4 4 # https://gist.github.com/jarosan/3124884
  5 + # http://www.elasticsearch.org/blog/changing-mapping-with-zero-downtime/
5 6 def reindex
6 7 alias_name = searchkick_index.name
7   - new_index = alias_name + "_" + Time.now.strftime("%Y%m%d%H%M%S%L")
8   - index = Tire::Index.new(new_index)
  8 + new_name = alias_name + "_" + Time.now.strftime("%Y%m%d%H%M%S%L")
  9 + index = Searchkick::Index.new(new_name)
9 10  
10 11 clean_indices
11 12  
12   - success = index.create searchkick_index_options
13   - raise index.response.to_s if !success
  13 + index.create searchkick_index_options
14 14  
15   - if a = Tire::Alias.find(alias_name)
  15 + # check if alias exists
  16 + if Searchkick.client.indices.exists_alias(name: alias_name)
16 17 searchkick_import(index) # import before swap
17 18  
18   - a.indices.each do |i|
19   - a.indices.delete i
20   - end
21   -
22   - a.indices.add new_index
23   - response = a.save
24   -
25   - if response.success?
26   - clean_indices
27   - else
28   - raise response.to_s
29   - end
  19 + # get existing indices to remove
  20 + old_indices = Searchkick.client.indices.get_alias(name: alias_name).keys
  21 + actions = old_indices.map{|name| {remove: {index: name, alias: alias_name}} } + [{add: {index: new_name, alias: alias_name}}]
  22 + Searchkick.client.indices.update_aliases body: {actions: actions}
  23 + clean_indices
30 24 else
31 25 searchkick_index.delete if searchkick_index.exists?
32   - response = Tire::Alias.create(name: alias_name, indices: [new_index])
33   - raise response.to_s if !response.success?
34   -
  26 + Searchkick.client.indices.update_aliases body: {actions: [{add: {index: new_name, alias: alias_name}}]}
35 27 searchkick_import(index) # import after swap
36 28 end
37 29  
... ... @@ -42,10 +34,10 @@ module Searchkick
42 34  
43 35 # remove old indices that start w/ index_name
44 36 def clean_indices
45   - all_indices = JSON.parse(Tire::Configuration.client.get("#{Tire::Configuration.url}/_aliases").body)
  37 + all_indices = Searchkick.client.indices.get_aliases
46 38 indices = all_indices.select{|k, v| v["aliases"].empty? && k =~ /\A#{Regexp.escape(searchkick_index.name)}_\d{14,17}\z/ }.keys
47 39 indices.each do |index|
48   - Tire::Index.new(index).delete
  40 + Searchkick::Index.new(index).delete
49 41 end
50 42 indices
51 43 end
... ... @@ -73,7 +65,7 @@ module Searchkick
73 65 items = []
74 66 scope.all.each do |item|
75 67 items << item if item.should_index?
76   - if items.length % batch_size == 0
  68 + if items.length == batch_size
77 69 index.import items
78 70 items = []
79 71 end
... ...
lib/searchkick/results.rb
  Diff comments   View file @29ab35b
1 1 module Searchkick
2   - class Results < Tire::Results::Collection
3   - attr_reader :response
  2 + class Results
  3 + include Enumerable
  4 + extend Forwardable
  5 +
  6 + attr_reader :klass, :response, :options
  7 +
  8 + def_delegators :results, :each, :empty?, :size, :slice, :[], :to_ary
  9 +
  10 + def initialize(klass, response, options = {})
  11 + @klass = klass
  12 + @response = response
  13 + @options = options
  14 + end
  15 +
  16 + def results
  17 + @results ||= begin
  18 + if options[:load]
  19 + hit_ids = hits.map{|hit| hit["_id"] }
  20 + records = klass
  21 + if options[:includes]
  22 + records = records.includes(options[:includes])
  23 + end
  24 + records = records.find(hit_ids)
  25 + hit_ids = hit_ids.map(&:to_s)
  26 + records.sort_by{|r| hit_ids.index(r.id.to_s) }
  27 + else
  28 + hits.map{|hit| Hashie::Mash.new(hit["_source"]) }
  29 + end
  30 + end
  31 + end
4 32  
5 33 def suggestions
6   - if @response["suggest"]
7   - @response["suggest"].values.flat_map{|v| v.first["options"] }.sort_by{|o| -o["score"] }.map{|o| o["text"] }.uniq
  34 + if response["suggest"]
  35 + response["suggest"].values.flat_map{|v| v.first["options"] }.sort_by{|o| -o["score"] }.map{|o| o["text"] }.uniq
8 36 else
9 37 raise "Pass `suggest: true` to the search method for suggestions"
10 38 end
11 39 end
12 40  
  41 + def each_with_hit(&block)
  42 + results.zip(hits).each(&block)
  43 + end
  44 +
13 45 def with_details
14 46 each_with_hit.map do |model, hit|
15 47 details = {}
... ... @@ -20,13 +52,48 @@ module Searchkick
20 52 end
21 53 end
22 54  
23   - # fixes deprecation warning
24   - def __find_records_by_ids(klass, ids)
25   - @options[:load] === true ? klass.find(ids) : klass.includes(@options[:load][:include]).find(ids)
  55 + def facets
  56 + response["facets"]
26 57 end
27 58  
28 59 def model_name
29   - @options[:model_name]
  60 + klass.model_name
  61 + end
  62 +
  63 + def total_count
  64 + response["hits"]["total"]
  65 + end
  66 + alias_method :total_entries, :total_count
  67 +
  68 + def current_page
  69 + options[:page]
  70 + end
  71 +
  72 + def per_page
  73 + options[:per_page]
  74 + end
  75 + alias_method :limit_value, :per_page
  76 +
  77 + def total_pages
  78 + (total_count / per_page.to_f).ceil
  79 + end
  80 +
  81 + def offset_value
  82 + current_page * per_page
  83 + end
  84 +
  85 + def previous_page
  86 + current_page > 1 ? (current_page - 1) : nil
  87 + end
  88 +
  89 + def next_page
  90 + current_page < total_pages ? (current_page + 1) : nil
  91 + end
  92 +
  93 + protected
  94 +
  95 + def hits
  96 + @response["hits"]["hits"]
30 97 end
31 98  
32 99 end
... ...
lib/searchkick/similar.rb
  Diff comments   View file @29ab35b
... ... @@ -2,8 +2,8 @@ module Searchkick
2 2 module Similar
3 3  
4 4 def similar(options = {})
5   - like_text = self.class.searchkick_index.retrieve(document_type, id).to_hash
6   - .keep_if{|k,v| k[0] != "_" and (!options[:fields] or options[:fields].map(&:to_sym).include?(k)) }
  5 + like_text = self.class.searchkick_index.retrieve(self).to_hash
  6 + .keep_if{|k,v| !options[:fields] || options[:fields].map(&:to_s).include?(k) }
7 7 .values.compact.join(" ")
8 8  
9 9 # TODO deep merge method
... ...
lib/searchkick/version.rb
  Diff comments   View file @29ab35b
1 1 module Searchkick
2   - VERSION = "0.5.3"
  2 + VERSION = "0.7.1"
3 3 end
... ...
searchkick.gemspec
  Diff comments   View file @29ab35b
... ... @@ -8,8 +8,8 @@ Gem::Specification.new do |spec|
8 8 spec.version = Searchkick::VERSION
9 9 spec.authors = ["Andrew Kane"]
10 10 spec.email = ["andrew@chartkick.com"]
11   - spec.description = %q{Search made easy}
12   - spec.summary = %q{Search made easy}
  11 + spec.description = %q{Intelligent search made easy}
  12 + spec.summary = %q{Searchkick learns what your users are looking for. As more people search, it gets smarter and the results get better. It’s friendly for developers - and magical for your users.}
13 13 spec.homepage = "https://github.com/ankane/searchkick"
14 14 spec.license = "MIT"
15 15  
... ... @@ -18,8 +18,9 @@ Gem::Specification.new do |spec|
18 18 spec.test_files = spec.files.grep(%r{^(test|spec|features)/})
19 19 spec.require_paths = ["lib"]
20 20  
21   - spec.add_dependency "tire"
22   - spec.add_dependency "tire-contrib"
  21 + spec.add_dependency "activemodel"
  22 + spec.add_dependency "elasticsearch", "~> 0.4.11"
  23 + spec.add_dependency "hashie"
23 24  
24 25 spec.add_development_dependency "bundler", "~> 1.3"
25 26 spec.add_development_dependency "rake"
... ...
test/index_test.rb
  Diff comments   View file @29ab35b
... ... @@ -3,8 +3,11 @@ require_relative &quot;test_helper&quot;
3 3 class TestIndex < Minitest::Unit::TestCase
4 4  
5 5 def test_clean_indices
6   - old_index = Tire::Index.new("products_test_20130801000000000")
7   - different_index = Tire::Index.new("items_test_20130801000000000")
  6 + old_index = Searchkick::Index.new("products_test_20130801000000000")
  7 + different_index = Searchkick::Index.new("items_test_20130801000000000")
  8 +
  9 + old_index.delete if old_index.exists?
  10 + different_index.delete if different_index.exists?
8 11  
9 12 # create indexes
10 13 old_index.create
... ... @@ -18,7 +21,7 @@ class TestIndex &lt; Minitest::Unit::TestCase
18 21 end
19 22  
20 23 def test_clean_indices_old_format
21   - old_index = Tire::Index.new("products_test_20130801000000")
  24 + old_index = Searchkick::Index.new("products_test_20130801000000")
22 25 old_index.create
23 26  
24 27 Product.clean_indices
... ...
test/inheritance_test.rb
  Diff comments   View file @29ab35b
... ... @@ -10,7 +10,7 @@ class TestInheritance &lt; Minitest::Unit::TestCase
10 10 end
11 11  
12 12 def test_child_index_name
13   - assert_equal "animals_test", Dog.searchkick_index.name
  13 + assert_equal "animals-#{Date.today.year}", Dog.searchkick_index.name
14 14 end
15 15  
16 16 def test_child_search
... ...
test/sql_test.rb
  Diff comments   View file @29ab35b
... ... @@ -26,6 +26,9 @@ class TestSql &lt; Minitest::Unit::TestCase
26 26 assert_equal 2, products.per_page
27 27 assert_equal 3, products.total_pages
28 28 assert_equal 5, products.total_count
  29 + assert_equal 5, products.total_entries
  30 + assert_equal 2, products.limit_value
  31 + assert_equal 4, products.offset_value
29 32 end
30 33  
31 34 def test_pagination_nil_page
... ... @@ -245,12 +248,17 @@ class TestSql &lt; Minitest::Unit::TestCase
245 248  
246 249 def test_load_false
247 250 store_names ["Product A"]
248   - assert_kind_of Tire::Results::Item, Product.search("product", load: false).first
  251 + assert_kind_of Hash, Product.search("product", load: false).first
  252 + end
  253 +
  254 + def test_load_false_methods
  255 + store_names ["Product A"]
  256 + assert_equal "Product A", Product.search("product", load: false).first.name
249 257 end
250 258  
251 259 def test_load_false_with_include
252 260 store_names ["Product A"]
253   - assert_kind_of Tire::Results::Item, Product.search("product", load: false, include: [:store]).first
  261 + assert_kind_of Hash, Product.search("product", load: false, include: [:store]).first
254 262 end
255 263  
256 264 # TODO see if Mongoid is loaded
... ...
test/test_helper.rb
  Diff comments   View file @29ab35b
... ... @@ -2,14 +2,12 @@ require &quot;bundler/setup&quot;
2 2 Bundler.require(:default)
3 3 require "minitest/autorun"
4 4 require "minitest/pride"
  5 +require "logger"
5 6  
6 7 ENV["RACK_ENV"] = "test"
7 8  
8 9 File.delete("elasticsearch.log") if File.exists?("elasticsearch.log")
9   -Tire.configure do
10   - logger "elasticsearch.log", :level => "debug"
11   - pretty true
12   -end
  10 +Searchkick.client.transport.logger = Logger.new("elasticsearch.log")
13 11  
14 12 if defined?(Mongoid)
15 13 Mongoid.configure do |config|
... ... @@ -145,7 +143,7 @@ class Store
145 143 end
146 144  
147 145 class Animal
148   - searchkick autocomplete: [:name], suggest: [:name]
  146 + searchkick autocomplete: [:name], suggest: [:name], index_name: -> { "#{self.name.tableize}-#{Date.today.year}" }
149 147 end
150 148  
151 149 Product.searchkick_index.delete if Product.searchkick_index.exists?
... ...