github / searchkick

Download as
Browse Code »

Commit 587a6f639fd61ecb1fd41101d36c9a47ea144f91

Authored by Andrew Kane
2 parents 454b59cc 6e3ffc6d
Exists in master and in 21 other branches allow_missing, conversions_period, conversions_v2, eager_load, find_in_batches, highlighted, index_name, lazy_index_name, less_deps, nested, reindex_async_threads, reindex_refactor, relation, relation_search, routing_queue, scope_attributes, search_as_you_type, synonyms_ci, test_language, threads, wildcard

Merge branch 'raise_dangerous_operations' into 1-0-take-2

Showing 9 changed files with 61 additions and 114 deletions   Show diff stats
.travis.yml
  Diff comments   View file @587a6f6
... ... @@ -21,6 +21,7 @@ gemfile:
21 21 - test/gemfiles/mongoid2.gemfile
22 22 - test/gemfiles/mongoid3.gemfile
23 23 - test/gemfiles/mongoid4.gemfile
  24 + - test/gemfiles/mongoid5.gemfile
24 25 matrix:
25 26 include:
26 27 - gemfile: test/gemfiles/nobrainer.gemfile
... ...
README.md
  Diff comments   View file @587a6f6
... ... @@ -45,7 +45,7 @@ Add this line to your application’s Gemfile:
45 45 gem 'searchkick'
46 46 ```
47 47  
48   -For Elasticsearch 2.0, use version `0.9.2` or above. For Elasticsearch 0.90, use version `0.6.3` and [this readme](https://github.com/ankane/searchkick/blob/v0.6.3/README.md).
  48 +For Elasticsearch 2.0, use the `elasticsearch2` branch and [this readme](https://github.com/ankane/searchkick/blob/elasticsearch2/README.md). For Elasticsearch 0.90, use version `0.6.3` and [this readme](https://github.com/ankane/searchkick/blob/v0.6.3/README.md).
49 49  
50 50 Add searchkick to models you want to search.
51 51  
... ... @@ -547,95 +547,11 @@ products = Product.search "peantu butta", suggest: true
547 547 products.suggestions # ["peanut butter"]
548 548 ```
549 549  
550   -### Aggregations
  550 +### Facets
551 551  
552   -[Aggregations](http://www.elasticsearch.org/guide/reference/api/search/facets/) provide aggregated search data.
  552 +[Facets](http://www.elasticsearch.org/guide/reference/api/search/facets/) provide aggregated search data.
553 553  
554   -![Aggregations](http://ankane.github.io/searchkick/facets.png)
555   -
556   -```ruby
557   -products = Product.search "chuck taylor", aggs: [:product_type, :gender, :brand]
558   -products.aggs
559   -```
560   -
561   -By default, `where` conditions apply to aggregations.
562   -
563   -```ruby
564   -Product.search "wingtips", where: {color: "brandy"}, aggs: [:size]
565   -# aggregations for brandy wingtips are returned
566   -```
567   -
568   -Change this with:
569   -
570   -```ruby
571   -Product.search "wingtips", where: {color: "brandy"}, aggs: [:size], smart_aggs: false
572   -# aggregations for all wingtips are returned
573   -```
574   -
575   -Set `where` conditions for each aggregation separately with:
576   -
577   -```ruby
578   -Product.search "wingtips", aggs: {size: {where: {color: "brandy"}}}
579   -```
580   -
581   -Limit
582   -
583   -```ruby
584   -Product.search "apples", aggs: {store_id: {limit: 10}}
585   -```
586   -
587   -#### Moving From Facets
588   -
589   -1. Replace `facets` with `aggs` in searches. **Note:** Range and stats facets are not supported at this time.
590   -
591   - ```ruby
592   - products = Product.search "chuck taylor", facets: [:brand]
593   - # to
594   - products = Product.search "chuck taylor", aggs: [:brand]
595   - ```
596   -
597   -2. Replace the `facets` method with `aggs` for results.
598   -
599   - ```ruby
600   - products.facets
601   - # to
602   - products.aggs
603   - ```
604   -
605   - The keys in results differ slightly. Instead of:
606   -
607   - ```json
608   - {
609   - "_type":"terms",
610   - "missing":0,
611   - "total":45,
612   - "other":34,
613   - "terms":[
614   - {"term":14.0,"count":11}
615   - ]
616   - }
617   - ```
618   -
619   - You get:
620   -
621   - ```json
622   - {
623   - "doc_count":45,
624   - "doc_count_error_upper_bound":0,
625   - "sum_other_doc_count":34,
626   - "buckets":[
627   - {"key":14.0,"doc_count":11}
628   - ]
629   - }
630   - ```
631   -
632   - Update your application to handle this.
633   -
634   -3. By default, `where` conditions apply to aggregations. This is equivalent to `smart_facets: true`. If you have `smart_facets: true`, you can remove it. If this is not desired, set `smart_aggs: false`.
635   -
636   -### Facets [deprecated]
637   -
638   -Facets have been deprecated in favor of aggregations as of Searchkick 0.9.2. See [how to upgrade](#moving-from-facets).
  554 +![Facets](http://ankane.github.io/searchkick/facets.png)
639 555  
640 556 ```ruby
641 557 products = Product.search "chuck taylor", facets: [:product_type, :gender, :brand]
... ... @@ -777,8 +693,6 @@ City.search "san", boost_by_distance: {field: :location, origin: [37, -122], fun
777 693  
778 694 Searchkick supports [Elasticsearch’s routing feature](https://www.elastic.co/blog/customizing-your-document-routing).
779 695  
780   -**Note:** Routing is not yet supported for Elasticsearch 2.0.
781   -
782 696 ```ruby
783 697 class Contact < ActiveRecord::Base
784 698 searchkick routing: :user_id
... ...
lib/searchkick.rb
  Diff comments   View file @587a6f6
... ... @@ -22,6 +22,7 @@ module Searchkick
22 22 class MissingIndexError < StandardError; end
23 23 class UnsupportedVersionError < StandardError; end
24 24 class InvalidQueryError < Elasticsearch::Transport::Transport::Errors::BadRequest; end
  25 + class DangerousOperation < StandardError; end
25 26  
26 27 class << self
27 28 attr_accessor :search_method_name
... ...
lib/searchkick/model.rb
  Diff comments   View file @587a6f6
... ... @@ -42,6 +42,12 @@ module Searchkick
42 42 end
43 43  
44 44 def searchkick_reindex(options = {})
  45 + unless options[:accept_danger]
  46 + if (respond_to?(:current_scope) && respond_to?(:default_scoped) && current_scope && current_scope.to_sql != default_scoped.to_sql) ||
  47 + (respond_to?(:queryable) && queryable != unscoped.with_default_scope)
  48 + raise Searchkick::DangerousOperation, "Only call reindex on models, not relations. Pass `accept_danger: true` if this is your intention."
  49 + end
  50 + end
45 51 searchkick_index.reindex_scope(searchkick_klass, options)
46 52 end
47 53 alias_method :reindex, :searchkick_reindex unless method_defined?(:reindex)
... ...
lib/searchkick/query.rb
  Diff comments   View file @587a6f6
... ... @@ -354,9 +354,9 @@ module Searchkick
354 354 }
355 355 }
356 356  
357   - where = {}
358   - where = (options[:where] || {}).reject { |k| k == field } unless options[:smart_aggs] == false
359   - agg_filters = where_filters(where.merge(agg_options[:where] || {}))
  357 + agg_options.deep_merge!(where: options.fetch(:where, {}).reject { |k| k == field }) if options[:smart_aggs] == true
  358 + agg_filters = where_filters(agg_options[:where])
  359 +
360 360 if agg_filters.any?
361 361 payload[:aggs][field] = {
362 362 filter: {
... ...
test/aggs_test.rb
  Diff comments   View file @587a6f6
... ... @@ -32,34 +32,34 @@ class AggsTest &lt; Minitest::Test
32 32 assert_equal(1, agg["sum_other_doc_count"]) if Gem::Version.new(Searchkick.server_version) >= Gem::Version.new("1.4.0")
33 33 end
34 34  
35   - def test_query_where
36   - assert_equal ({1 => 1}), store_agg(where: {in_stock: true}, aggs: [:store_id])
  35 + def test_where_no_smart_aggs
  36 + assert_equal ({2 => 2}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false}}})
  37 + assert_equal ({2 => 2}), store_agg(where: {color: "blue"}, aggs: {store_id: {where: {in_stock: false}}})
37 38 end
38 39  
39   - def test_two_wheres
40   - assert_equal ({2 => 1}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false}}})
  40 + def test_smart_aggs
  41 + assert_equal ({1 => 1}), store_agg(where: {in_stock: true}, aggs: [:store_id], smart_aggs: true)
41 42 end
42 43  
43   - def test_where_override
44   - assert_equal ({}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false, color: "blue"}}})
45   - assert_equal ({2 => 1}), store_agg(where: {color: "blue"}, aggs: {store_id: {where: {in_stock: false, color: "red"}}})
  44 + def test_smart_aggs_where
  45 + assert_equal ({2 => 1}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false}}}, smart_aggs: true)
46 46 end
47 47  
48   - def test_skip
49   - assert_equal ({1 => 1, 2 => 2}), store_agg(where: {store_id: 2}, aggs: [:store_id])
  48 + def test_smart_aggs_where_override
  49 + assert_equal ({2 => 1}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false, color: "blue"}}}, smart_aggs: true)
  50 + assert_equal ({}), store_agg(where: {color: "blue"}, aggs: {store_id: {where: {in_stock: false, color: "red"}}}, smart_aggs: true)
50 51 end
51 52  
52   - def test_skip_complex
53   - assert_equal ({1 => 1, 2 => 1}), store_agg(where: {store_id: 2, price: {gt: 5}}, aggs: [:store_id])
  53 + def test_smart_aggs_skip_agg
  54 + assert_equal ({1 => 1, 2 => 2}), store_agg(where: {store_id: 2}, aggs: [:store_id], smart_aggs: true)
54 55 end
55 56  
56   - def test_multiple
57   - assert_equal ({"store_id" => {1 => 1, 2 => 2}, "color" => {"blue" => 1, "green" => 1, "red" => 1}}), store_multiple_aggs(aggs: [:store_id, :color])
  57 + def test_smart_aggs_skip_agg_complex
  58 + assert_equal ({1 => 1, 2 => 1}), store_agg(where: {store_id: 2, price: {gt: 5}}, aggs: [:store_id], smart_aggs: true)
58 59 end
59 60  
60   - def test_smart_aggs_false
61   - assert_equal ({2 => 2}), store_agg(where: {color: "red"}, aggs: {store_id: {where: {in_stock: false}}}, smart_aggs: false)
62   - assert_equal ({2 => 2}), store_agg(where: {color: "blue"}, aggs: {store_id: {where: {in_stock: false}}}, smart_aggs: false)
  61 + def test_multiple_aggs
  62 + assert_equal ({"store_id" => {1 => 1, 2 => 2}, "color" => {"blue" => 1, "green" => 1, "red" => 1}}), store_multiple_aggs(aggs: [:store_id, :color])
63 63 end
64 64  
65 65 protected
... ...
test/gemfiles/mongoid5.gemfile 0 → 100644
  Diff comments   View file @587a6f6
... ... @@ -0,0 +1,7 @@
  1 +source 'https://rubygems.org'
  2 +
  3 +# Specify your gem's dependencies in searchkick.gemspec
  4 +gemspec path: "../../"
  5 +
  6 +gem "mongoid", "~> 5.0.0"
  7 +gem "activejob_backport"
... ...
test/index_test.rb
  Diff comments   View file @587a6f6
... ... @@ -102,6 +102,18 @@ class IndexTest &lt; Minitest::Test
102 102 assert_raises(Searchkick::InvalidQueryError) { Product.search(query: {boom: true}) }
103 103 end
104 104  
  105 + def test_dangerous_reindex
  106 + skip if mongoid2? || nobrainer?
  107 + assert_raises(Searchkick::DangerousOperation) { Product.where(id: [1, 2, 3]).reindex }
  108 + end
  109 +
  110 + def test_dangerous_reindex_accepted
  111 + skip if nobrainer?
  112 + store_names ["Product A", "Product B"]
  113 + Product.where(name: "Product A").reindex(accept_danger: true)
  114 + assert_search "product", ["Product A"]
  115 + end
  116 +
105 117 if defined?(ActiveRecord)
106 118  
107 119 def test_transaction
... ... @@ -110,7 +122,7 @@ class IndexTest &lt; Minitest::Test
110 122 raise ActiveRecord::Rollback
111 123 end
112 124  
113   - assert_search "product", []
  125 + assert_search "product", [], conversions: false
114 126 end
115 127  
116 128 end
... ...
test/test_helper.rb
  Diff comments   View file @587a6f6
... ... @@ -22,11 +22,17 @@ def elasticsearch2?
22 22 Searchkick.server_version.starts_with?("2.")
23 23 end
24 24  
25   -if defined?(Mongoid)
  25 +def mongoid2?
  26 + defined?(Mongoid) && Mongoid::VERSION.starts_with?("2.")
  27 +end
26 28  
27   - def mongoid2?
28   - Mongoid::VERSION.starts_with?("2.")
29   - end
  29 +def nobrainer?
  30 + defined?(NoBrainer)
  31 +end
  32 +
  33 +if defined?(Mongoid)
  34 + Mongoid.logger.level = Logger::INFO
  35 + Mongo::Logger.logger.level = Logger::INFO if defined?(Mongo::Logger)
30 36  
31 37 if mongoid2?
32 38 # enable comparison of BSON::ObjectIds
... ...