github / searchkick

Download as
Browse Code »

Commit 16243ac48933529c49478aeeb901189100a5d801

Authored by Andrew
1 parent 90895e10
Exists in master and in 18 other branches allow_missing, conversions_period, conversions_v2, eager_load, find_in_batches, index_name, lazy_index_name, less_deps, nested, reindex_refactor, relation, relation_search, routing_queue, scope_attributes, search_as_you_type, synonyms_ci, test_language, wildcard

Added support for Japanese and Polish [skip ci]

Showing 5 changed files with 58 additions and 13 deletions   Show diff stats
CHANGELOG.md
  Diff comments   View file @16243ac
1 1 ## 3.0.1 [unreleased]
2 2  
3 3 - Added `scope` option for partial reindex
  4 +- Added support for Japanese, Polish, and Ukranian
4 5  
5 6 ## 3.0.0
6 7  
... ...
README.md
  Diff comments   View file @16243ac
... ... @@ -19,6 +19,7 @@ Plus:
19 19 - easily personalize results for each user
20 20 - autocomplete
21 21 - “Did you mean” suggestions
  22 +- supports many languages
22 23 - works with ActiveRecord, Mongoid, and NoBrainer
23 24  
24 25 :speech_balloon: Get [handcrafted updates](http://chartkick.us7.list-manage.com/subscribe?u=952c861f99eb43084e0a49f98&id=6ea6541e8e&group[0][4]=true) for new features
... ... @@ -299,15 +300,12 @@ end
299 300  
300 301 [See the list of stemmers](https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-stemmer-tokenfilter.html)
301 302  
302   -For Chinese, install the [elasticsearch-analysis-ik plugin](https://github.com/medcl/elasticsearch-analysis-ik) and use:
  303 +A few languages require plugins:
303 304  
304   -```ruby
305   -class Product < ApplicationRecord
306   - searchkick language: "chinese"
307   -end
308   -```
309   -
310   -For Ukranian, install the [analysis-ukrainian plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/analysis-ukrainian.html) and use `language: "ukrainian"`. [master]
  305 +- `chinese` - [elasticsearch-analysis-ik plugin](https://github.com/medcl/elasticsearch-analysis-ik)
  306 +- `japanese` - [analysis-kuromoji plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/analysis-kuromoji.html)
  307 +- `polish` - [analysis-stempel plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/analysis-stempel.html)
  308 +- `ukrainian` - [analysis-ukrainian plugin](https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/analysis-ukrainian.html)
311 309  
312 310 ### Synonyms
313 311  
... ...
lib/searchkick/index_options.rb
  Diff comments   View file @16243ac
... ... @@ -157,6 +157,30 @@ module Searchkick
157 157 )
158 158  
159 159 settings[:analysis][:filter].delete(:searchkick_stemmer)
  160 + when "japanese"
  161 + settings[:analysis][:analyzer].merge!(
  162 + default_analyzer => {
  163 + type: "kuromoji"
  164 + },
  165 + searchkick_search: {
  166 + type: "kuromoji"
  167 + },
  168 + searchkick_search2: {
  169 + type: "kuromoji"
  170 + }
  171 + )
  172 + when "polish"
  173 + settings[:analysis][:analyzer].merge!(
  174 + default_analyzer => {
  175 + type: "polish"
  176 + },
  177 + searchkick_search: {
  178 + type: "polish"
  179 + },
  180 + searchkick_search2: {
  181 + type: "polish"
  182 + }
  183 + )
160 184 when "ukrainian"
161 185 settings[:analysis][:analyzer].merge!(
162 186 default_analyzer => {
... ...
lib/searchkick/query.rb
  Diff comments   View file @16243ac
... ... @@ -317,7 +317,7 @@ module Searchkick
317 317 qs << shared_options.merge(analyzer: "searchkick_search")
318 318  
319 319 # searchkick_search and searchkick_search2 are the same for ukrainian
320   - unless searchkick_options[:language] == "ukranian"
  320 + unless %w(japanese polish ukrainian).include?(searchkick_options[:language])
321 321 qs << shared_options.merge(analyzer: "searchkick_search2")
322 322 end
323 323 exclude_analyzer = "searchkick_search2"
... ...
test/language_test.rb
  Diff comments   View file @16243ac
... ... @@ -11,9 +11,27 @@ class LanguageTest &lt; Minitest::Test
11 11 # requires https://github.com/medcl/elasticsearch-analysis-ik
12 12 with_options(Song, language: "chinese") do
13 13 store_names ["中华人民共和国国歌"], Song
14   - assert_search "中华人民共和国", ["中华人民共和国国歌"], {}, Song
15   - assert_search "国歌", ["中华人民共和国国歌"], {}, Song
16   - assert_search "人", [], {}, Song
  14 + assert_language_search "中华人民共和国", ["中华人民共和国国歌"]
  15 + assert_language_search "国歌", ["中华人民共和国国歌"]
  16 + assert_language_search "人", []
  17 + end
  18 + end
  19 +
  20 + def test_japanese
  21 + # requires https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/analysis-kuromoji.html
  22 + with_options(Song, language: "japanese") do
  23 + store_names ["JR新宿駅の近くにビールを飲みに行こうか"], Song
  24 + assert_language_search "飲む", ["JR新宿駅の近くにビールを飲みに行こうか"]
  25 + assert_language_search "jr", ["JR新宿駅の近くにビールを飲みに行こうか"]
  26 + assert_language_search "新", []
  27 + end
  28 + end
  29 +
  30 + def test_polish
  31 + # requires https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/analysis-stempel.html
  32 + with_options(Song, language: "polish") do
  33 + store_names ["polski"], Song
  34 + assert_language_search "polskimi", ["polski"]
17 35 end
18 36 end
19 37  
... ... @@ -21,7 +39,11 @@ class LanguageTest &lt; Minitest::Test
21 39 # requires https://www.elastic.co/guide/en/elasticsearch/plugins/6.2/analysis-ukrainian.html
22 40 with_options(Song, language: "ukrainian") do
23 41 store_names ["ресторани"], Song
24   - assert_search "ресторан", ["ресторани"], {misspellings: false}, Song
  42 + assert_language_search "ресторан", ["ресторани"]
25 43 end
26 44 end
  45 +
  46 + def assert_language_search(term, expected)
  47 + assert_search term, expected, {misspellings: false}, Song
  48 + end
27 49 end
... ...