ElasticSearch與Solr搜索引擎特性對比-new

1、ElasticSearch與Solr搜索引擎特性對比1、關鍵特性對比對比項Solr 5.xElasticSearch 2.x開發(fā)人員開發(fā)人員少，官方提供功能比ElasticSearch多開發(fā)人員多，官方功能少，但是第三方插件很豐富，擴展能力更強建立索引和查詢效率建立索引的速度和ElasticSearch差不多，當索引完全建立好了之后搜索會很快，但是一邊建立索引一邊搜索就會非常慢。經過詳細對比測試后發(fā)現，Solr在建索引時的查詢性能非常之差，因為Solr在建索引時會產生io的阻塞，造成搜索性能的下降。建立索引的速度和Solr差不多，第一次搜索速度比Solr慢一些，但之后會緩存結果，第二次搜索就會

2、很快。另外ElasticSearch邊建立索引邊搜索速度沒有太大變化，因為它是先把索引的內容保存到內存之中，當內存不夠時再把索引持久化到硬盤中，同時它還有一個隊列，是在系統空閑時自動把索引寫到硬盤中。Sharding沒有自動shard rebalancing的功能shard 必須一次設置好，之后不能修改，如果要修改則需要重新建立索引Schema模式需要 Schema (但是4.3版本之后有schemaless模式)無需Schema，創(chuàng)建索引更加靈活高級查詢沒有Query DSL有Query DSL，能夠支持更加高級和復雜的查詢語法，而且還可以以此擴展實現類SQL語法的查詢實時性傳統搜索應用實時

3、搜索應用（這個是ElasticSearch作者說的）可擴展性不支持插件開發(fā)模式支持插件開發(fā)模式，提供豐富的開源插件庫社區(qū)社區(qū)活躍社區(qū)活躍并有大數據公司支持Hadoop集成 支持支持，v2.3.0提供Graph圖結構數據存儲存儲Solr暫不支持ElasticSearch支持多盤存儲2、基于ElasticSearch數據交叉模型功能測試2.1、環(huán)境情況1.測試環(huán)境（市局環(huán)境）IP43/slave77/slave35CPU16* Intel(R) Xeon(R) CPU E5-2650 v2 2.60GHzMEM64G(ElasticSearch獨享16G)OS

4、CentOS release 6.5 (Final)2.測試數據數據來源省廳數據：VW_SHARE_ST_JJ_KKXX（交警_卡口信息）時間范圍2015100120160331數據量kettle清洗，數據量為11.1億測試數據查詢地址43:8200/_plugin/head/ 3.測試地址url41:18099/swagger-ui.html 2.2、測試結果場景一：一個車牌交叉滿足N個條件序號交叉條件個數測試用例預期結果實際結果3次測試耗時(s)備注13"datestart": "2016-

5、01-05T08:40:59", "location": "犍為縣", "dateend": "2016-01-05T08:53:59", "querystr": "犍為外灘", "datestart": "2016-02-03 09:48:00", "location": "樂山市", "dateend": "2016-02-03 19:48:00&quo

6、t;, "querystr": "月卡口", "datestart": "2016-01-07 19:47:18", "location": "南充市", "dateend": "2016-01-07 19:57:18", "querystr": "綠心路" 川LFC550 3 川LFC550 3 10在之前的測試時，曾出現結果為川LFC550 2的情況，經定位是對“犍為縣”的分詞不完善，修改代碼

7、后，結果正確2223"datestart": "2016-01-15T07:08:00", "location": "成華區(qū)", "dateend": "2016-01-15T07:18:22", "querystr": "雙橋子", "datestart": "2016-01-05T06:39:44", "location": "錦江區(qū)", "d

8、ateend": "2016-01-05T06:41:44", "querystr": "老成仁路口", "datestart": "2016-01-25T20:40:46", "location": "武侯區(qū)", "dateend": "2016-01-25T20:44:46", "querystr": "科華北路" 川AW7D16 3 川AW7D16 3 1在之前

9、測試中，曾出現結果為川AW7D16 0 的情況，經定位，是分詞不夠完善，修改代碼后，結果正確1133"datestart": "2016-02-06T17:48:41", "location": "四川省", "dateend": "2016-02-06T17:49:41", "querystr": "海棠路", "datestart": "2016-02-04T10:36:43", "

10、location": "富順縣", "dateend": "2016-02-04T10:36:44", "querystr": "鐘秀街東段", "datestart": "2016-02-04T13:10:50", "location": "富順縣", "dateend": "2016-02-04T13:10:53", "querystr": &

11、quot;吉安莊路" 貴JL7321 3 貴JL7321 3112144"datestart": "2016-02-16T15:06:57", "location": "瀘定縣", "dateend": "2016-02-16T15:06:57", "querystr": "石油路", "datestart": "2015-12-09T16:10:58", "location&

12、quot;: "新龍縣", "dateend": "2015-12-09T16:10:58", "querystr": "卡口抓拍機", "datestart": "2016-01-07T10:46:17", "location": "瀘定縣", "dateend": "2016-01-07T10:46:17", "querystr": "船頭&q

13、uot;, "datestart": "2016-02-16T18:10:20", "location": "雅安市", "dateend": "2016-02-16T18:10:20", "querystr": "名山紅星" 川AE15F8 4 川AE15F8 425352"datestart": "2015-11-05T13:39:46", "location": &quo

14、t;內江市", "dateend": "2015-11-05T13:39:46", "querystr": "蘭桂", "datestart": "2015-11-21T11:24:24", "location": "內江市", "dateend": "2015-11-21T11:24:24", "querystr": "牌樓路" 川K0103學

15、2 川K0103學 2119165"datestart": "2015-12-01T15:06:39", "location": "廣安市華鎣市", "dateend": "2015-12-01T15:06:39", "querystr": "古橋職中", "datestart": "2015-09-30T15:13:34", "location": "廣安市&quo

16、t;, "dateend": "2015-09-30T15:16:34", "querystr": "會展街", "datestart": "2016-02-12T12:15:10", "location": "廣安市", "dateend": "2016-02-12T12:15:10", "querystr": "高速", "datestart&

17、quot;: "2016-02-10T08:11:49", "location": "廣安市", "dateend": "2016-02-10T08:11:59", "querystr": "前華段出城", "datestart": "2016-02-05T12:36:06", "location": "廣安市", "dateend": "2016

18、-02-05T12:39:06", "querystr": "馬石梯" 渝B18D02 5 渝B18D02 5 46376"datestart": "2016-01-01T22:06:37", "location": "青羊區(qū)", "dateend": "2016-01-01T22:06:37", "querystr": "蘇坡立交", "datestart": &q

19、uot;2016-01-22T21:17:18", "location": "武侯區(qū)", "dateend": "2016-01-22T21:17:18", "querystr": "人民南路三段", "datestart": "2016-02-14T00:01:26", "location": "金牛區(qū)", "dateend": "2016-02-14T0

20、0:01:26", "querystr": "營門口", "datestart": "2016-02-08T11:47:08", "location": "成華區(qū)", "dateend": "2016-02-08T11:47:08", "querystr": "桃溪路", "datestart": "2015-11-02T15:00:22", &q

21、uot;location": "錦江區(qū)", "dateend": "2015-11-02T15:00:22", "querystr": "琉璃", "datestart": "2015-10-13T09:34:12", "location": "武侯區(qū)", "dateend": "2015-10-13T09:34:12", "querystr":

22、"雙楠路內側" 川ATS958 6 川ATS958 6 206186"datestart": "2016-01-01T22:06:37", "location": "市青羊", "dateend": "2016-01-01T22:06:37", "querystr": "蘇坡立交", "datestart": "2016-01-22T21:17:18", "locat

23、ion": "武侯區(qū)", "dateend": "2016-01-22T21:17:18", "querystr": "人民南路三段", "datestart": "2016-02-14T00:01:26", "location": "金牛區(qū)", "dateend": "2016-02-14T00:01:26", "querystr": "

24、;營門口", "datestart": "2016-02-08T11:47:08", "location": "成華區(qū)", "dateend": "2016-02-08T11:47:08", "querystr": "桃溪路", "datestart": "2015-11-02T15:00:22", "location": "錦江區(qū)", "

25、;dateend": "2015-11-02T15:00:22", "querystr": "琉璃", "datestart": "2015-10-13T09:34:12", "location": "武侯區(qū)", "dateend": "2015-10-13T09:34:12", "querystr": "雙楠路內側" 川ATS958 5川ATS958 532291&

26、quot;datestart": "2016-01-05T08:40:59", "location": "犍為縣", "dateend": "2016-01-05T08:53:59", "querystr": "犍為外灘" 川LFC550 1 川LFC550 1 121場景二：關注時間和地點，多個車牌滿足其中1N個條件序號交叉條件個數測試用例預期結果實際結果3次測試耗時(s)備注13"datestart": "201

27、6-02-07T16:44:10", "location": "廣安市", "dateend": "2016-02-07T16:44:20", "querystr": "人民南路", "datestart": "2016-02-10T11:08:08", "location": "廣安市", "dateend": "2016-02-10T11:09:09&q

28、uot;, "querystr": "民族小學", "datestart": "2016-01-14T19:53:10", "location": "武勝縣", "dateend": "2016-01-14T19:53:12", "querystr": "58KM" 粵YQJ221 2川XW6238 2 粵YQJ221 2川XW6238 1 84924"datestart":

29、"2016-01-01T00:00:00", "location": "德陽市", "dateend": "2016-01-04T00:00:00", "querystr": "西山", "datestart": "2016-02-22T08:10:55", "location": "德陽市", "dateend": "2016-02-22T08:

30、10:55", "querystr": "108國道", "datestart": "2015-11-16T06:17:58", "location": "羅江縣", "dateend": "2015-11-16T06:17:58", "querystr": "學校", "datestart": "2016-01-03T16:53:16", &qu

31、ot;location": "綿陽市", "dateend": "2016-01-03T16:53:16", "querystr": "東路" 川FAH567 2川FL1930 2川A9EB55 2 川FAH567 2川FL1930 2川A9EB55 23163、ElasticSearch與Solr的壓力測試對比3.1、測試環(huán)境1.集群環(huán)境此次用于測試Solr的集群包括4個節(jié)點：0（master），13（slave1sl

32、ave3）。用于測試ElasticSearch的集群包括4個節(jié)點：0(datanode,不可選舉為masternode)，13（datanode、可選舉為masternode）。測試集群信息：IP03CPU8* Intel(R) Xeon(R) CPU E5-2609 0 2.40GHzMemSolr：4G， ElasticSearch：4G(sys32G)OSCentOS release 6.5 測試客戶機（臨時使用slave3）：IP3CPU8* I

33、ntel(R) Xeon(R) CPU E5-2609 0 2.40GHzMemSolr4G ElasticSearch4G(sys32G)OSCentOS release 6.52.數據環(huán)境Solr中單索引庫中索引條數為5000W，ElasticSearch中單索引庫中索引條數為1億。3.2、測試工具工具名稱http_load輸入參數說明http_load -p 5 -f 1000 train_hc.txt-parallel 簡寫-p：并發(fā)的用戶進程數-fetches 簡寫-f：總計的訪問次數-rate 簡寫-r：每秒的訪問頻率-seconds 簡寫-

34、s：總計的訪問時間輸出參數說明1000 fetches, 5 max parallel, 1.13104e+06 bytes, in 4.50961 seconds 運行的請求數、最大的并發(fā)進程數、總計傳輸的數據、運行的時間1131.04 mean bytes/connection 平均每個請求連接傳輸的數據量221.748 fetches/sec, 250807 bytes/sec 每秒響應請求數、每秒傳輸數據量msecs/connect: 0.485064 mean, 1.337 max, 0.384 min連接平均響應時間、最大響應時間、最小響應時間msec

35、s/first-response: 21.7613 mean, 436.541 max, 13.468 min連接平均返回時間321 bad byte countsHTTP response codes:請求響應碼返回個數code 200 - 10003.3、測試結果注：每組測試均重復進行三次。場景一：模糊數字查詢查詢條件："q":"31997"統計測試結果如下：并發(fā)數總訪問次數搜索引擎平均每秒響應請求數(fetches/s)總消耗時間(s)連接平均響應時間(ms)連接平均返回時間(ms)平均成功次數平均失敗次數1010000Solr239.66441.

36、7250.16340.467100000ElasticSearch552.47118.1010.13017.6411000002510000Solr530.81918.8390.77744.675100000ElasticSearch677.38514.7630.12836.4281000005010000Solr664.68015.0450.20573.096100000ElasticSearch686.34014.5700.13072.28710000010010000Solr718.75113.9130.243136.493100000ElasticSearch699.83814.289

37、0.440141.77310000020010000Solr631.86915.8260.289310.823100000ElasticSearch632.78915.8034.370310.03010000050010000Solr38.925256.9081.9711848.6906479353ElasticSearch680.43214.6974.941716.01010000080010000Solr166.50360.0592.433278.3741139887ElasticSearch521.34919.18139.6641425.5809782218100010000Solr16

38、6.12360.1967.802386.7981489852ElasticSearch692.36114.44381.5321276.0009848152120010000Solr41.609240.33192.813407.1501019899ElasticSearch164.70560.715187.910841.63715528448場景二：精確中文查詢查詢條件："q":"人民南路"統計測試結果如下：并發(fā)數總訪問次數搜索引擎平均每秒響應請求數(fetches/s)總消耗時間(s)連接平均響應時間(ms)連接平均返回時間(ms)平均成功次數平均失敗次

39、數1010000Solr274.66536.4080.15935.091100000ElasticSearch606.13516.4980.13316.0401000002510000Solr496.52420.1400.17348.379100000ElasticSearch832.06612.0180.12229.5811000005010000Solr528.96718.9050.39291.614100000ElasticSearch822.15112.1630.13260.28210000010010000Solr499.61420.0160.313196.558100000Elas

40、ticSearch848.29911.7881.236115.98910000020010000Solr463.91021.5560.358422.760100000ElasticSearch785.50912.7312.071249.89810000050010000Solr155.62664.25610.400183.3117989202ElasticSearch827.33112.0879.174576.56710000080010000Solr166.10660.2037.612343.5382209780ElasticSearch814.86012.27230.071912.0441

41、00000100010000Solr166.01860.2342.354451.4892289772ElasticSearch867.41811.52946.7301026.2909859141120010000Solr165.70260.35079.148428.1622789722ElasticSearch165.51660.417185.596734.681159984013.4、總結² 在本次測試中，每次測試總的訪問次數為10000，從以上圖表中可以看出隨著并發(fā)數的增加，Solr和ElasticSearch的每秒響應請求數都是先增加，然后穩(wěn)定在一定數值，在超過某個閾值之后極速下降；² 從平均每秒響應請求次數來看，ElasticSearch性能指標優(yōu)