Elasticsearch7.5 經驗分享
1查詢
ES是非關係型數據庫，可將兩張表設置成父子關聯表，查詢一張表可將另一張表的相關信息帶出來，
更多表的查詢只能先查第一張表，遍歷第一次查詢結果，循環裏進行二次、三次...的查詢，後面的查詢用Multi並行多個表的查詢。
分頁展示的話，一次展示幾條，僅僅幾條進行關聯查詢也只是勉強夠用，做Excel批量導出這種功能，是完全不可以的，太慢了。
做分頁展示它總條數默認顯示一萬條，也可以設置顯示真實條數，但當點擊最後一頁時，會報錯，因爲它是遊標計算的，點擊最後一頁遊標要跑到最後，後臺撐不住太大的計算量，最多隻查出幾萬條，一般採用百度的做法，不顯示總頁數，無點擊最後一頁按鈕，而且每次查詢排序可能不一樣，因爲是計算出來的。
聚合查詢(按條件統計)很糟糕（慢）。
2.增、刪、改
ES的增、刪、改功能並不理想，速度比較慢。修改是線程不安全的。不適合做頻繁改動的表。
3.總結
ES做單表查詢是很好的速度非常快。做分頁、關聯、聚合都不合適。
Elasticsearch重要概念
cluster(集羣)
集羣中有多個節點，其中有一個爲主節點，這個主節點是可以通過選舉產生的，主從節點是對於集羣內部來說的。
es是去中心化的，就是無中心節點，從外部來看es集羣，邏輯上是個整體，與任何一個節點的通信和與整個es集羣通信是等價的。
node(節點)
每個運行實例(服務器進程)爲一個節點,每個節點可在同一機器上,也可在不同的機器上。
routing(路由)
找到此文檔的主分片(文檔的位置),默認情況下,這個值是由文檔的id生成。
shards(分片)
es可以把一個完整的索引分成多個分片,把一個大的索引拆分成多個,分佈到不同的節點上。構成分佈式搜索。分片的數量只能在索引創建前指定,並且索引創建後不能更改。
primary shard(主分片)
每個文檔都存儲在一個分片中，當存一個文檔的時候，會先存儲在主分片中，然後複製到副本中。默認一個索引有5個主分片。可在制定分片的數量，當分片一旦建立，分片的數量則不能修改。
replica shard(副本分片)
每一個分片有零個或多個副本。副本主要是主分片的複製，其中有兩個目的：
1、增加高可用性：當主分片失敗的時候，可以從副本分片中選擇一個作爲主分片。
2、提高性能：當查詢的時候可以到主分片或者副本分片中進行查詢。
默認情況下，一個主分配有一個副本，但副本的數量可以在後面動態的配置增加。副本必須部署在不同的節點上，不能部署在和主分片相同的節點上。
template(模板)
索引可使用預定義的模板進行創建,這個模板稱作Index templates。模板設置包括settings和mappings。
index(索引)
索引就是一個擁有相似特徵的文檔的集合。
一個索引由一個名字來標識(必須全部是小寫字母的),對此索引中的文檔進行搜索、更新和刪除的時候,都要用到這個名字。在一個集羣中,你能夠創建任意多個索引。
type(類型)
通常，會爲具有一組相同字段的文檔定義一個類型。一個type下的document,都有相同的field。
一種type就像一類表。如用戶表、員工表等。
注意:
- ES 5.x中一個index可以有多種type。
- ES 6.x中一個index只能有一種type。
- ES 7.x以後，將移除type這個概念。
mapping(映射)
定義每個字段的類型、字段所使用的分詞器等。相當於關係型數據庫中的表結構。一個映射可以事先被定義，或者在第一次存儲文檔的時候自動識別。
document(文檔)
可被索引的基礎信息單元,一個document可以是一條數據,通常用JSON數據結構表示,每個index下的type中,可以去存儲多個document。一個document裏面有多個field。
field(字段)
一個文檔中包含零個或者多個字段，字段可以是一個簡單的值（例如字符串、整數、日期），也可以是一個數組或對象的嵌套結構。字段類似於關係數據庫中的表中的列。
source field(源)
默認情況下，你的原文檔將被存儲在_source這個字段中，當你查詢的時候也是返回這個字段。
這可從搜索結果中訪問原始的對象，這個對象返回一個精確的json字符串，這個對象不顯示索引分析後的其他任何數據。
id
id是一個文件的唯一標識，如果在存庫的時候沒有提供id，系統會自動生成一個id，文檔的index/type/id必須是唯一的。
recovery(恢復)
代表數據恢復或叫數據重新分佈，es在有節點加入或退出時會根據機器的負載對索引分片進行重新分配，掛掉的節點重新啓動時也會進行數據恢復。
River(數據源)
是其它存儲方式（如：數據庫）同步數據到es的一個方法。它是以插件方式存在的一個es服務，通過讀取river中的數據並把它索引到es中。
gateway(持久化)
代表es索引的持久化存儲方式，es默認是先把索引存放到內存中，當內存滿了時再持久化到硬盤。當這個es集羣關閉再重新啓動時就會從gateway中讀取索引數據。
es支持多種類型的gateway，有本地文件系統（默認），分佈式文件系統，Hadoop的HDFS和amazon的s3雲存儲服務。
discovery.zen(找節點)
代表es的自動發現節點機制，es是一個基於p2p的系統，它先通過廣播尋找存在的節點，再通過多播協議來進行節點之間的通信，同時也支持點對點的交互。
Transport(交互)
代表es內部節點或集羣與客戶端的交互方式，默認內部是使用tcp協議進行交互，同時它支持http協議（json格式）、thrift、servlet、memcached、zeroMQ等的傳輸協議（通過插件方式集成）。
term(索引詞)
索引詞，在elasticsearch中索引詞(term)是一個能夠被索引的精確值。foo，Foo Foo幾個單詞是不相同的索引詞。索引詞(term)是可以通過term查詢進行準確的搜索。
text(文本)
是一段普通的非結構化文字，通常，文本會被分析稱一個個的索引詞，存儲在elasticsearch的索引庫中，爲了讓文本能夠進行搜索，
文本字段需要事先進行分析；當對文本中的關鍵詞進行查詢的時候，搜索引擎應該根據搜索條件搜索出原文本。
analysis(分析)
將文本轉換爲索引詞的過程，分析的結果依賴於分詞器，比如： FOO BAR, Foo-Bar, foo bar這幾個單詞有可能會被分析成相同的索引詞foo和bar，這些索引詞存儲在elasticsearch的索引庫中。當用 FoO:bAR進行全文搜索的時候，搜索引擎根據匹配計算也能在索引庫中搜索出之前的內容。這就是elasticsearch的搜索分析。
replica(副本)
shard的replica副本,replica可以在shard故障時提供備用服務。多個replica可提升搜索操作的吞吐量和性能。
segment(分段)
一個shard包含多個segment,每個segment都是倒排索引。查詢時,每個shard會把所有segment結果彙總作爲shard的結果返回。
Elasticsearch官網JavaAPI
SpringBoot的pom文件
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.jackchen</groupId>
<artifactId>EsDemo4</artifactId>
<version>1.0-SNAPSHOT</version>
<properties>
<elastic-version>7.5.0</elastic-version>
</properties>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.2.RELEASE</version>
</parent>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.5.0</version>
<exclusions>
<exclusion>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
</exclusion>
<exclusion>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.5.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.5.0</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.9.4</version>
</dependency>
<dependency>
<groupId>com.alibaba</groupId>
<artifactId>fastjson</artifactId>
<version>1.2.62</version>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.18.4</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!--反射用到-->
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.7</version>
</dependency>
</dependencies>
<build>
<plugins>
<!-- java編譯插件 -->
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.2</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
<encoding>UTF-8</encoding>
</configuration>
</plugin>
</plugins>
</build>
</project>
java客戶端的重要組件
SearchRequest 、SearchSourceBuilder 、QueryBuilder 、SearchResponse 、SearchHit
public static void testRequest()throws Exception{
// 創建請求對象,設置查詢多個文檔庫,也可指定單個文檔庫。
SearchRequest request = new SearchRequest("index01","index02","index03");
// 也可通過 indices 方法指定文檔庫中
request.indices("posts01","posts02", "posts03");
// 設置指定查詢的路由分片
request.routing("routing");
// 指定優先去某個分片上去查詢（默認的是隨機先去某個分片）
request.preference("_local");
// 設置緩存
request.requestCache();
// 取出查詢語句
request.toString();
}
public static void testSource()throws Exception{
//創建源
SearchSourceBuilder source= new SearchSourceBuilder();
// 第幾頁
source.from(0);
// 每頁多少條數據(默認是10條)
source.size(100);
// 設置排序規則
source.sort(new ScoreSortBuilder().order(SortOrder.DESC));
source.sort(new FieldSortBuilder("id").order(SortOrder.ASC));
//獲取的字段（列）和不需要獲取的列
String[] includeFields = new String[]{"birthday","name"};
String[] excludeFields = new String[]{"age","address"};
source.fetchSource(includeFields,excludeFields);
// 設置超時時間
source.timeout(new TimeValue(60, TimeUnit.SECONDS));
source.highlighter();// 高亮
source.aggregation(AggregationBuilders.terms("by_company"));// 聚合
//分詞查詢
source.profile(true);
source.query();
}
public static void testBuilder()throws Exception{
//全匹配(查出全部)
MatchAllQueryBuilder matchAllQuery = QueryBuilders.matchAllQuery();
//匹配查詢
MatchQueryBuilder matchQuery = QueryBuilders.matchQuery("","").analyzer("");
//匹配文本查詢
MatchPhraseQueryBuilder matchPhraseQuery = QueryBuilders.matchPhraseQuery("","");
//匹配文本前綴查詢
MatchPhrasePrefixQueryBuilder matchPhrasePrefixQuery = QueryBuilders.matchPhrasePrefixQuery("","");
//判斷莫子是否有值（String）
ExistsQueryBuilder existsQuery = QueryBuilders.existsQuery("");
//前綴查詢
PrefixQueryBuilder prefixQuery = QueryBuilders.prefixQuery("","");
//精確查詢
TermQueryBuilder termQuery = QueryBuilders.termQuery("","");
//範圍查詢
RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("birthday").from("2016-01-01 00:00:00");
QueryStringQueryBuilder queryBuilder009 = QueryBuilders.queryStringQuery("");
QueryBuilders.disMaxQuery();
HighlightBuilder highlightBuilder = new HighlightBuilder();
HighlightBuilder.Field highlightTitle =
new HighlightBuilder.Field("title");
highlightTitle.highlighterType("unified");
highlightBuilder.field(highlightTitle);
HighlightBuilder.Field highlightUser = new HighlightBuilder.Field("user");
highlightBuilder.field(highlightUser);
// 組合器
BoolQueryBuilder builder = QueryBuilders.boolQuery();
//過濾
builder.filter();
//且
builder.must();
//非
builder.mustNot();
//或
builder.should();
}
public static void testResponse()throws Exception {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("127.0.0.1", 9200, "http")));
SearchRequest searchRequest = new SearchRequest("user");
// 同步
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
RestStatus status = response.status();
TimeValue took = response.getTook();
Boolean terminatedEarly = response.isTerminatedEarly();
boolean timedOut = response.isTimedOut();
int totalShards = response.getTotalShards();
int successfulShards = response.getSuccessfulShards();
int failedShards = response.getFailedShards();
for (ShardSearchFailure failure : response.getShardFailures()) {
// failures should be handled here
}
// 異步
ActionListener<SearchResponse> listener = new ActionListener<SearchResponse>() {
@Override
public void onResponse(SearchResponse searchResponse) {
}
@Override
public void onFailure(Exception e) {
}
};
client.searchAsync(searchRequest, RequestOptions.DEFAULT, listener);
}
public static void testHits()throws Exception {
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("127.0.0.1", 9200, "http")));
SearchRequest searchRequest = new SearchRequest("user");
// 同步
SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);
SearchHits hits = response.getHits();
TotalHits totalHits = hits.getTotalHits();
//總數
long numHits = totalHits.value;
//
TotalHits.Relation relation = totalHits.relation;
float maxScore = hits.getMaxScore();
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
String index = hit.getIndex();
String id = hit.getId();
float score = hit.getScore();
String sourceAsString = hit.getSourceAsString();
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String documentTitle = (String) sourceAsMap.get("title");
List<Object> users = (List<Object>) sourceAsMap.get("user");
Map<String, Object> innerObject =
(Map<String, Object>) sourceAsMap.get("innerObject");
}
// 高亮獲取
for (SearchHit hit : response.getHits()) {
Map<String, HighlightField> highlightFields = hit.getHighlightFields();
HighlightField highlight = highlightFields.get("title");
Text[] fragments = highlight.fragments();
String fragmentString = fragments[0].string();
}
// 獲取聚合結果
Aggregations aggregations = response.getAggregations();
Terms byCompanyAggregation = aggregations.get("by_company");
Terms.Bucket elasticBucket = byCompanyAggregation.getBucketByKey("Elastic");
Avg averageAge = elasticBucket.getAggregations().get("average_age");
double avg = averageAge.getValue();
// 獲取大量聚合結果
Map<String, Aggregation> aggregationMap = aggregations.getAsMap();
Terms companyAggregation = (Terms) aggregationMap.get("by_company");
List<Aggregation> aggregationList = aggregations.asList();
for (Aggregation agg : aggregations) {
String type = agg.getType();
if (type.equals(TermsAggregationBuilder.NAME)) {
Terms.Bucket elasticBucket2 = ((Terms) agg).getBucketByKey("Elastic");
long numberOfDocs = elasticBucket2.getDocCount();
}
}
}
CountRequest（統計查出數量）（快）
public static void test001()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(
new HttpHost("127.0.0.1", 9200, "http")));
//統計匹配數量(不查出表的內容，更快。)
CountRequest countRequest = new CountRequest("user");
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
MatchQueryBuilder queryBuilder = QueryBuilders.matchQuery("name", "小");
sourceBuilder.query(queryBuilder);
countRequest.source(sourceBuilder);
CountResponse response =client.count(countRequest, RequestOptions.DEFAULT);
long count = response.getCount();
RestStatus status = response.status();
Boolean terminatedEarly = response.isTerminatedEarly();
int totalShards = response.getTotalShards();
int skippedShards = response.getSkippedShards();
int successfulShards = response.getSuccessfulShards();
int failedShards = response.getFailedShards();
System.out.println("**************************count: "+count);
System.out.println("status: "+status);
System.out.println("terminatedEarly: "+terminatedEarly);
System.out.println("totalShards: "+totalShards);
System.out.println("skippedShards: "+skippedShards);
System.out.println("successfulShards: "+successfulShards);
System.out.println("failedShards: "+failedShards);
for (ShardSearchFailure failure : response.getShardFailures()) {
System.out.println("failure: "+failure);
}
client.close();
}
ES的增、刪、改、批量操作
//單條增
public static void addDocment()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("127.0.0.1", 9200, "http")));
//Map提供供文檔源
Map<String, Object> jsonMap = new HashMap<>();
jsonMap.put("name", "小紅");
jsonMap.put("sex", "女");
jsonMap.put("age", 22);
jsonMap.put("birthDay", new Date());
jsonMap.put("message", "測試");
IndexRequest indexRequest1 = new IndexRequest("user2", "doc", "5")
.source(jsonMap);
// 同步執行
IndexResponse indexResponse1 =client.index(indexRequest1,RequestOptions.DEFAULT);
client.close();
//XContentBuilder提供供文檔源
XContentBuilder builder = XContentFactory.jsonBuilder();
builder.startObject();
{
builder.field("name", "South");
builder.timeField("birthDay", new Date());
builder.field("message", "第二個小demo");
}
builder.endObject();
IndexRequest indexRequest2 = new IndexRequest("user", "doc", "2")
.source(builder);
// 同步執行
IndexResponse indexResponse2 =client.index(indexRequest2,RequestOptions.DEFAULT);
String index = indexResponse1.getIndex();
String type = indexResponse1.getType();
String id = indexResponse1.getId();
long version = indexResponse1.getVersion();
RestStatus restStatus = indexResponse1.status();
DocWriteResponse.Result result = indexResponse1.getResult();
ReplicationResponse.ShardInfo shardInfo = indexResponse1.getShardInfo();
client.close();
}
//刪
public void deleteTest()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(
new HttpHost("127.0.0.1", 9200, "http")));
DeleteRequest request = new DeleteRequest("posts","1");
DeleteResponse deleteResponse = client.delete(request, RequestOptions.DEFAULT);
}
//單個改
public static void updateDocment()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("127.0.0.1", 9200, "http")));
Map<String, Object> jsonMap = new HashMap<>();
jsonMap.put("name", "JunSouth");
UpdateRequest updateRequest = new UpdateRequest("user","doc","6").doc(jsonMap);
UpdateResponse updateResponse =client.update(updateRequest,RequestOptions.DEFAULT);
String index = updateResponse.getIndex();
String type = updateResponse.getType();
String id = updateResponse.getId();
long version = updateResponse.getVersion();
System.out.println("index:"+index+" type:"+type+" id:"+id+" version:"+version);
if(updateResponse.getResult() == DocWriteResponse.Result.CREATED) {
System.out.println("文檔已創建");
}else if(updateResponse.getResult() == DocWriteResponse.Result.UPDATED) {
System.out.println("文檔已更新");
}else if(updateResponse.getResult() == DocWriteResponse.Result.DELETED) {
System.out.println("文檔已刪除");
}else if(updateResponse.getResult() == DocWriteResponse.Result.NOOP) {
System.out.println("文檔不受更新的影響");
}
client.close();
}
//批量操作
public static void bulkDocment()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(
new HttpHost("127.0.0.1", 9200, "http")));
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(new IndexRequest("user","doc","5")
.source(XContentType.JSON,"name", "test")); // 將第一個 IndexRequest 添加到批量請求中
bulkRequest.add(new IndexRequest("user","doc","6")
.source(XContentType.JSON,"name","test")); // 第二個
BulkResponse bulkResponse = client.bulk(bulkRequest,RequestOptions.DEFAULT);
boolean falgs = bulkResponse.hasFailures(); // true 表示至少有一個操作失敗
System.out.println("falgs: "+falgs);
for (BulkItemResponse bulkItemResponse : bulkResponse) { // 遍歷所有的操作結果
DocWriteResponse itemResponse = bulkItemResponse.getResponse(); // 獲取操作結果的響應，可以是 IndexResponse,UpdateResponse or DeleteResponse,它們都可以慚怍是 DocWriteResponse 實例。
if (bulkItemResponse.getOpType() == DocWriteRequest.OpType.INDEX || bulkItemResponse.getOpType() == DocWriteRequest.OpType.CREATE) {
IndexResponse indexResponse = (IndexResponse) itemResponse;
System.out.println("index 操作後的響應結果");
}else if(bulkItemResponse.getOpType() == DocWriteRequest.OpType.UPDATE) {
UpdateResponse updateResponse = (UpdateResponse) itemResponse;
System.out.println("update 操作後的響應結果");
}else if(bulkItemResponse.getOpType() == DocWriteRequest.OpType.DELETE) {
DeleteResponse deleteResponse = (DeleteResponse) itemResponse;
System.out.println("delete 操作後的響應結果");
}
}
for (BulkItemResponse bulkItemResponse : bulkResponse) {
if (bulkItemResponse.isFailed()) { // 檢測給定的操作是否失敗
BulkItemResponse.Failure failure = bulkItemResponse.getFailure();
System.out.println("獲取失敗信息: "+failure);
}
}
client.close();
}
IndexRequest、DeleteRequest、UpdateRequest、BulkRequest
IndexRequest indexRequest
設置路由值。
indexRequest.routing("routing");
設置parent值。
indexRequest.parent("parent");
等待主碎片可用的作爲TimeValue的超時。
等待主碎片可用的作爲String的超時。
indexRequest.timeout(TimeValue.timeValueSeconds(1));
indexRequest.timeout("1s");
刷新策略作爲WriteRequest.RefreshPolicy實例提供。
刷新策略作爲String提供。
indexRequest.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
indexRequest.setRefreshPolicy("wait_for");
設置版本。
indexRequest.version(2);
設置版本類型。
indexRequest.versionType(VersionType.EXTERNAL);
操作類型作爲DocWriteRequest.OpType值提供。
作爲String提供的操作類型：可以爲create或update（默認）。
indexRequest.opType(DocWriteRequest.OpType.CREATE);
indexRequest.opType("create");
索引文檔之前要執行的攝取管道的名稱。
indexRequest.setPipeline("pipeline");
DeleteRequest deleteRequest
設置路由值。
deleteRequest.routing("routing");
設置parent值。
deleteRequest.parent("parent");
等待主碎片可用的作爲TimeValue的超時。
等待主碎片可用的作爲String的超時。
deleteRequest.timeout(TimeValue.timeValueMinutes(2));
deleteRequest.timeout("2m");
將刷新策略作爲WriteRequest.RefreshPolicy實例。
將刷新策略作爲String。
deleteRequest.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
deleteRequest.setRefreshPolicy("wait_for");
設置版本。
deleteRequest.version(2);
設置版本類型。
deleteRequest.versionType(VersionType.EXTERNAL);
UpdateRequest updateRequest
設置超時時間。
updateRequest.timeout(TimeValue.timeValueSeconds(1));
updateRequest.timeout("1s");
設置刷新策略。
updateRequest.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
updateRequest.setRefreshPolicy("wait_for");
設置衝突後重試次數。
updateRequest.retryOnConflict(3);
獲取數據源，默認是開啓的。
updateRequest.fetchSource(true);
包括特定字段。
String[] includes = new String[]{"updated", "r*"};
String[] excludes = Strings.EMPTY_ARRAY;
updateRequest.fetchSource(new FetchSourceContext(true, includes, excludes));
排除特定字段。
String[] includes = Strings.EMPTY_ARRAY;
String[] excludes = new String[]{"updated"};
updateRequest.fetchSource(new FetchSourceContext(true, includes, excludes));
指定版本。
updateRequest.version(2);
禁用 noop detection
updateRequest.scriptedUpsert(true);
設置如果更新的文檔不存在,就必須要創建一個。
updateRequest.docAsUpsert(true);
BulkRequest bulkRequest
設置超時時間
bulkRequest.timeout(TimeValue.timeValueMinutes(2));
request.timeout("2m");
設置刷新策略
bulkRequest.setRefreshPolicy(WriteRequest.RefreshPolicy.WAIT_UNTIL);
bulkRequest.setRefreshPolicy("wait_for");
設置在批量操作前必須有幾個分片處於激活狀態。
bulkRequest.waitForActiveShards(2);
bulkRequest.waitForActiveShards(ActiveShardCount.ALL); // 全部分片都處於激活狀態
bulkRequest.waitForActiveShards(ActiveShardCount.DEFAULT); // 默認
bulkRequest.waitForActiveShards(ActiveShardCount.ONE); // 一個
ES查詢
//查詢某索引下全部數據
public static void searchAll()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("127.0.0.1", 9200, "http")));
SearchRequest searchRequest = new SearchRequest("user"); // 設置搜索的 index 。
QueryBuilder queryBuilder = QueryBuilders.matchAllQuery();
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(queryBuilder); //設置搜索,可以是任何類型的 QueryBuilder.
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest,RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
float maxScore = hits.getMaxScore();
for (SearchHit hit : hits.getHits()) {
System.out.println("hit: "+hit);
String sourceAsString = hit.getSourceAsString();
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
System.out.println("name: "+name);
}
client.close();
//匹配查詢器
QueryBuilder matchQueryBuilder = QueryBuilders.matchQuery("user", "kimchy")
.fuzziness(Fuzziness.AUTO)
.prefixLength(3)
.maxExpansions(10);
searchSourceBuilder.query(matchQueryBuilder);
//高亮
HighlightBuilder highlightBuilder = new HighlightBuilder();
HighlightBuilder.Field highlightTitle = new HighlightBuilder.Field("name"); // title 字段高亮
highlightTitle.highlighterType("unified"); // 配置高亮類型
highlightBuilder.field(highlightTitle); // 添加到 builder
HighlightBuilder.Field highlightUser = new HighlightBuilder.Field("user");
highlightBuilder.field(highlightUser);
searchSourceBuilder.highlighter(highlightBuilder);
}
//普通條件查詢
public static void search01()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("127.0.0.1", 9200, "http")));
SearchRequest searchRequest = new SearchRequest("user"); // 設置搜索的 index 。
// 查詢器
QueryBuilder queryBuilder01 = QueryBuilders.termQuery("name", "test"); //完全匹配
QueryBuilder queryBuilder02 =QueryBuilders.fuzzyQuery("name", "t"); //模糊查詢
QueryBuilder queryBuilder03 =QueryBuilders.prefixQuery("name", "小"); //前綴查詢
QueryBuilder queryBuilder04 =QueryBuilders.matchQuery("name", "小"); //匹配查詢
WildcardQueryBuilder queryBuilder = QueryBuilders.wildcardQuery("name","*jack*");//搜索名字中含有jack文檔（name中只要包含jack即可）
// 搜索器(排序、分頁...)。
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(queryBuilder04); // 設置搜索條件
searchSourceBuilder.from(0); // 起始 index
searchSourceBuilder.size(5); // 大小 size
// searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS)); // 設置搜索的超時時間
// searchSourceBuilder.sort(new ScoreSortBuilder().order(SortOrder.DESC)); // 根據分數 _score 降序排列 (默認行爲)
// searchSourceBuilder.sort(new FieldSortBuilder("_uid").order(SortOrder.ASC)); // 根據 id 降序排列
searchRequest.source(searchSourceBuilder); // 將 SearchSourceBuilder 添加到 SeachRequest 中。
SearchResponse searchResponse = client.search(searchRequest,RequestOptions.DEFAULT);
SearchHits hits = searchResponse.getHits();
float maxScore = hits.getMaxScore();
for (SearchHit hit : hits.getHits()) {
String sourceAsString = hit.getSourceAsString();
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
System.out.println("hit: "+hit);
System.out.println("name: "+name);
}
client.close();
}
// 聚合查詢
public static void search02()throws Exception{
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("127.0.0.1", 9200, "http")));
SearchRequest searchRequest = new SearchRequest("user2");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
// 根據 sex 字段分組
TermsAggregationBuilder aggregation = AggregationBuilders.terms("my_sex")
.field("sex.keyword");
aggregation.subAggregation(AggregationBuilders.avg("avg_age")
.field("age")); // age(統計的字段)需是數值型
aggregation.subAggregation(AggregationBuilders.max("max_age")
.field("age"));
aggregation.subAggregation(AggregationBuilders.min("min_age")
.field("age"));
searchSourceBuilder.aggregation(aggregation);
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
Aggregations aggregations = searchResponse.getAggregations();
Terms sexTerms = aggregations.get("my_sex");
//獲取每組的信息
for (Terms.Bucket bucket : sexTerms.getBuckets()) {
System.out.println("分組的字段名: " + bucket.getKeyAsString());
System.out.println("每組數量: " + bucket.getDocCount());
}
//求平均
Terms.Bucket elasticBucket1 = sexTerms.getBucketByKey("女");
Avg averageAge1 = elasticBucket1.getAggregations().get("avg_age");
double avg1 = averageAge1.getValue();
System.out.println("女性平均年齡："+avg1);
Terms.Bucket elasticBucket2 = sexTerms.getBucketByKey("男");
Avg averageAge2 = elasticBucket2.getAggregations().get("avg_age");
double avg2 = averageAge2.getValue();
System.out.println("男性平均年齡："+avg2);
//求最大最小
Terms.Bucket elasticBucket3 = sexTerms.getBucketByKey("女");
Max maxAge3 = elasticBucket3.getAggregations().get("max_age");
double maxAge = maxAge3.getValue();
System.out.println("女性最大年齡："+maxAge);
Terms.Bucket elasticBucket4 = sexTerms.getBucketByKey("女");
Min maxAge4 = elasticBucket4.getAggregations().get("min_age");
double minAge = maxAge4.getValue();
System.out.println("女性最大年齡："+minAge);
client.close();
}
// 多查詢
public static void multiSearch()throws Exception{
MultiSearchRequest multiSearchRequest = new MultiSearchRequest();
// 查兩個張索引
SearchRequest firstSearchRequest = new SearchRequest("user");
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchQuery("name", "大黑"));
firstSearchRequest.source(searchSourceBuilder);
multiSearchRequest.add(firstSearchRequest);
SearchRequest secondSearchRequest = new SearchRequest("car");
searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(QueryBuilders.matchQuery("weight", "3T"));
secondSearchRequest.source(searchSourceBuilder);
multiSearchRequest.add(secondSearchRequest);
// 取值1
MultiSearchResponse multiSearchResponse = client.msearch(multiSearchRequest,RequestOptions.DEFAULT);
MultiSearchResponse.Item firstResponse = multiSearchResponse.getResponses()[0];
SearchResponse firstSearchResponse = firstResponse.getResponse();
for (SearchHit hit : firstSearchResponse.getHits()) {
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
}
MultiSearchResponse.Item secondResponse = response.getResponses()[1];
SearchResponse secondSearchResponse = secondResponse.getResponse();
for (SearchHit hit : secondSearchResponse.getHits()) {
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("weight");
}
// 取值2
for (MultiSearchResponse.Item item : multiSearchResponse.getResponses()) {
SearchResponse response = item.getResponse();
for (SearchHit hit : response.getHits()) {
String index=hit.getIndex();
//根據不同索引名作不同的處理。
if(index.equals("user")){
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("name");
}else if(index.equals("car")){
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
String name = (String) sourceAsMap.get("weight");
}
}
}
//滾動查詢
public static void scrollSerach()throws Exception{
System.out.print("11111111111111111");
RestHighLevelClient client = new RestHighLevelClient(
RestClient.builder(new HttpHost("127.0.0.1", 9200, "http")));
SearchRequest searchRequest = new SearchRequest("user"); // 設置搜索的 index 。
QueryBuilder queryBuilder = QueryBuilders.matchAllQuery();
SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.query(queryBuilder); //設置搜索,可以是任何類型的 QueryBuilder.
//設置每次查詢數量
searchSourceBuilder.size(3);
//設置滾動等待時間
final Scroll scroll = new Scroll(TimeValue.timeValueMinutes(1));
searchRequest.scroll(scroll);
searchRequest.source(searchSourceBuilder);
//第一次獲取查詢結果
SearchResponse searchResponse = client.search(searchRequest, RequestOptions.DEFAULT);
String scrollId = searchResponse.getScrollId();
SearchHit[] searchHits = searchResponse.getHits().getHits();
for (SearchHit hit : searchHits) {
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
System.out.print("第一次獲取查詢結果,此處可做一些操作。");
String name = (String) sourceAsMap.get("name");
System.out.println("name: "+name);
}
//遍歷剩餘結果
while (searchHits != null && searchHits.length > 0) {
SearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);
scrollRequest.scroll(scroll);
searchResponse = client.scroll(scrollRequest, RequestOptions.DEFAULT);
scrollId = searchResponse.getScrollId();
searchHits = searchResponse.getHits().getHits();
for (SearchHit hit : searchHits) {
Map<String, Object> sourceAsMap = hit.getSourceAsMap();
System.out.print("遍歷剩餘結果,此處可做一些操作。");
String name = (String) sourceAsMap.get("name");
System.out.println("name: "+name);
}
}
// 清除遊標
ClearScrollRequest clearScrollRequest = new ClearScrollRequest();
clearScrollRequest.addScrollId(scrollId);
ClearScrollResponse clearScrollResponse = client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);
boolean succeeded = clearScrollResponse.isSucceeded();
client.close();
}
}