TransportClient：網上流傳最多的客戶端，目前最新版本
Java REST Client：官方推薦的客戶端，
官方：我們要在Elasticsearch 7.0的版本中不贊成使用TransportClient，在Elasticsearch 8.0的版本中完全移除TransportClient。轉而使用Java REST Client
照這個勢頭看，現在都6.5了，8.0還會遠嘛。使用客戶端要注意版本對應的問題，最好版本完全一致，實在不行也要保障主版本一致，比如5.X 、 6.X
官方文檔要對比着來看，我覺得好理解一些
* https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/index.html * https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
TransportClient
依賴
<dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>transport</artifactId> <version>6.5.0</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-core</artifactId> <version>2.11.1</version> </dependency> <dependency> <groupId>org.apache.logging.log4j</groupId> <artifactId>log4j-to-slf4j</artifactId> <version>2.11.1</version> </dependency> <dependency> <groupId>org.slf4j</groupId> <artifactId>slf4j-api</artifactId> <version>1.7.25</version> </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-client</artifactId> <version>6.5.0</version> </dependency> <dependency> <groupId>org.elasticsearch.client</groupId> <artifactId>elasticsearch-rest-high-level-client</artifactId> <version>6.5.0</version> </dependency>
連接
import org.elasticsearch.client.transport.TransportClient; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.transport.TransportAddress; import org.elasticsearch.transport.client.PreBuiltTransportClient; import java.net.InetAddress; import java.net.UnknownHostException; public class TransportClientFactory { private TransportClientFactory(){} private static class Inner{ private static final TransportClientFactory instance = new TransportClientFactory(); } public static TransportClientFactory getInstance(){ return Inner.instance; } public TransportClient getClient() throws UnknownHostException { Settings settings = Settings.builder() .put("cluster.name", "my-elasticsearch") // 默認的集羣名稱是elasticsearch，如果不是要指定 .build(); return new PreBuiltTransportClient(settings) //.addTransportAddress(new TransportAddress(InetAddress.getByName("localhost"), 9301)) .addTransportAddress(new TransportAddress(InetAddress.getByName("localhost"), 9300)); } }
Index
\1. 添加文檔
public static TransportClient addDoc1() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); // 構建對象 XContentBuilder builder = jsonBuilder() .startObject() .field("brand", "ANTA") .field("color", "red") .field("model", "S") .field("postDate", new Date()) .endObject(); // 轉成JSON格式 String json = Strings.toString(builder); /** * 參數1：index * 參數2：type * 參數3：id */ IndexRequestBuilder indexRequestBuilder = client.prepareIndex("clothes", "young", "1"); IndexResponse response = indexRequestBuilder.setSource(json, XContentType.JSON).get(); System.out.println("Index:" + response.getIndex() + "," + "Type:" + response.getType() + "," + "ID:" + response.getId() + "," + "Version:" + response.getVersion() + "," + "Status:" + response.status().name() ); return client; }
執行結果：
Index:clothes,Type:young,ID:1,Version:1,Status:CREATED
\2. 添加文檔還可以不指定id，這個時候默認生成一個唯一id
public static TransportClient addDoc2() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); // 構建對象 XContentBuilder builder = jsonBuilder() .startObject() .field("brand", "YISHION") .field("color", "Blue") .field("model", "S") .field("postDate", new Date()) .endObject(); // 轉成JSON格式 String json = Strings.toString(builder); /** * 參數1：index * 參數2：type */ IndexRequestBuilder indexRequestBuilder = client.prepareIndex("clothes", "young"); IndexResponse response = indexRequestBuilder.setSource(json, XContentType.JSON).get(); System.out.println("Index:" + response.getIndex() + "," + "Type:" + response.getType() + "," + "ID:" + response.getId() + "," + "Version:" + response.getVersion() + "," + "Status:" + response.status().name() ); return client; }
執行結果：
Index:clothes,Type:young,ID:J5uAoWcBb9TcvgEh2GJ3,Version:1,Status:CREATED
\3. 根據id獲取文檔
/** * 根據id獲取 * @throws IOException */ public static TransportClient getDoc() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); // prepareGet 參數分別爲index、type、id GetResponse response = client.prepareGet("clothes", "young", "1").get(); String id = response.getId(); String index = response.getIndex(); Map<String, DocumentField> fields = response.getFields(); // 返回的source，也就是數據源 Map<String, Object> source = response.getSource(); System.out.println("ID:" + id + ",Index:" + index); Set<String> fieldKeys = fields.keySet(); for (String s : fieldKeys){ DocumentField documentField = fields.get(s); String name = documentField.getName(); List<Object> values = documentField.getValues(); System.out.println(name + ":" + values.toString()); } System.out.println("=========="); Set<String> sourceKeys = source.keySet(); for(String s : sourceKeys){ Object o = source.get(s); System.out.println(s + ":" + o); } return client; }
執行結果：
ID:1,Index:clothes ========== color:red postDate:2018-12-12T08:18:21.509Z model:S brand:ANTA
\4. 根據id刪除，我們刪除那個默認生成的那個id
/** * 根據id刪除 * @throws IOException */ public static TransportClient delDoc() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); DeleteResponse response = client.prepareDelete("clothes", "young", "J5uAoWcBb9TcvgEh2GJ3").get(); String id = response.getId(); String index = response.getIndex(); String status = response.status().name(); System.out.println("ID:" + id + ",Index:" + index + ",Status:" + status); return client; }
執行結果：
ID:J5uAoWcBb9TcvgEh2GJ3,Index:clothes,Status:OK
\5. 根據條件刪除
/** * 根據條件刪除 * @throws IOException */ public static TransportClient delDocByQuery() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); BulkByScrollResponse response = DeleteByQueryAction.INSTANCE.newRequestBuilder(client) .filter(QueryBuilders.matchQuery("brand", "ANTA")) // 屬性-值 .source("clothes") // index .get(); long deleted = response.getDeleted(); System.out.println(deleted); return client; }
執行結果：
1
這裏第一次遇到QueryBuilders，這個東西很常用，回頭我介紹這個類。
根據條件刪除還可以指定type
DeleteByQueryRequestBuilder builder = DeleteByQueryAction.INSTANCE.newRequestBuilder(client); builder.filter(QueryBuilders.matchQuery("brand", "ANTA")) // 屬性-值 .source("clothes") // index .source().setTypes("young"); // type BulkByScrollResponse response = builder.get();
\6. 批量插入（這個bulk不僅可以批量創建，也可以更新或者刪除）
/** * 批量插入 * @throws IOException */ public static TransportClient bulkDoc() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); BulkRequestBuilder bulk = client.prepareBulk(); bulk.add(client.prepareIndex("car", "model", "1") .setSource(jsonBuilder() .startObject() .field("name", "法拉利488") .field("price", "315.50-418.80萬") .field("postDate", new Date()) .endObject() ) ); bulk.add(client.prepareIndex("car", "model", "2") .setSource(jsonBuilder() .startObject() .field("name", "法拉利LaFerrari") .field("price", "2250.00萬") .field("postDate", new Date()) .endObject() ) ); bulk.add(client.prepareIndex("car", "model", "3") .setSource(jsonBuilder() .startObject() .field("name", "法拉利GTC4Lusso") .field("price", "322.80-485.80萬") .field("postDate", new Date()) .endObject() ) ); BulkResponse responses = bulk.get(); String status = responses.status().name(); System.out.println(status); return client; }
執行結果：
OK
\7. 獲取多個結果
/** * 批量獲取 * @throws IOException */ public static TransportClient multiGetDoc() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); MultiGetResponse multiGetItemResponses = client.prepareMultiGet() // 可以指定多個index，多個id .add("clothes", "young", "1", "2") .add("car", "model", "1","2","3") .get(); for (MultiGetItemResponse itemResponse : multiGetItemResponses) { GetResponse response = itemResponse.getResponse(); if (response.isExists()) { String json = response.getSourceAsString(); System.out.println(json); } } return client; }
執行結果：由於之前clothes裏面沒數據了，所以只顯示了下面三條數據
{"name":"法拉利488","price":"315.50-418.80萬","postDate":"2018-12-12T08:38:09.107Z"} {"name":"法拉利LaFerrari","price":"2250.00萬","postDate":"2018-12-12T08:38:09.129Z"} {"name":"法拉利GTC4Lusso","price":"322.80-485.80萬","postDate":"2018-12-12T08:38:09.129Z"}
\8. 更新
/** * 更新方式一：通過UpdateRequest * @throws IOException * @throws ExecutionException * @throws InterruptedException */ public static TransportClient updateDoc1() throws IOException, ExecutionException, InterruptedException { TransportClient client = TransportClientFactory.getInstance().getClient(); UpdateRequest updateRequest = new UpdateRequest(); updateRequest.index("car"); // 指定index updateRequest.type("model");// 指定type updateRequest.id("3"); // 指定id // 更新內容 updateRequest.doc(jsonBuilder() .startObject() .field("name", "Aventador") .field("price", "630.00-755.94萬") .field("postDate", new Date()) .field("extra", "Extra Data") // 不存在的會自動添加 .endObject()); UpdateResponse updateResponse = client.update(updateRequest).get(); System.out.println(updateResponse.status().name()); return client; }
執行結果：
OK
在Kibana上查看結果：GET /car/model/3
客戶端有兩種請求方式，一種是Request（比如UpdateRequest ），另一種是prepare（比如prepareUpdate），我更喜歡用prepare***
/** * 更新方式二：通過prepareUpdate * @throws IOException */ public static TransportClient updateDoc2() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); client.prepareUpdate("car", "model", "1") .setDoc(jsonBuilder() .startObject() .field("name", "法拉利812 Superfast") .field("price", "498.80萬") .field("postDate", new Date()) .endObject() ) .get(); return client; }
\9. upset更新
/** * 文檔存在則更新doc，不存在則添加upsert * @throws IOException * @throws ExecutionException * @throws InterruptedException */ public static TransportClient upsert() throws IOException, ExecutionException, InterruptedException { TransportClient client = TransportClientFactory.getInstance().getClient(); IndexRequest indexRequest = new IndexRequest("clothes", "young", "3") .source(jsonBuilder() .startObject() .field("brand", "Pierre Cardin") .field("color", "Black") .field("model", "L") .field("postDate", new Date()) .endObject()); UpdateRequest updateRequest = new UpdateRequest("clothes", "young", "3") .doc(jsonBuilder() .startObject() .field("model", "XL") .endObject()) .upsert(indexRequest); UpdateResponse response = client.update(updateRequest).get(); System.out.println(response.status().name()); return client; }
什麼意思呢，如果文檔存在，則只更新model字段，相反會添加IndexRequest裏面的內容。
第一次執行：（文檔不存在）
CREATED
GET /clothes/young/3
第二次執行：
OK
查看Kibana
\10. bulkProcessor 另外一個批量工具
基本的配置
.setBulkActions(10000) // 每10000個request，bulk一次 .setBulkSize(new ByteSizeValue(5, ByteSizeUnit.MB)) // 每5M的數據刷新一次 .setFlushInterval(TimeValue.timeValueSeconds(5)) // 每5s刷新一次，而不管有多少數據量 .setConcurrentRequests(0) // 設置併發請求的數量。值爲0意味着只允許執行一個請求。值爲1意味着在積累新的批量請求時允許執行1個併發請求。 .setBackoffPolicy( // 設置一個自定義的重試策略，該策略最初將等待100毫秒，按指數增長，最多重試3次。當一個或多個批量項請求失敗時，如果出現EsRejectedExecutionException異常，將嘗試重試，該異常表明用於處理請求的計算資源太少。要禁用backoff，請傳遞BackoffPolicy.noBackoff()。 BackoffPolicy.exponentialBackoff(TimeValue.timeValueMillis(100), 3))
測試
/** * 造數據 * @throws IOException */ public static TransportClient scrollSearchPreData() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); BulkProcessor bulkProcessor = BulkProcessor.builder( client, new BulkProcessor.Listener() { @Override public void beforeBulk(long executionId, BulkRequest request) { // bulk 執行之前 System.out.println("beforeBulk-----" + request.getDescription()); } @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) { // bulk 執行之後 System.out.println("afterBulk------" + request.getDescription() + ",是否有錯誤：" + response.hasFailures()); } @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) { //bulk 失敗 System.out.println("報錯-----" + request.getDescription() + "," + failure.getMessage()); } }) .setBulkActions(100) // 每100個request，bulk一次 .setConcurrentRequests(0) // 設置併發請求的數量。值爲0意味着只允許執行一個請求。值爲1意味着在積累新的批量請求時允許執行1個併發請求。 .build(); Random random = new Random(); for (int i = 1; i <= 1000; i++){ bulkProcessor.add(new IndexRequest("book", "elasticsearch", i+"").source(jsonBuilder() .startObject() .field("name", "book_" + i) .field("price", random.nextDouble()*1000) .field("postDate", new Date()) .endObject())); } bulkProcessor.flush(); bulkProcessor.close(); return client; }
執行結果：1000條數據，bulk10次
beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false beforeBulk-----requests[100], indices[book] afterBulk------requests[100], indices[book],是否有錯誤：false
\11. scroll（讓數據都滾出來）
/** * 當搜索請求返回一個結果的“頁面”時，滾動API可以用於從一個搜索請求檢索大量的結果(甚至所有結果) * 其方式與在傳統數據庫中使用遊標非常類似。滾動不是爲了實時的用戶請求，而是爲了處理大量的數據 * @throws UnknownHostException */ public static TransportClient scrollSearch() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse response = client.prepareSearch("book") .addSort("price", SortOrder.ASC) .setScroll(new TimeValue(30000)) .setSize(1000).get(); // 每次滾出1000條就返回 do { System.out.println("========Begin======="); for (SearchHit hit : response.getHits().getHits()) { System.out.println(hit.getSourceAsString()); } System.out.println("========End======="); response = client.prepareSearchScroll(response.getScrollId()).setScroll(new TimeValue(30000)).execute().actionGet(); } while(response.getHits().getHits().length != 0); return client; }
執行結果：
========Begin======= {"name":"book_233","price":0.7903630819869889,"postDate":"2018-12-12T09:27:32.629Z"} {"name":"book_46","price":1.9862330698061648,"postDate":"2018-12-12T09:27:30.722Z"} {"name":"book_18","price":2.8024592316934216,"postDate":"2018-12-12T09:27:30.721Z"} {"name":"book_512","price":3.5739663933835875,"postDate":"2018-12-12T09:27:33.275Z"} {"name":"book_275","price":5.449351054677254,"postDate":"2018-12-12T09:27:32.632Z"} {"name":"book_112","price":8.035476335226166,"postDate":"2018-12-12T09:27:32.424Z"} ...此處省略 ========End=======
\12. 根據查詢更新
/** * 當版本匹配時，updateByQuery更新文檔並增加版本號。 * 所有更新和查詢失敗都會導致updateByQuery中止。這些故障可從BulkByScrollResponse#getBulkFailures方法中獲得。 * 任何成功的更新都會保留並不會回滾。當第一個失敗導致中止時，響應包含由失敗的批量請求生成的所有失敗。 * 當文檔在快照時間和索引請求過程時間之間發生更改時，就會發生版本衝突 * 爲了防止版本衝突導致updateByQuery中止，設置abortOnVersionConflict(false)。 * ScriptType.INLINE:在大量查詢中指定內聯腳本並動態編譯。它們將基於腳本的lang和代碼進行緩存。 * ScriptType.STORED:存儲的腳本作爲{@link org.elasticsearch.cluster.ClusterState}的一部分保存基於用戶請求。它們將在查詢中首次使用時被緩存。 * https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-docs-update-by-query.html * https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-update-by-query.html * @throws UnknownHostException */ public static TransportClient updateByQuery() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); UpdateByQueryRequestBuilder updateByQuery = UpdateByQueryAction.INSTANCE.newRequestBuilder(client); updateByQuery.source("book") .size(100) // 嘗試獲取的最大文檔數 .filter(QueryBuilders.termsQuery("name","book_233", "book_46", "book_18", "book_512")) // 注意term，value要變成小寫！！ // 以下腳本：保留id=781的，刪除id=316的，其它的價格都變爲79 .script(new Script( ScriptType.INLINE,Script.DEFAULT_SCRIPT_LANG, "if (ctx._source['id'] == 781) {" + " ctx.op='noop'" // ctx.op='noop' 不做處理 + "} else if (ctx._source['id'] == '316') {" + " ctx.op='delete'" // ctx.op='delete'刪除 + "} else {" + "ctx._source['price'] = 79}", Collections.emptyMap())) .abortOnVersionConflict(false); // 版本衝突策略：abortOnVersionConflict 版本衝突時不終止 // .source().setTypes("young") // 指定type // .setSize(10) // 返回搜索的命中數 // .addSort("postDate", SortOrder.DESC); BulkByScrollResponse response = updateByQuery.get(); System.out.println("Deleted:" + response.getDeleted() + ",Created:" + response.getCreated() + ",Updated:" + response.getUpdated() + ",Noops:" + response.getNoops()); List<BulkItemResponse.Failure> failures = response.getBulkFailures(); System.out.println(failures.size()); // 如果目標值是Cat，更新內容也是Cat，則不會去更新 return client; }
執行結果：（這個term查詢有點坑，value必須爲小寫，並且不能帶-，我之前生成的格式爲book-100，結果查詢不出來。。。）
Deleted:0,Created:0,Updated:4,Noops:0 0
查看數據：
GET /book/elasticsearch/_mget { "ids" : ["", "", "", ""] }
結果：
{ "docs" : [ { "_index" : "book", "_type" : "elasticsearch", "_id" : "", "_version" : , "found" : true, "_source" : { "price" : , "name" : "book_233", "postDate" : "2018-12-12T09:27:32.629Z" } }, { "_index" : "book", "_type" : "elasticsearch", "_id" : "", "_version" : , "found" : true, "_source" : { "price" : , "name" : "book_46", "postDate" : "2018-12-12T09:27:30.722Z" } }, { "_index" : "book", "_type" : "elasticsearch", "_id" : "", "_version" : , "found" : true, "_source" : { "price" : , "name" : "book_18", "postDate" : "2018-12-12T09:27:30.721Z" } }, { "_index" : "book", "_type" : "elasticsearch", "_id" : "", "_version" : , "found" : true, "_source" : { "price" : , "name" : "book_512", "postDate" : "2018-12-12T09:27:33.275Z" } } ] }
\13. 簡單查詢
/** * 簡單查詢【通配符查詢，篩選價格範圍，設定返回數量，排序】 * @throws UnknownHostException */ public static TransportClient search() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse response = client.prepareSearch("book") // index,可以多個 .setSearchType(SearchType.DFS_QUERY_THEN_FETCH) .setQuery(QueryBuilders.wildcardQuery("name", "*book_1*")) // Query .setPostFilter(QueryBuilders.rangeQuery("price").from(800).to(900)) // Filter .setFrom(0).setSize(100).setExplain(true).addSort("postDate", SortOrder.DESC) .get(); response.getHits().forEach(e ->{ System.out.println(e.getSourceAsString()); }); return client; }
執行結果：
{"name":"book_1000","price":811.3812414198577,"postDate":"2018-12-12T09:27:34.095Z"} {"name":"book_194","price":828.6484294585816,"postDate":"2018-12-12T09:27:32.433Z"} {"name":"book_171","price":839.1475764183831,"postDate":"2018-12-12T09:27:32.432Z"} {"name":"book_170","price":869.7835076374234,"postDate":"2018-12-12T09:27:32.431Z"} {"name":"book_161","price":838.5131747806441,"postDate":"2018-12-12T09:27:32.429Z"} {"name":"book_153","price":805.041724108352,"postDate":"2018-12-12T09:27:32.429Z"} {"name":"book_154","price":893.982844708382,"postDate":"2018-12-12T09:27:32.429Z"} {"name":"book_105","price":883.039302643907,"postDate":"2018-12-12T09:27:32.424Z"} {"name":"book_19","price":877.0523728410054,"postDate":"2018-12-12T09:27:30.721Z"}
\14. 多個查詢MultiSearch
/** * 多個查詢 */ public static TransportClient multiSearch() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); // 第一個查詢 SearchRequestBuilder srb1 = client .prepareSearch("book") .setQuery(QueryBuilders.queryStringQuery("book_9*").field("name")) .setFrom(0) // 開始位置 .setSize(10); // 設置返回的最大條數 // 第二個查詢 SearchRequestBuilder srb2 = client .prepareSearch("car") .setQuery(QueryBuilders.queryStringQuery("*r*")) .setSize(10); // 組合 MultiSearchResponse sr = client.prepareMultiSearch() .add(srb1) .add(srb2) .get(); // You will get all individual responses from MultiSearchResponse#getResponses() long nbHits = 0; for (MultiSearchResponse.Item item : sr.getResponses()) { SearchResponse response = item.getResponse(); response.getHits().forEach(e ->{ System.out.println(e.getSourceAsString()); }); long hits = response.getHits().getTotalHits(); System.out.println("Hits:" + hits); nbHits += hits; } System.out.println("Total:" + nbHits); return client; }
執行結果：
{"name":"book_92","price":176.35847694096162,"postDate":"2018-12-12T09:27:30.724Z"} {"name":"book_98","price":611.4318589503413,"postDate":"2018-12-12T09:27:30.724Z"} {"name":"book_99","price":214.4653626273969,"postDate":"2018-12-12T09:27:30.724Z"} {"name":"book_900","price":973.3382073380857,"postDate":"2018-12-12T09:27:33.892Z"} {"name":"book_915","price":35.30856326485343,"postDate":"2018-12-12T09:27:34.091Z"} {"name":"book_922","price":299.58144612743064,"postDate":"2018-12-12T09:27:34.091Z"} {"name":"book_930","price":591.6598815227311,"postDate":"2018-12-12T09:27:34.092Z"} {"name":"book_933","price":287.18727780940037,"postDate":"2018-12-12T09:27:34.092Z"} {"name":"book_935","price":693.6036227965725,"postDate":"2018-12-12T09:27:34.092Z"} {"name":"book_942","price":701.4129722487066,"postDate":"2018-12-12T09:27:34.092Z"} Hits:111 {"name":"法拉利LaFerrari","price":"2250.00萬","postDate":"2018-12-12T08:38:09.129Z"} {"name":"Aventador","price":"630.00-755.94萬","postDate":"2018-12-12T08:49:01.736Z","extra":"Extra Data"} Hits:2 Total:113
聚合
\15. 聚合查詢
/** * 聚合查詢 * 搜索是查找某些具體的文檔.然而聚合就是對這些搜索到的文檔進行統計 * https://www.elastic.co/guide/en/elasticsearch/reference/current/search-aggregations.html * https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-aggs.html * https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/java-search-aggs.html * 可以在聚合中定義子聚合 * @return * @throws UnknownHostException */ public static TransportClient aggregationsSearch() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse sr = client.prepareSearch("book") .setQuery(QueryBuilders.matchAllQuery()) .addAggregation( AggregationBuilders.stats("agg1").field("price") ) .addAggregation( AggregationBuilders.dateHistogram("agg2") .field("postDate") .dateHistogramInterval(DateHistogramInterval.YEAR) ) .get(); // Short version of execute().actionGet(). // Get your facet results Aggregation agg1 = sr.getAggregations().get("agg1"); System.out.println(agg1.getClass()); // class org.elasticsearch.search.aggregations.metrics.stats.InternalStats Aggregation agg2 = sr.getAggregations().get("agg2"); System.out.println(agg2.getClass()); // class org.elasticsearch.search.aggregations.bucket.histogram.InternalDateHistogram return client; }
15.1 metrics聚合
/** * metrics聚合 * 主要爲了統計信息 * org.elasticsearch.search.aggregations.metrics.percentiles.Percentiles * org.elasticsearch.search.aggregations.metrics.percentiles.PercentileRanks * org.elasticsearch.search.aggregations.metrics.cardinality.Cardinality * 地理位置聚合：https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/_metrics_aggregations.html#java-aggs-metrics-geobounds * https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/_metrics_aggregations.html#java-aggs-metrics-tophits * https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/_metrics_aggregations.html#java-aggs-metrics-scripted-metric * @return * @throws UnknownHostException */ public static TransportClient metricsAggregationsSearch() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse sr = client.prepareSearch("book") .setQuery(QueryBuilders.matchAllQuery()) .addAggregation( AggregationBuilders.min("agg1").field("price") ) .addAggregation( AggregationBuilders.max("agg2").field("price") ) .addAggregation( AggregationBuilders.sum("agg3").field("price") ) .addAggregation( AggregationBuilders.avg("agg4").field("price") ) .addAggregation( AggregationBuilders.count("agg5").field("price") ) .addAggregation( AggregationBuilders.stats("agg6").field("price") ) .get(); Min agg1 = sr.getAggregations().get("agg1"); Max agg2 = sr.getAggregations().get("agg2"); Sum agg3 = sr.getAggregations().get("agg3"); Avg agg4 = sr.getAggregations().get("agg4"); ValueCount agg5 = sr.getAggregations().get("agg5"); Stats agg6 = sr.getAggregations().get("agg6"); System.out.println("Min:" + agg1.getValue() + ",Max:" + agg2.getValue() + ",Sum:" + agg3.getValue() + ",Avg:" + agg4.getValue() + ",Count:" + agg5.getValue() + ",Stats:(" + agg6.getMin() + "," + agg6.getMax() + "," + agg6.getSum() + "," + agg6.getAvg() + "," + agg6.getCount() + ")"); return client; }
執行結果：
Min:5.449350833892822,Max:999.3211669921875,Sum:502966.58267736435,Avg:502.96658267736433,Count:1000,Stats:(5.449350833892822,999.3211669921875,502966.58267736435,502.96658267736433,1000)
15.2地理位置聚合（計算座標的左上/右下邊界值）
/** * 準備地理位置信息 * @return * @throws IOException */ public static TransportClient geoSearchPreData() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); // 建立索引 CreateIndexResponse indexResponse = client.admin().indices().prepareCreate("area") .setSettings(Settings.builder() .put("index.number_of_shards", 1) // 分片 .put("index.number_of_replicas", 1) // 副本 ) .addMapping("hospital", "message", "type=text", "location", "type=geo_point") .get(); System.out.println("Index:" + indexResponse.index() + ",ACK:" + indexResponse.isAcknowledged()); BulkProcessor bulkProcessor = BulkProcessor.builder( client, new BulkProcessor.Listener() { @Override public void beforeBulk(long executionId, BulkRequest request) { // bulk 執行之前 System.out.println("beforeBulk-----" + request.getDescription()); } @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) { // bulk 執行之後 System.out.println("afterBulk------" + request.getDescription() + ",hasFailures：" + response.hasFailures()); } @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) { //bulk 失敗 System.out.println("報錯-----" + request.getDescription() + "," + failure.getMessage()); } }) .setBulkActions(100) // 每100個request，bulk一次 .setConcurrentRequests(0) // 設置併發請求的數量。值爲0意味着只允許執行一個請求。值爲1意味着在積累新的批量請求時允許執行1個併發請求。 .build(); Random random = new Random(); for (int i = 1; i <= 200; i++){ String lo = new DecimalFormat("#.############").format(random.nextDouble() * 100); String la = new DecimalFormat("#.############").format(random.nextDouble() * 100); bulkProcessor.add(new IndexRequest("area", "hospital", i+"").source(jsonBuilder() .startObject() .field("name", "hospital-" + i) .field("location", lo + "," + la) .endObject())); } bulkProcessor.flush(); bulkProcessor.close(); return client; } /** * 地理信息查詢 * @return * @throws UnknownHostException */ public static TransportClient geoAggregation() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse sr = client.prepareSearch("area") .setQuery(QueryBuilders.matchQuery("name", "hospital-1")) .addAggregation( AggregationBuilders.geoBounds("agg").field("location").wrapLongitude(true) ) .get(); GeoBounds agg = sr.getAggregations().get("agg"); GeoPoint left = agg.topLeft(); GeoPoint right = agg.bottomRight(); System.out.println(left + " | " + right); return client; }
執行結果：
89.9911705031991, 0.03342803567647934 | 0.049703302793204784, 99.9249867349863
15.3桶聚合
/** * 桶聚合,我這裏只列舉了部分 * https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/_bucket_aggregations.html * @return * @throws UnknownHostException */ public static TransportClient bucketAggregationsSearch() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse sr = client.prepareSearch() .setQuery(QueryBuilders.matchAllQuery()) // .addAggregation(AggregationBuilders // .global("agg0") // .subAggregation(AggregationBuilders.terms("sub_agg").field("name")) // ) .addAggregation(AggregationBuilders .filter("agg1", QueryBuilders.termQuery("name", "book_199"))) .addAggregation(AggregationBuilders .filters("agg2", new FiltersAggregator.KeyedFilter("key1", QueryBuilders.termQuery("name", "book_1")), new FiltersAggregator.KeyedFilter("key2", QueryBuilders.termQuery("name", "book_52")) )) .get(); // Global agg0 = sr.getAggregations().get("agg0"); // System.out.println("GlobalCount:" + agg0.getDocCount()); Filter agg1 = sr.getAggregations().get("agg1"); System.out.println("FilterCount:" + agg1.getDocCount()); Filters agg2 = sr.getAggregations().get("agg2"); for (Filters.Bucket entry : agg2.getBuckets()) { String key = entry.getKeyAsString(); // bucket key long docCount = entry.getDocCount(); // Doc count System.out.println("key [" + key + "], doc_count ["+ docCount +"]"); } return client; }
執行結果：Global會屏蔽其它的Agg
FilterCount:1 key [key1], doc_count [1] key [key2], doc_count [1]
查詢DSL
16. Query DSL
16.1 MatchAll，最簡單的查詢，它會匹配所有文檔
client.prepareSearch().setQuery(QueryBuilders.matchAllQuery());
16.2 全文檢索【高級全文查詢通常用於在全文字段(如電子郵件正文)上運行全文查詢，在執行之前有分析的過程。】
16.2.1 Match Query（全文查詢的標準查詢，包括模糊匹配和短語或鄰近查詢）
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("book") .setQuery(QueryBuilders .matchQuery("name", "book_1") .fuzziness(Fuzziness.AUTO) // 模糊查詢 .zeroTermsQuery(MatchQuery.ZeroTermsQuery.ALL) // 與MatchAll等價，匹配所有文檔。默認none，不匹配任何文檔 ).get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：（爲什麼會命中250條呢？這是因爲模糊查詢，如果你註釋掉模糊查詢，就只會查到一條）
{"name":"book_1","price":541.5683324629698,"postDate":"2018-12-12T09:27:30.695Z"} {"name":"book_2","price":859.0268161692424,"postDate":"2018-12-12T09:27:30.720Z"} {"name":"book_4","price":666.0331749730802,"postDate":"2018-12-12T09:27:30.720Z"} {"name":"book_6","price":797.3826369337273,"postDate":"2018-12-12T09:27:30.720Z"} {"name":"book_15","price":764.0761667524818,"postDate":"2018-12-12T09:27:30.721Z"} {"name":"book_51","price":969.2863955131567,"postDate":"2018-12-12T09:27:30.722Z"} {"name":"book_3","price":467.29468328850055,"postDate":"2018-12-12T09:27:30.720Z"} {"name":"book_11","price":365.2274741512962,"postDate":"2018-12-12T09:27:30.720Z"} {"name":"book_17","price":498.8900836459158,"postDate":"2018-12-12T09:27:30.721Z"} {"name":"book_31","price":377.2822748558652,"postDate":"2018-12-12T09:27:30.721Z"} 命中：250
16.2.2 Multi Match Query（標準查詢的多字段版本）
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() // 關鍵字Aventador，匹配多個字段*ame、brand。字段名稱可以使用通配符 .setQuery(QueryBuilders.multiMatchQuery("Aventador", "*ame","brand")) .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
{"name":"Aventador","price":"630.00-755.94萬","postDate":"2018-12-12T08:49:01.736Z","extra":"Extra Data"} 命中：1
16.2.3 Common Terms Query（一個更專業的查詢，偏好不常見的關鍵字）
...待補充
16.2.4 Query String Query（解析輸入並圍繞操作符拆分文本，每個文本部分都是獨立分析的）
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() // 關鍵字和字段【均可以】可以使用通配符（?匹配一個字符，*匹配0個或多個字符，AND，OR）等等 // 有一些您不希望作爲操作符的必須轉義處理：+ - = && || > < ! ( ) { } [ ] ^ " ~ * ? : \ / //.setQuery(QueryBuilders.queryStringQuery("(book_111) OR (book_999)")) // or //.setQuery(QueryBuilders.queryStringQuery("(book_111) AND (book_999)")) // AND //.setQuery(QueryBuilders.queryStringQuery("(book_111) && (book_999)")) // AND與&&等價 //.setQuery(QueryBuilders.queryStringQuery("(book_111) & (book_999)")) // &不會短路計算 //.setQuery(QueryBuilders.queryStringQuery("book_1?1").field("name")) // ? 並且指定字段 //.setQuery(QueryBuilders.queryStringQuery("name:book_1?1 OR color:B*")) // 在查詢裏指定字段 //.setQuery(QueryBuilders.queryStringQuery("name:book_1?1 | color:B*")) //.setQuery(QueryBuilders.queryStringQuery("name:book_1?1 || color:B*")) // OR與||等價 //.setQuery(QueryBuilders.queryStringQuery("price:[990 TO *]")) // 範圍查詢 // 默認情況下操作符都是可選的，有兩個特殊的->首選操作符是：+(這一項必須存在)和-(這一項必須不存在) .setQuery(QueryBuilders.queryStringQuery("price:[990 TO *] -book*")) // 不顯示book*的數據 .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
{"name":"法拉利LaFerrari","price":"2250.00萬","postDate":"2018-12-12T08:38:09.129Z"} {"name":"法拉利488","price":"315.50-418.80萬","postDate":"2018-12-12T08:38:09.107Z"} {"name":"Aventador","price":"630.00-755.94萬","postDate":"2018-12-12T08:49:01.736Z","extra":"Extra Data"} 命中：3
16.2.5 Simple Query String Query（查詢永遠不會拋出異常，並丟棄查詢的無效部分）
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("book") // + 表示與操作 // | 表示或操作 // - 表示否定 // * 在關鍵字末尾表示前綴查詢 // 小括號()表示優先級 // 這裏通配符失效！！比如book_1?1或者book_1*1只會查出一條記錄 .setQuery(QueryBuilders.simpleQueryStringQuery("book_11*") // .flags(SimpleQueryStringFlag.AND,SimpleQueryStringFlag.OR,SimpleQueryStringFlag.NOT) // 指定啓用哪些解析功能，默認全部啓用ALL。SimpleQueryStringFlag ) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
這個功能感覺有點雞肋，不如QueryStringQuery
執行結果：
{"name":"book_110","price":562.7866825701938,"postDate":"2018-12-12T09:27:32.424Z"} {"name":"book_116","price":764.6107139750268,"postDate":"2018-12-12T09:27:32.425Z"} {"name":"book_119","price":612.4310597411385,"postDate":"2018-12-12T09:27:32.425Z"} {"name":"book_113","price":693.0418946151275,"postDate":"2018-12-12T09:27:32.425Z"} {"name":"book_117","price":955.1701411798869,"postDate":"2018-12-12T09:27:32.425Z"} {"name":"book_114","price":255.65009974873198,"postDate":"2018-12-12T09:27:32.425Z"} {"name":"book_118","price":915.8228473795552,"postDate":"2018-12-12T09:27:32.425Z"} {"name":"book_111","price":185.85669328040666,"postDate":"2018-12-12T09:27:32.424Z"} {"name":"book_112","price":8.035476335226166,"postDate":"2018-12-12T09:27:32.424Z"} {"name":"book_115","price":651.4301559069825,"postDate":"2018-12-12T09:27:32.425Z"} {"name":"book_11","price":365.2274741512962,"postDate":"2018-12-12T09:27:30.720Z"} 命中：11
16.3 Term level queries【查詢通常用於數字、日期和枚舉等結構化數據，而不是全文字段。或者，它們允許您在分析過程之前處理低級查詢】
造一些結構化數據
/** * 準備結構化數據 * @return * @throws IOException */ public static TransportClient structuredData() throws IOException { TransportClient client = TransportClientFactory.getInstance().getClient(); AdminClient admin = client.admin(); IndicesAdminClient indicesAdminClient = admin.indices(); // 指數管理 CreateIndexResponse createIndexResponse = indicesAdminClient.prepareCreate("school") .setSettings(Settings.builder() .put("index.number_of_shards", 1) // 分片 .put("index.number_of_replicas", 1) // 副本 ) .addMapping("student", "sName", "type=text", "sAge", "type=integer", "sClass", "type=keyword", "sTime", "type=date") // mapping .get(); System.out.println("創建索引：" + createIndexResponse.isAcknowledged()); BulkProcessor bulkProcessor = BulkProcessor.builder( client, new BulkProcessor.Listener() { @Override public void beforeBulk(long executionId, BulkRequest request) { // bulk 執行之前 System.out.println("beforeBulk-----" + request.getDescription()); } @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) { // bulk 執行之後 System.out.println("afterBulk------" + request.getDescription() + ",是否有錯誤：" + response.hasFailures()); } @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) { //bulk 失敗 System.out.println("報錯-----" + request.getDescription() + "," + failure.getMessage()); } }) .setBulkActions(300) // 每300個request，bulk一次 .setConcurrentRequests(0) // 設置併發請求的數量。值爲0意味着只允許執行一個請求。值爲1意味着在積累新的批量請求時允許執行1個併發請求。 .build(); Random random = new Random(); for (int i = 1; i <= 1000; i++){ bulkProcessor.add(new IndexRequest("school", "student", i+"").source(jsonBuilder() .startObject() .field("sName", "Student_" + i) .field("sAge", random.nextInt(100)) .field("sClass", "Class_" + (i % 20)) .field("sTime", new Date()) .endObject())); } bulkProcessor.flush(); bulkProcessor.close(); return client; }
查看索引數據：GET /school/_search
注意我這裏面有大寫字母
查看索引信息：GET /school
16.3.1 Term Query
這裏有很多彩蛋~
下面我們查詢class字段
代碼：
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") //.setQuery(QueryBuilders.termQuery("sName", "student_1")) // 使用term查詢類型爲text的字段的時候，內容要全部小寫 .setQuery(QueryBuilders.termQuery("sClass", "Class_2")) // 使用term查詢類型爲keyword的字段的時候，內容區分大小寫 .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
爲什麼會出現上述彩蛋呢？
類型爲text，會解析其中的內容，比如 "Hey Guy"，倒排索引會包括以下術語[hey,guy]
類型爲keyword，在倒排索引中會出現以下術語[Hey Guy]
所以，查詢全文字段就用match，因爲它知道如何分析，term比較傻，只知道精確值，擅長結構化的字段。
16.3.2 TermsQuery
多個查詢關鍵字
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // 一個字段，多個value .setQuery(QueryBuilders.termsQuery("sName","student_1","student_2")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
{"sName":"Student_1","sAge":28,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.666Z"} {"sName":"Student_2","sAge":20,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.689Z"} 命中：2
16.3.3 RangeQuery 範圍查詢
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") .setQuery(QueryBuilders.rangeQuery("sAge") // .gte(10) // 大於等於 // .gt(10) // 大於 // .lte(20) // 小於等於 // .lt(20) // 小於 .from(0) .to(5) .includeLower(true) // 包含下限 .includeUpper(true) // 包含上限 ) // .setQuery(QueryBuilders.rangeQuery("sTime") // .lte("now") // 小於等於當前時間 // .timeZone("-01:00")) // 時區 .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
...省略 {"sName":"Student_340","sAge":5,"sClass":"Class_0","sTime":"2018-12-13T08:39:57.821Z"} {"sName":"Student_372","sAge":1,"sClass":"Class_12","sTime":"2018-12-13T08:39:57.822Z"} {"sName":"Student_428","sAge":2,"sClass":"Class_8","sTime":"2018-12-13T08:39:57.823Z"} {"sName":"Student_432","sAge":0,"sClass":"Class_12","sTime":"2018-12-13T08:39:57.823Z"} 命中：52
16.3.4 ExistQuery
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") .setQuery(QueryBuilders.existsQuery("sName")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
...省略 {"sName":"Student_18","sAge":72,"sClass":"Class_18","sTime":"2018-12-13T08:39:57.690Z"} {"sName":"Student_19","sAge":43,"sClass":"Class_19","sTime":"2018-12-13T08:39:57.690Z"} {"sName":"Student_20","sAge":93,"sClass":"Class_0","sTime":"2018-12-13T08:39:57.690Z"} 命中：1000
全命中了，什麼意思呢？
這個方法的作用：對於某個字段，返回至少含有一個非空值的所有文檔 。
套用官方解釋，比如：查詢user字段，可以返回下面格式的數據【有值，空字符串，長度大於0的數組】
然而下面這些就不行了【null，空數組，非查詢字段】
16.3.5 Prefix Query 前綴查詢（顧名思義，肯定是把關鍵字作爲前綴啦）
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") .setQuery(QueryBuilders.prefixQuery("sName", "student_19")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
{"sName":"Student_19","sAge":43,"sClass":"Class_19","sTime":"2018-12-13T08:39:57.690Z"} {"sName":"Student_190","sAge":53,"sClass":"Class_10","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_191","sAge":45,"sClass":"Class_11","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_192","sAge":2,"sClass":"Class_12","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_193","sAge":9,"sClass":"Class_13","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_194","sAge":67,"sClass":"Class_14","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_195","sAge":62,"sClass":"Class_15","sTime":"2018-12-13T08:39:57.696Z"} {"sName":"Student_196","sAge":77,"sClass":"Class_16","sTime":"2018-12-13T08:39:57.696Z"} {"sName":"Student_197","sAge":82,"sClass":"Class_17","sTime":"2018-12-13T08:39:57.696Z"} {"sName":"Student_198","sAge":38,"sClass":"Class_18","sTime":"2018-12-13T08:39:57.696Z"} {"sName":"Student_199","sAge":50,"sClass":"Class_19","sTime":"2018-12-13T08:39:57.696Z"} 命中：11
16.3.6 Wildcard Query
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // 支持通配符查詢[?,*]，不建議以通配符開頭，那樣會很慢 .setQuery(QueryBuilders.wildcardQuery("sName", "student_1?1")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
{"sName":"Student_101","sAge":87,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.692Z"} {"sName":"Student_111","sAge":98,"sClass":"Class_11","sTime":"2018-12-13T08:39:57.693Z"} {"sName":"Student_121","sAge":96,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.693Z"} {"sName":"Student_131","sAge":89,"sClass":"Class_11","sTime":"2018-12-13T08:39:57.694Z"} {"sName":"Student_141","sAge":90,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.694Z"} {"sName":"Student_151","sAge":52,"sClass":"Class_11","sTime":"2018-12-13T08:39:57.694Z"} {"sName":"Student_161","sAge":21,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_171","sAge":76,"sClass":"Class_11","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_181","sAge":18,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_191","sAge":45,"sClass":"Class_11","sTime":"2018-12-13T08:39:57.695Z"} 命中：10
16.3.7 Regexp Query
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // regexp查詢的性能在很大程度上取決於所選擇的正則表達式。匹配諸如.*之類的所有內容非常慢。*?+ 主要降低性能 .setQuery(QueryBuilders.regexpQuery("sName", "student_.*0")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
...省略 {"sName":"Student_170","sAge":14,"sClass":"Class_10","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_180","sAge":16,"sClass":"Class_0","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_190","sAge":53,"sClass":"Class_10","sTime":"2018-12-13T08:39:57.695Z"} {"sName":"Student_200","sAge":12,"sClass":"Class_0","sTime":"2018-12-13T08:39:57.696Z"} 命中：100
16.3.8 Fuzzy Query
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // 模糊查詢https://www.elastic.co/guide/en/elasticsearch/reference/current/common-options.html#fuzziness .setQuery(QueryBuilders.fuzzyQuery("sName", "student_33")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
...省略 {"sName":"Student_83","sAge":94,"sClass":"Class_3","sTime":"2018-12-13T08:39:57.692Z"} {"sName":"Student_93","sAge":88,"sClass":"Class_13","sTime":"2018-12-13T08:39:57.692Z"} {"sName":"Student_133","sAge":34,"sClass":"Class_13","sTime":"2018-12-13T08:39:57.694Z"} {"sName":"Student_233","sAge":90,"sClass":"Class_13","sTime":"2018-12-13T08:39:57.696Z"} 命中：50
16.3.9 Type Query
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") .setQuery(QueryBuilders.typeQuery("student")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
就是查詢type下的內容..
...省略 {"sName":"Student_18","sAge":72,"sClass":"Class_18","sTime":"2018-12-13T08:39:57.690Z"} {"sName":"Student_19","sAge":43,"sClass":"Class_19","sTime":"2018-12-13T08:39:57.690Z"} {"sName":"Student_20","sAge":93,"sClass":"Class_0","sTime":"2018-12-13T08:39:57.690Z"} 命中：1000
16.3.10 Ids Query
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // 根據多個id查詢 .setQuery(QueryBuilders.idsQuery("student").addIds("1","111","999")) .setSize(20) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println(e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
{"sName":"Student_1","sAge":28,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.666Z"} {"sName":"Student_111","sAge":98,"sClass":"Class_11","sTime":"2018-12-13T08:39:57.693Z"} {"sName":"Student_999","sAge":6,"sClass":"Class_19","sTime":"2018-12-13T08:39:58.120Z"} 命中：3
17. Compound queries（複合查詢）
17.1 constant_score query
比如查詢好喫的水果，提高桃子的權重
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // 通過設置boost提高查詢的權重 .setQuery(QueryBuilders.boolQuery() .should(QueryBuilders.constantScoreQuery(QueryBuilders.termQuery("sName","student_1")).boost(3.0f)) .should(QueryBuilders.constantScoreQuery(QueryBuilders.termQuery("sName","student_2")).boost(0.3f))) .setSize(10) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
Score:3.0,Source:{"sName":"Student_1","sAge":28,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.666Z"} Score:0.3,Source:{"sName":"Student_2","sAge":20,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.689Z"} 命中：2
17.2 bool query
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // 查詢上下文 .setQuery(QueryBuilders.boolQuery() // (查詢結果)必須出現在匹配的文檔中，並將有助於得分 .must(QueryBuilders.rangeQuery("sAge").lte(30)) // (查詢結果)不能出現在匹配的文檔中，評分被忽略 .mustNot(QueryBuilders.termsQuery("sName","student_2","student_4")) // (查詢結果)應該出現在匹配的文檔中。如果bool查詢位於查詢上下文中，並且具有must或filter子句，should查詢將失效； // 如果bool查詢位於篩選上下文，或者既沒有must也沒有filter，那麼至少應該有一個查詢必須匹配文檔以匹配bool查詢 //.should(QueryBuilders.termQuery("sClass","Class_1")) // (查詢結果)必須出現在匹配的文檔中，然而，與must不同的是，查詢的分數將被忽略 .filter(QueryBuilders.rangeQuery("sAge").gte(5))) .setSize(500) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
如果只有must查詢，我測試查詢出304條結果；加上mustNot查詢，篩選出302條數據；加上should，仍然是302條數據，因爲有must；不要should，加上filter篩選出256條數據。
...省略 Score:1.0,Source:{"sName":"Student_992","sAge":16,"sClass":"Class_12","sTime":"2018-12-13T08:39:58.120Z"} Score:1.0,Source:{"sName":"Student_997","sAge":14,"sClass":"Class_17","sTime":"2018-12-13T08:39:58.120Z"} Score:1.0,Source:{"sName":"Student_999","sAge":6,"sClass":"Class_19","sTime":"2018-12-13T08:39:58.120Z"} 命中：256
下面測試篩選上下文
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") .setQuery(QueryBuilders.boolQuery() .must(QueryBuilders.rangeQuery("sAge").gte(10)) // 篩選上下文 .filter(QueryBuilders.boolQuery() // (查詢結果)必須出現在匹配的文檔中，並將有助於得分 .must(QueryBuilders.rangeQuery("sAge").lte(30)) // (查詢結果)不能出現在匹配的文檔中，評分被忽略 .mustNot(QueryBuilders.termsQuery("sName","student_2","student_4")) // (查詢結果)應該出現在匹配的文檔中。如果bool查詢位於查詢上下文中，並且具有must或filter子句，should查詢將失效； // 如果bool查詢位於篩選上下文，或者既沒有must也沒有filter，那麼至少應該有一個查詢必須匹配文檔以匹配bool查詢 .should(QueryBuilders.termQuery("sClass","Class_1")) )) .setSize(500) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
Score:1.0,Source:{"sName":"Student_1","sAge":28,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.666Z"} Score:1.0,Source:{"sName":"Student_81","sAge":23,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.692Z"} Score:1.0,Source:{"sName":"Student_161","sAge":21,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.695Z"} Score:1.0,Source:{"sName":"Student_181","sAge":18,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.695Z"} Score:1.0,Source:{"sName":"Student_281","sAge":21,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.698Z"} Score:1.0,Source:{"sName":"Student_341","sAge":19,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.822Z"} Score:1.0,Source:{"sName":"Student_361","sAge":14,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.822Z"} Score:1.0,Source:{"sName":"Student_381","sAge":11,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.822Z"} Score:1.0,Source:{"sName":"Student_501","sAge":23,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.825Z"} Score:1.0,Source:{"sName":"Student_541","sAge":21,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.826Z"} Score:1.0,Source:{"sName":"Student_601","sAge":27,"sClass":"Class_1","sTime":"2018-12-13T08:39:58.045Z"} Score:1.0,Source:{"sName":"Student_901","sAge":15,"sClass":"Class_1","sTime":"2018-12-13T08:39:58.119Z"} 命中：12
query&filter
17.3 dis_max query
...待補充
17.4 function_score query（function_score允許您修改查詢檢索的文檔的得分）
...待補充
17.5 boosting query
public static TransportClient queryDSL() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // 一種複合查詢，分爲positive子查詢和negitive子查詢，兩者的查詢結構都會返回 // positive子查詢的score保持不變，negative子查詢的值將會根據negative_boost的值做相應程度的降低。 .setQuery(QueryBuilders.boostingQuery(QueryBuilders.termQuery("sName","student_1"), QueryBuilders.termQuery("sClass","Class_1")).negativeBoost(0.3f)) .setSize(500) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
Score:1.950987,Source:{"sName":"Student_1","sAge":28,"sClass":"Class_1","sTime":"2018-12-13T08:39:57.666Z"} 命中：1
18 Joining queries
18.1 嵌套查詢
...待補充
18.2 Has Child Query
...待補充
18.3 Has Parent Query
...待補充
19. Geo queries（地理查詢）
地理形狀查詢需要相關依賴
<dependency> <groupId>org.locationtech.spatial4j</groupId> <artifactId>spatial4j</artifactId> <version>0.7</version> </dependency> <dependency> <groupId>org.locationtech.jts</groupId> <artifactId>jts-core</artifactId> <version>1.16.0</version> <exclusions> <exclusion> <groupId>xerces</groupId> <artifactId>xercesImpl</artifactId> </exclusion> </exclusions> </dependency>
關於geo_point
PUT my_index { "mappings": { "_doc": { "properties": { "location": { "type": "geo_point" } } } } } PUT my_index/_doc/1 { "text": "Geo-point as an object", "location": { "lat": 41.12, // 緯度 "lon": -71.34 // 經度 } } PUT my_index/_doc/2 { "text": "Geo-point as a string", "location": "41.12,-71.34" // 緯度,經度 } PUT my_index/_doc/3 { "text": "Geo-point as an array", "location": [ -71.34, 41.12 ] // 經度,緯度 } GET my_index/_search { "query": { "geo_bounding_box": { // 查詢落在框中的所有點 "location": { "top_left": { "lat": 42, "lon": -72 }, "bottom_right": { "lat": 40, "lon": -74 } } } } }
19.1 geo_shape查詢
查找與指定的地理形狀相交、包含或不相交的地理形狀的文檔。
19.2 geo_bounding_box查詢
查找落在指定矩形中的文檔
初始化數據：數據文件點擊這裏
/** * 從文件讀取地理位置 * 地理位置選擇：http://www.gpsspg.com/maps.htm * @return * @throws IOException */ private static List<String> getChinaCity() throws IOException { List<String> list = new ArrayList<>(); FileInputStream in = new FileInputStream("China.txt"); BufferedReader reader = new BufferedReader(new InputStreamReader(in)); String s = null; while ((s = reader.readLine()) != null){ list.add(s); } System.out.println("讀取【" + list.size() + "】條數據"); reader.close(); in.close(); return list; } /** * 初始化地理數據 * @return * @throws IOException */ public static TransportClient geoPointData() throws IOException { List<String> list = getChinaCity(); TransportClient client = TransportClientFactory.getInstance().getClient(); AdminClient admin = client.admin(); IndicesAdminClient indicesAdminClient = admin.indices(); // 指數管理 CreateIndexResponse createIndexResponse = indicesAdminClient.prepareCreate("china_index") .setSettings(Settings.builder() .put("index.number_of_shards", 1) // 分片 .put("index.number_of_replicas", 1) // 副本 ) .addMapping("city", "pName", "type=keyword","cName", "type=text", "location", "type=geo_point") // mapping .get(); System.out.println("創建索引：" + createIndexResponse.isAcknowledged()); BulkProcessor bulkProcessor = BulkProcessor.builder( client, new BulkProcessor.Listener() { @Override public void beforeBulk(long executionId, BulkRequest request) { // bulk 執行之前 System.out.println("beforeBulk-----" + request.getDescription()); } @Override public void afterBulk(long executionId, BulkRequest request, BulkResponse response) { // bulk 執行之後 System.out.println("afterBulk------" + request.getDescription() + ",是否有錯誤：" + response.hasFailures() + ",Msg:" + response.buildFailureMessage()); } @Override public void afterBulk(long executionId, BulkRequest request, Throwable failure) { //bulk 失敗 System.out.println("報錯-----" + request.getDescription() + "," + failure.getMessage()); } }) .setBulkActions(300) // 每300個request，bulk一次 .setConcurrentRequests(0) // 設置併發請求的數量。值爲0意味着只允許執行一個請求。值爲1意味着在積累新的批量請求時允許執行1個併發請求。 .build(); int i = 1; for (String s : list){ String[] data = s.split(","); Double latitude,longitude; // [經度：-180~180；緯度：-90~90] latitude = Double.parseDouble(data[3]); longitude = Double.parseDouble(data[2]); bulkProcessor.add(new IndexRequest("china_index", "city", i++ +"").source(jsonBuilder() .startObject() .field("pName", data[0]) .field("cName", data[1]) .field("location", new GeoPoint(latitude, longitude)) .endObject())); } bulkProcessor.flush(); bulkProcessor.close(); return client; }
執行之後：
GET /china_index/_search
.
下面查詢：
public static TransportClient geoBoundingBoxQuery() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("china_index") // top、left、bottom、right 圍成一個矩形，查詢這個矩形裏面的位置 .setQuery(QueryBuilders.geoBoundingBoxQuery("location").setCorners(40.3800284025,114.8730468750,38.8568201347,118.9599609375)) .setSize(100) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
查詢結果：
Score:1.0,Source:{"pName":"北京市","cName":"東城區","location":{"lat":39.92855,"lon":116.41637}} Score:1.0,Source:{"pName":"北京市","cName":"西城區","location":{"lat":39.91231,"lon":116.36611}} Score:1.0,Source:{"pName":"北京市","cName":"朝陽區","location":{"lat":39.927289,"lon":116.4498}} Score:1.0,Source:{"pName":"北京市","cName":"豐臺區","location":{"lat":39.85856,"lon":116.28616}} Score:1.0,Source:{"pName":"北京市","cName":"石景山區","location":{"lat":39.90569,"lon":116.22299}} Score:1.0,Source:{"pName":"北京市","cName":"海淀區","location":{"lat":39.95933,"lon":116.29845}} Score:1.0,Source:{"pName":"北京市","cName":"順義區","location":{"lat":40.13012,"lon":116.65477}} Score:1.0,Source:{"pName":"北京市","cName":"通州區","location":{"lat":39.916195,"lon":116.662852}} Score:1.0,Source:{"pName":"北京市","cName":"大興區","location":{"lat":39.72684,"lon":116.34159}} Score:1.0,Source:{"pName":"北京市","cName":"房山區","location":{"lat":39.74788,"lon":116.14294}} Score:1.0,Source:{"pName":"北京市","cName":"門頭溝區","location":{"lat":39.94048,"lon":116.10146}} Score:1.0,Source:{"pName":"北京市","cName":"昌平區","location":{"lat":40.22077,"lon":116.23128}} Score:1.0,Source:{"pName":"北京市","cName":"平谷區","location":{"lat":40.14062,"lon":117.12141}} Score:1.0,Source:{"pName":"北京市","cName":"密雲區","location":{"lat":40.37625,"lon":116.84317}} Score:1.0,Source:{"pName":"北京市","cName":"懷柔區","location":{"lat":40.316,"lon":116.63177}} Score:1.0,Source:{"pName":"天津市","cName":"和平區","location":{"lat":39.11712,"lon":117.2147}} Score:1.0,Source:{"pName":"天津市","cName":"河東區","location":{"lat":39.12827,"lon":117.25228}} Score:1.0,Source:{"pName":"天津市","cName":"河西區","location":{"lat":39.10954,"lon":117.22336}} Score:1.0,Source:{"pName":"天津市","cName":"南開區","location":{"lat":39.13815,"lon":117.15011}} Score:1.0,Source:{"pName":"天津市","cName":"河北區","location":{"lat":39.14784,"lon":117.19674}} Score:1.0,Source:{"pName":"天津市","cName":"紅橋區","location":{"lat":39.16734,"lon":117.15161}} Score:1.0,Source:{"pName":"天津市","cName":"濱海新區","location":{"lat":39.0032,"lon":117.71071}} Score:1.0,Source:{"pName":"天津市","cName":"東麗區","location":{"lat":39.08652,"lon":117.31428}} Score:1.0,Source:{"pName":"天津市","cName":"西青區","location":{"lat":39.14111,"lon":117.00739}} Score:1.0,Source:{"pName":"天津市","cName":"津南區","location":{"lat":38.9375,"lon":117.3571}} Score:1.0,Source:{"pName":"天津市","cName":"北辰區","location":{"lat":39.22393,"lon":117.13544}} Score:1.0,Source:{"pName":"天津市","cName":"武清區","location":{"lat":39.38326,"lon":117.04412}} Score:1.0,Source:{"pName":"天津市","cName":"寶坻區","location":{"lat":39.71755,"lon":117.30983}} Score:1.0,Source:{"pName":"天津市","cName":"寧河區","location":{"lat":39.33091,"lon":117.82478}} Score:1.0,Source:{"pName":"天津市","cName":"靜海區","location":{"lat":38.94737,"lon":116.97428}} Score:1.0,Source:{"pName":"天津市","cName":"薊州區","location":{"lat":40.04577,"lon":117.40829}} Score:1.0,Source:{"pName":"天津市","cName":"天津","location":{"lat":39.13,"lon":117.2}} Score:1.0,Source:{"pName":"天津市","cName":"寧河","location":{"lat":39.33,"lon":117.83}} Score:1.0,Source:{"pName":"天津市","cName":"靜海","location":{"lat":38.93,"lon":116.92}} Score:1.0,Source:{"pName":"天津市","cName":"薊縣","location":{"lat":40.05,"lon":117.4}} Score:1.0,Source:{"pName":"天津市","cName":"寶坻","location":{"lat":39.75,"lon":117.3}} Score:1.0,Source:{"pName":"天津市","cName":"武清","location":{"lat":39.4,"lon":117.05}} Score:1.0,Source:{"pName":"河北省","cName":"唐山","location":{"lat":39.63,"lon":118.02}} Score:1.0,Source:{"pName":"河北省","cName":"淶水","location":{"lat":39.39,"lon":115.71}} Score:1.0,Source:{"pName":"河北省","cName":"涿縣","location":{"lat":39.48,"lon":115.98}} Score:1.0,Source:{"pName":"河北省","cName":"定興","location":{"lat":39.28,"lon":115.78}} Score:1.0,Source:{"pName":"河北省","cName":"容城","location":{"lat":39.06,"lon":115.86}} Score:1.0,Source:{"pName":"河北省","cName":"安新","location":{"lat":38.92,"lon":115.92}} Score:1.0,Source:{"pName":"河北省","cName":"易縣","location":{"lat":39.35,"lon":115.49}} Score:1.0,Source:{"pName":"河北省","cName":"新城","location":{"lat":39.34,"lon":115.84}} Score:1.0,Source:{"pName":"河北省","cName":"雄縣","location":{"lat":38.98,"lon":116.1}} Score:1.0,Source:{"pName":"河北省","cName":"徐水","location":{"lat":39.02,"lon":115.65}} Score:1.0,Source:{"pName":"河北省","cName":"滿城","location":{"lat":38.95,"lon":115.45}} Score:1.0,Source:{"pName":"河北省","cName":"涿鹿","location":{"lat":40.37,"lon":115.2}} Score:1.0,Source:{"pName":"河北省","cName":"遷西","location":{"lat":40.15,"lon":118.3}} Score:1.0,Source:{"pName":"河北省","cName":"遷安","location":{"lat":40.02,"lon":118.69}} Score:1.0,Source:{"pName":"河北省","cName":"盧龍","location":{"lat":39.89,"lon":118.85}} Score:1.0,Source:{"pName":"河北省","cName":"灤南","location":{"lat":39.49,"lon":118.67}} Score:1.0,Source:{"pName":"河北省","cName":"玉田","location":{"lat":39.9,"lon":117.9}} Score:1.0,Source:{"pName":"河北省","cName":"唐海","location":{"lat":39.31,"lon":118.54}} Score:1.0,Source:{"pName":"河北省","cName":"遵化","location":{"lat":40.2,"lon":117.97}} Score:1.0,Source:{"pName":"河北省","cName":"樂亭","location":{"lat":39.43,"lon":118.9}} Score:1.0,Source:{"pName":"河北省","cName":"灤縣","location":{"lat":39.74,"lon":118.73}} Score:1.0,Source:{"pName":"河北省","cName":"豐南","location":{"lat":39.58,"lon":118.1}} Score:1.0,Source:{"pName":"河北省","cName":"豐潤","location":{"lat":39.82,"lon":118.13}} Score:1.0,Source:{"pName":"河北省","cName":"廊坊","location":{"lat":39.53,"lon":116.7}} Score:1.0,Source:{"pName":"河北省","cName":"安次","location":{"lat":39.52,"lon":116.69}} Score:1.0,Source:{"pName":"河北省","cName":"三河","location":{"lat":39.97,"lon":117.06}} Score:1.0,Source:{"pName":"河北省","cName":"香河","location":{"lat":39.76,"lon":117.0}} Score:1.0,Source:{"pName":"河北省","cName":"霸縣","location":{"lat":39.12,"lon":116.38}} Score:1.0,Source:{"pName":"河北省","cName":"固安","location":{"lat":39.44,"lon":116.29}} Score:1.0,Source:{"pName":"河北省","cName":"文安","location":{"lat":38.87,"lon":116.45}} Score:1.0,Source:{"pName":"河北省","cName":"永清","location":{"lat":39.32,"lon":116.48}} Score:1.0,Source:{"pName":"河北省","cName":"大廠","location":{"lat":39.98,"lon":116.98}} 命中：
我通過這個網站找出兩個點，用來測試數據 http://www.gpsspg.com/maps.htm
19.3 geo_distance查詢
查找固定中心，指定半徑範圍內的文檔
public static TransportClient geoBoundingBoxQuery() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("china_index") // top、left、bottom、right 圍成一個矩形，查詢這個矩形裏面的位置 //.setQuery(QueryBuilders.geoBoundingBoxQuery("location").setCorners(40.3800284025,114.8730468750,38.8568201347,118.9599609375)) // 查找以北京爲中心，半徑爲200千米範圍內的城市 .setQuery(QueryBuilders.geoDistanceQuery("location").point(39.6056881783,116.6748046875).distance(20, DistanceUnit.KILOMETERS)) .setSize(100) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果：
Score:1.0,Source:{"pName":"河北省","cName":"廊坊","location":{"lat":39.53,"lon":116.7}} Score:1.0,Source:{"pName":"河北省","cName":"安次","location":{"lat":39.52,"lon":116.69}} 命中：
19.4 geo_polygon查詢
查找指定多邊形內的文檔
我先選擇一個多邊形的幾個點
23.6042618471,113.0932617188
22.3703963443,112.4780273438
21.4530686331,113.7963867188
22.8571947010,115.1147460938
下面查詢
public static TransportClient geoPolygonQuery() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); List<GeoPoint> list = new ArrayList<>(); list.add(new GeoPoint(23.6042618471,113.0932617188)); list.add(new GeoPoint(22.3703963443,112.4780273438)); list.add(new GeoPoint(21.4530686331,113.7963867188)); list.add(new GeoPoint(22.8571947010,115.1147460938)); SearchResponse searchResponse = client.prepareSearch() .setIndices("china_index") // 查找多邊形內的城市 .setQuery(QueryBuilders.geoPolygonQuery("location", list)) .setSize(100) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果
Score:1.0,Source:{"pName":"廣東省","cName":"廣州","location":{"lat":23.12908,"lon":113.26436}} Score:1.0,Source:{"pName":"廣東省","cName":"越秀","location":{"lat":23.12901,"lon":113.2668}} Score:1.0,Source:{"pName":"廣東省","cName":"荔灣","location":{"lat":23.12586,"lon":113.24428}} Score:1.0,Source:{"pName":"廣東省","cName":"海珠","location":{"lat":23.08331,"lon":113.3172}} Score:1.0,Source:{"pName":"廣東省","cName":"天河","location":{"lat":23.12463,"lon":113.36199}} Score:1.0,Source:{"pName":"廣東省","cName":"白雲","location":{"lat":23.157032,"lon":113.273238}} Score:1.0,Source:{"pName":"廣東省","cName":"黃埔","location":{"lat":23.18139,"lon":113.48067}} Score:1.0,Source:{"pName":"廣東省","cName":"花都","location":{"lat":23.40391,"lon":113.22026}} Score:1.0,Source:{"pName":"廣東省","cName":"番禺","location":{"lat":22.93772,"lon":113.38424}} Score:1.0,Source:{"pName":"廣東省","cName":"南沙","location":{"lat":22.80169,"lon":113.52497}} Score:1.0,Source:{"pName":"廣東省","cName":"增城","location":{"lat":23.26093,"lon":113.8109}} Score:1.0,Source:{"pName":"廣東省","cName":"深圳","location":{"lat":22.54286,"lon":114.05956}} Score:1.0,Source:{"pName":"廣東省","cName":"福田","location":{"lat":22.52291,"lon":114.05454}} Score:1.0,Source:{"pName":"廣東省","cName":"羅湖","location":{"lat":22.54836,"lon":114.13166}} Score:1.0,Source:{"pName":"廣東省","cName":"南山","location":{"lat":22.53325,"lon":113.93069}} Score:1.0,Source:{"pName":"廣東省","cName":"鹽田","location":{"lat":22.557,"lon":114.23679}} Score:1.0,Source:{"pName":"廣東省","cName":"寶安","location":{"lat":22.55329,"lon":113.88308}} Score:1.0,Source:{"pName":"廣東省","cName":"龍崗","location":{"lat":22.71991,"lon":114.24779}} Score:1.0,Source:{"pName":"廣東省","cName":"龍華","location":{"lat":22.696694,"lon":114.044612}} Score:1.0,Source:{"pName":"廣東省","cName":"坪山","location":{"lat":22.69084,"lon":114.34632}} Score:1.0,Source:{"pName":"廣東省","cName":"珠海","location":{"lat":22.27073,"lon":113.57668}} Score:1.0,Source:{"pName":"廣東省","cName":"香洲","location":{"lat":22.265939,"lon":113.543202}} Score:1.0,Source:{"pName":"廣東省","cName":"斗門","location":{"lat":22.20903,"lon":113.29658}} Score:1.0,Source:{"pName":"廣東省","cName":"金灣","location":{"lat":22.14589,"lon":113.3642}} Score:1.0,Source:{"pName":"廣東省","cName":"佛山","location":{"lat":23.02185,"lon":113.12192}} Score:1.0,Source:{"pName":"廣東省","cName":"禪城","location":{"lat":23.00944,"lon":113.12249}} Score:1.0,Source:{"pName":"廣東省","cName":"高明","location":{"lat":22.90026,"lon":112.89262}} Score:1.0,Source:{"pName":"廣東省","cName":"三水","location":{"lat":23.15585,"lon":112.89672}} Score:1.0,Source:{"pName":"廣東省","cName":"南海","location":{"lat":23.02882,"lon":113.14278}} Score:1.0,Source:{"pName":"廣東省","cName":"順德","location":{"lat":22.80536,"lon":113.29321}} Score:1.0,Source:{"pName":"廣東省","cName":"江門","location":{"lat":22.57865,"lon":113.08161}} Score:1.0,Source:{"pName":"廣東省","cName":"江海","location":{"lat":22.56028,"lon":113.11103}} Score:1.0,Source:{"pName":"廣東省","cName":"蓬江","location":{"lat":22.59514,"lon":113.07845}} Score:1.0,Source:{"pName":"廣東省","cName":"新會","location":{"lat":22.45877,"lon":113.03231}} Score:1.0,Source:{"pName":"廣東省","cName":"台山","location":{"lat":22.25156,"lon":112.7939}} Score:1.0,Source:{"pName":"廣東省","cName":"鶴山","location":{"lat":22.7653,"lon":112.96436}} Score:1.0,Source:{"pName":"廣東省","cName":"開平","location":{"lat":22.37638,"lon":112.69851}} Score:1.0,Source:{"pName":"廣東省","cName":"惠州","location":{"lat":23.11075,"lon":114.41679}} Score:1.0,Source:{"pName":"廣東省","cName":"惠城","location":{"lat":23.08383,"lon":114.38273}} Score:1.0,Source:{"pName":"廣東省","cName":"惠陽","location":{"lat":22.78851,"lon":114.45646}} Score:1.0,Source:{"pName":"廣東省","cName":"惠東","location":{"lat":22.98486,"lon":114.71999}} Score:1.0,Source:{"pName":"廣東省","cName":"東莞","location":{"lat":23.02067,"lon":113.75179}} Score:1.0,Source:{"pName":"廣東省","cName":"中山","location":{"lat":22.51595,"lon":113.3926}} Score:1.0,Source:{"pName":"澳門","cName":"澳門","location":{"lat":22.13,"lon":113.33}} Score:1.0,Source:{"pName":"香港","cName":"香港","location":{"lat":22.2,"lon":114.1}} 命中：
20 腳本查詢
public static TransportClient scriptQuery() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); Map<String, Object> parameters = new HashMap<>(); parameters.put("param1", "Class_2"); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") // inlined script //.setQuery(QueryBuilders.scriptQuery(new Script("doc['sName'].value == 'Student_666'"))) // 帶參數的 .setQuery(QueryBuilders.scriptQuery(new Script(ScriptType.INLINE,"painless", "doc['sClass'].value == params.param1",parameters))) .setSize() // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果
Score:1.0,Source:{"sName":"Student_2","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.689Z"} Score:1.0,Source:{"sName":"Student_22","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.690Z"} Score:1.0,Source:{"sName":"Student_42","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.690Z"} Score:1.0,Source:{"sName":"Student_62","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.691Z"} Score:1.0,Source:{"sName":"Student_82","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.692Z"} Score:1.0,Source:{"sName":"Student_102","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.693Z"} Score:1.0,Source:{"sName":"Student_122","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.693Z"} Score:1.0,Source:{"sName":"Student_142","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.694Z"} Score:1.0,Source:{"sName":"Student_162","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.695Z"} Score:1.0,Source:{"sName":"Student_182","sAge":,"sClass":"Class_2","sTime":"2018-12-13T08:39:57.695Z"} 命中：
21 Span 查詢
Span查詢是低級的位置查詢，提供對指定術語的順序和接近程度的專業控制。這些通常用於實現對法律文件或專利的非常具體的查詢。
public static TransportClient spanQuery() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); SearchResponse searchResponse = client.prepareSearch() .setIndices("school") .setQuery(QueryBuilders.spanTermQuery("sName","student_666")) .setSize(10) // 返回數量 .get(); searchResponse.getHits().forEach(e -> { System.out.println("Score:" + e.getScore() + ",Source:" + e.getSourceAsString()); }); System.out.println("命中：" + searchResponse.getHits().totalHits); return client; }
執行結果
Score:6.5032897,Source:{"sName":"Student_666","sAge":10,"sClass":"Class_6","sTime":"2018-12-13T08:39:58.046Z"} 命中：1
22 Administration
22.1 添加索引
public static TransportClient createIndex() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); AdminClient admin = client.admin(); IndicesAdminClient indicesAdminClient = admin.indices(); // 指數管理 // 創建index CreateIndexResponse createIndexResponse = indicesAdminClient.prepareCreate("twitter") .setSettings(Settings.builder() .put("index.number_of_shards", 3) // 分片 .put("index.number_of_replicas", 2) // 副本 ) .addMapping("_doc", "message", "type=text") // mapping .addAlias(new Alias("twitter_alias")) .get(); System.out.println(createIndexResponse.isAcknowledged()); return client; }
返回true
22.2 查看索引信息
public static TransportClient getIndex() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); AdminClient admin = client.admin(); IndicesAdminClient indicesAdminClient = admin.indices(); // 指數管理 // 獲取index GetIndexResponse getIndexResponse = indicesAdminClient.prepareGetIndex().addIndices("twitter").get(); getIndexResponse.getMappings().forEach(e -> { String key = e.key; ImmutableOpenMap<String, MappingMetaData> map = e.value; System.out.println("Index:" + key); map.forEach(n -> { String type = n.key; MappingMetaData metaData = n.value; System.out.println(type + "|" + metaData.getSourceAsMap()); }); }); System.out.println("**********************************"); getIndexResponse.getAliases().forEach(e -> { String key = e.key; List<AliasMetaData> value = e.value; System.out.println("----" + key + "----"); value.forEach(a -> { System.out.println(a.alias()); }); }); System.out.println("**********************************"); getIndexResponse.getSettings().forEach(e -> { String key = e.key; Settings value = e.value; System.out.println("----" + key + "----"); value.keySet().forEach(k -> { System.out.println(k + ":" + value.get(k)); }); }); return client; }
執行結果
Index:twitter _doc|{properties={message={type=text}}} ********************************** ----twitter---- twitter_alias ********************************** ----twitter---- index.creation_date:1545383477041 index.number_of_replicas:2 index.number_of_shards:3 index.provided_name:twitter index.uuid:Zggv0JmiSGCM27S2A_D-og index.version.created:6050099
22.3 刪除索引
public static TransportClient delIndex() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); AcknowledgedResponse response = client.admin().indices().prepareDelete("twitter").get(); System.out.println(response.isAcknowledged()); return client; }
返回true
22.4 如果你建立好索引，想要添加mapping【已經建立好的mapping字段不允許修改】
public static TransportClient putIndexMapping() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); AdminClient admin = client.admin(); IndicesAdminClient indicesAdminClient = admin.indices(); // 指數管理 // 增加新的mapping AcknowledgedResponse response = indicesAdminClient.preparePutMapping("twitter") .setType("_doc") .setSource("afterAdd", "type=keyword") .get(); System.out.println(response.isAcknowledged()); return client; }
返回true
再來看index信息（通過getIndex）
Index:twitter _doc|{properties={afterAdd={type=keyword}, message={type=text}}} ********************************** ----twitter---- twitter_alias ********************************** ----twitter---- index.creation_date:1545383985875 index.number_of_replicas:2 index.number_of_shards:3 index.provided_name:twitter index.uuid:K2_5S8pIR4mqLGga7-0ZHQ index.version.created:6050099
23 集羣信息
/** * 集羣信息，節點信息，索引信息，狀態信息 * @return * @throws UnknownHostException */ public static TransportClient clusterInfo() throws UnknownHostException { TransportClient client = TransportClientFactory.getInstance().getClient(); AdminClient admin = client.admin(); // 集羣管理 ClusterHealthResponse healths = admin.cluster().prepareHealth().get(); String clusterName = healths.getClusterName(); int numberOfDataNodes = healths.getNumberOfDataNodes(); int numberOfNodes = healths.getNumberOfNodes(); System.out.println("集羣名稱：" + clusterName); System.out.println("數據節點數：" + numberOfDataNodes); System.out.println("節點數：" + numberOfNodes); for (ClusterIndexHealth health : healths.getIndices().values()) { String index = health.getIndex(); int numberOfShards = health.getNumberOfShards(); int numberOfReplicas = health.getNumberOfReplicas(); ClusterHealthStatus status = health.getStatus(); System.out.println("索引：" + index + ",分片數：" + numberOfShards + ",副本數：" + numberOfReplicas + ",狀態：" + status.name()); } return client; }
執行結果
集羣名稱：my-elasticsearch 數據節點數： 節點數： 索引：area,分片數：,副本數：,狀態：YELLOW 索引：china_index,分片數：,副本數：,狀態：YELLOW 索引：book,分片數：,副本數：,狀態：YELLOW 索引：test_ik_index,分片數：,副本數：,狀態：YELLOW 索引：posts,分片數：,副本數：,狀態：YELLOW 索引：.kibana_1,分片數：,副本數：,狀態：GREEN 索引：clothes,分片數：,副本數：,狀態：YELLOW 索引：all_books,分片數：,副本數：,狀態：YELLOW 索引：twitter,分片數：,副本數：,狀態：YELLOW 索引：it_book,分片數：,副本數：,狀態：YELLOW 索引：car,分片數：,副本數：,狀態：YELLOW 索引：school,分片數：,副本數：,狀態：YELLOW 索引：story,分片數：,副本數：,狀態：YELLOW