elasticsearch 常見查詢及聚合的JAVA API

最近在研究ES 有點心得記錄下備忘

ES 常見查詢

根據ID 進行單個查詢

    GetResponse response = client.prepareGet("accounts", "person", "1")
                    .setOperationThreaded(false)
                    .get();

相對於sql 的 select * from accounts.person where id=1 ;

分頁查詢所有記錄

    QueryBuilder qb=new MatchAllQueryBuilder();
            SearchResponse response= client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFrom(0)
                    .setSize(100).get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }

response.getHits()是所有命中記錄相較於sql select * from accounts.person limit 100;

根據多條件組合與查詢

QueryBuilder qb=QueryBuilders.boolQuery().must(QueryBuilders.termQuery("title","JAVA開發工程師")).must(QueryBuilders.termQuery("age",30)) ;

            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFrom(0)
                    .setSize(100);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }

must 就像sql裏的and 相較於sql select * from accounts.person where title='JAVA開發工程師' and age=30

多條件或查詢

QueryBuilder qb=QueryBuilders.termQuery("user","kimchy14");
QueryBuilder qb1=QueryBuilders.termQuery("user","kimchy15");

            SortBuilder sortBuilder=SortBuilders.fieldSort("age");
            sortBuilder.order(SortOrder.DESC);
            QueryBuilder s=QueryBuilders.boolQuery().should(qb).should(qb1);//.must(qb5);
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).addSort(sortBuilder).setFrom(0)
                    .setSize(100);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }

should 就像sql裏的or SortBuilder 的作用不言而喻就是用來排序以上代碼相較於sql select * from accounts.person where user='kimchy14' or user='kimchy15' ;

範圍查詢

    // RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
           // RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gt(30 );
            RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").gte(30 );
            QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0)
                    .setSize(100);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }

範圍查詢rangeQuery.from（30，true）方法是大於30 後面的參數是是否包含爲true的話就是大於等於30 to就相當於小於如果也有包含參數爲true的話就是小於等於 gt 是大於 gte是大於等於 lt是小於 lte是小於等於第一句的builder就相當於 select * from accounts.person where age >=30 and age<=30;

包含查詢

     List<String> strs=new ArrayList<>();
            strs.add("kimchy14");
            strs.add("kimchy15");
            strs.add("kimchy16");
            QueryBuilder qb=QueryBuilders.termsQuery("user",strs);

            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("age",null).setFrom(0)
                    .setSize(100);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }

包含查詢使用termsQuery 可以傳列表也可以傳多個參數或者數組 setFetchSource有兩個參數第一個參數是包含哪些參數第二個參數是排除哪些參數以上這段代碼就相當於sql select age from accounts.person where user in ('kimchy14','kimchy15','kimchy16');

專門按id進行的包含查詢

QueryBuilder qb=QueryBuilders.idsQuery(0+"");

按通配符查詢

QueryBuilder qb = QueryBuilders.wildcardQuery("user", "k*hy17*");
//Fuzziness fuzziness=Fuzziness.fromEdits(2);

          // QueryBuilder qb = QueryBuilders.fuzzyQuery("user","mchy2").fuzziness(fuzziness);
            //QueryBuilder qb = QueryBuilders.prefixQuery("user", "kimchy2");
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(qb).setFetchSource("user",null).setFrom(0)
                    .setSize(100);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }

通配符查詢像我們sql裏的like 但是還不一樣 like的百分號可以加到前後 elasticsearch技術解析與實戰中有一句話是這麼說的爲了避免極端緩慢的通配符查詢通配符索引詞不應該以一個通配符開頭通配符查詢應該避免以通配符開頭謝謝androidtoutou的指正

常見統計統計分爲指標和桶桶就是我們統計的樣本指標就是我們平時所查的count sum 與sql不一樣的是我們還可以將統計的樣本拿到就是response.getHits

統計count

AggregationBuilder termsBuilder = AggregationBuilders.count("ageCount").field("age");

            RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
            QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }
            ValueCount valueCount= response.getAggregations().get("ageCount");
            long value=valueCount.getValue();

這段代碼就相當於 sql select count(age) ageCount form accounts.person where age >=30 and age<=30

查詢最大值

AggregationBuilder termsBuilder = AggregationBuilders.max("max").field("age");

            RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
            QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }
            Max valueCount= response.getAggregations().get("max");
            double value=valueCount.getValue();

統計總和

AggregationBuilder termsBuilder = AggregationBuilders.sum("sum").field("age");

            RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
            QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }
            Sum valueCount= response.getAggregations().get("sum");
            double value=valueCount.getValue();

平均數

AggregationBuilder termsBuilder = AggregationBuilders.avg("avg").field("age");

            RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
            QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }
            Avg valueCount= response.getAggregations().get("avg");
            double value=valueCount.getValue();

統計樣本基本指標

AggregationBuilder termsBuilder = AggregationBuilders.stats("stats").field("age");

            RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(30,true);
            QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
            SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFrom(0).setSize(100).addAggregation(termsBuilder);
            logger.log(Level.INFO,sv.toString());
            SearchResponse response= sv.get();
            SearchHits searchHits = response.getHits();
            for(SearchHit hit:searchHits.getHits()){
                logger.log(Level.INFO , hit.getSourceAsString());
            }
            Stats valueCount= response.getAggregations().get("stats");
            logger.log(Level.INFO,"max"+valueCount.getMaxAsString());
            logger.log(Level.INFO,"avg"+valueCount.getAvgAsString());
            logger.log(Level.INFO,"sum"+valueCount.getSumAsString());
            logger.log(Level.INFO,"min"+valueCount.getMinAsString());
            logger.log(Level.INFO,"count"+valueCount.getCount());

分組統計相當於group by 後拿各組指標進行統計

分組求各組數據

AggregationBuilder termsBuilder = AggregationBuilders.terms("by_age").field("age");
        AggregationBuilder sumBuilder=AggregationBuilders.sum("ageSum").field("age");
        AggregationBuilder avgBuilder=AggregationBuilders.avg("ageAvg").field("age");
        AggregationBuilder countBuilder=AggregationBuilders.count("ageCount").field("age");

        termsBuilder.subAggregation(sumBuilder).subAggregation(avgBuilder).subAggregation(countBuilder);
        //TermsAggregationBuilder all = AggregationBuilders.terms("age").field("age");
        //all.subAggregation(termsBuilder);
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(36,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(s).setFetchSource(null,"gender").setFrom(0).setSize(100).addAggregation(termsBuilder);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response= sv.get();

        Aggregations terms= response.getAggregations();
        for (Aggregation a:terms){
            LongTerms teamSum= (LongTerms)a;
            for(LongTerms.Bucket bucket:teamSum.getBuckets()){
                logger.info(bucket.getKeyAsString()+"   "+bucket.getDocCount()+"    "+((Sum)bucket.getAggregations().asMap().get("ageSum")).getValue()+"    "+((Avg)bucket.getAggregations().asMap().get("ageAvg")).getValue()+"    "+((ValueCount)bucket.getAggregations().asMap().get("ageCount")).getValue());

}
}

第一行 termsBuilder 就相當於根據年齡對數據進行分組 group by 後面對sumBuilder avgBuilder countBuilder等就是在組內求和求平均數求數量

多分組求各組數據

        TermsAggregationBuilder all = AggregationBuilders.terms("by_gender").field("gender");
         AggregationBuilder age = AggregationBuilders.terms("by_age").field("age");
        AggregationBuilder sumBuilder=AggregationBuilders.sum("ageSum").field("age");
        //AggregationBuilder avgBuilder=AggregationBuilders.avg("ageAvg").field("age");
       // AggregationBuilder countBuilder=AggregationBuilders.count("ageCount").field("age");
         all.subAggregation(age.subAggregation(sumBuilder));
        RangeQueryBuilder rangeQueryBuilder = QueryBuilders.rangeQuery("age").from(30,true).to(32,true);
        QueryBuilder s=QueryBuilders.boolQuery().must(rangeQueryBuilder);//.must(qb5);
        SearchRequestBuilder sv=client.prepareSearch("accounts").setTypes("person").setQuery(rangeQueryBuilder).addAggregation(all);
        logger.log(Level.INFO,sv.toString());
        SearchResponse response= sv.get();

        Aggregations terms= response.getAggregations();
        for (Aggregation a:terms){
            StringTerms stringTerms= (StringTerms)a;
             for(StringTerms.Bucket bucket:stringTerms.getBuckets()){
              // logger.info(bucket.getKeyAsString());
                Aggregation aggs=bucket.getAggregations().getAsMap().get("by_age");
                 LongTerms terms1= (LongTerms)aggs;
                for (LongTerms.Bucket bu:terms1.getBuckets()){
                     logger.info(bucket.getKeyAsString()+" "+bu.getKeyAsString()+" "+bu.getDocCount()+"    "+((Sum)bu.getAggregations().asMap().get("ageSum")).getValue());
                }

}
}

每增加一個分組指標就需要多加一個termsBuilder 其他等一切跟普通分組一樣每次拿到

以上就是我總結的基本的查詢聚合等常見功能其他等諸如求各組前多少數據是用topHits 這些基本夠我們日常操作了。

最後我們總結下精確查詢用term 組合查詢用bool 範圍用range and查詢用must or查詢用should not查詢用must not 常見的接收聚合返回結果的類型 ValueCount AVG SUM MAX MIN 按照英文意義就可以理解分組聚合查詢時候還需要根據實際情況看是返回那種terms
---------------------
作者：獨一無二的小馬哥
來源：CSDN
原文：https://blog.csdn.net/majun_guang/article/details/81103623
版權聲明：本文爲博主原創文章，轉載請附上博文鏈接！

elasticsearch 常見查詢及聚合的JAVA API

更換網絡以後 github進不去

vue cli 初始化

springboot vue hutool實現前後端分離導出excel

idea2019 win系統啓動報 error

mac下安裝nginx 啓動和關閉重啓

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結