穀粒商城學習筆記，第六天：ES全文檢索+SpringBoot

穀粒商城學習筆記，第六天：ES全文檢索+SpringBoot

協議	方式	描述
9300 TCP	spring-data-elasticsearch：transport-api	ES7.X不建議使用，ES8準備啓用
9200 HTTP	JestClient	非官方，更新慢
9200 HTTP	RestTemplate	ES很多操作需要自己封裝，麻煩
9200 HTTP	HttpClient	ES很多操作需要自己封裝，麻煩
9200 HTTP	Elasticsearch-Rest-Client	官方RestClient，封裝了ES操作，API層次分明，上手簡單

一、Elasticsearch-Rest-Client

elasticsearch-rest-hight-level-client

文檔地址

1、引入POM

<!--導入ES-->
<dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>7.4.2</version>
</dependency>

2、修改elasticsearch版本

##由於springboot2.3.5.RELEASE默認引用的是elasticsearch7.6.2版本的
##我們的服務器是7.4.2版本的，我們在配置中替換springboot自帶的版本
<properties>
    <!--替換springboot自帶版本-->
    <elasticsearch.version>7.4.2</elasticsearch.version>
</properties>

3、配置ES

@Configuration
public class ElasticSearchConfig {

    @Bean
    public RestHighLevelClient restHighLevelClient(){
        RestHighLevelClient client = new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost("182.92.191.49", 9200, "http")
                ));

        return client;
    }
}

4、測試

@SpringBootTest
public class GulimallSearchApplicationTests {

    @Autowired
    private RestHighLevelClient client;

    @Test
    public void contextLoads() {
        System.out.println(client);
    }

}

5、RequestOptions

RequestOptions類保留應在同一應用程序中的許多請求之間共享的部分請求。 您可以創建一個單例實例並在所有請求之間共享它：

• 添加所有請求所需的header

• 自定義響應的消費者等

可以在配置文件ElasticSearchConfig 添加如下：

public static final RequestOptions COMMON_OPTIONS;
static {
    RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
    //TODO
    COMMON_OPTIONS = builder.build();
}

6、添加數據和修改數據

    @Autowired
    private RestHighLevelClient client;

    @Data
    class User{
        private String username;
        private Integer age;
        private String gender;
    }

    @Test
    public void addAndEditIndex() throws IOException {
        //封裝請求body
        IndexRequest request = new IndexRequest("users");
        User user = new User();
        user.setUsername("lee");
        user.setAge(18);
        user.setGender("Male");
        String userJson = JSON.toJSONString(user);
        request.source(userJson, XContentType.JSON);

        //發送請求
        IndexResponse resp = client.index(request, ElasticSearchConfig.COMMON_OPTIONS);
        System.out.println(resp);
    }

7、查詢數據

@Data
    @ToString
    static class BankData {
        private int account_number;
        private int balance;
        private String firstname;
        private String lastname;
        private int age;
        private String gender;
        private String address;
        private String employer;
        private String email;
        private String city;
        private String state;
    }


    //搜索address中包含mill的所有人的年齡分佈以及這個年齡段的平均薪資
    @Test
    public void searchIndex() throws IOException {
       //創建搜索請求
        SearchRequest searchRequest = new SearchRequest();
        //指定索引
        searchRequest.indices("bank");

        //創建檢索語句
        SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
            //搜索：QueryBuilders是搜索的工具類
            sourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
            //聚合: AggregationBuilders是聚合的工具類
            TermsAggregationBuilder ageTermsAggs = AggregationBuilders.terms("ageTermsAggs").field("age");
            AvgAggregationBuilder balanceAvgAggs = AggregationBuilders.avg("balanceAvgAggs").field("balance");
            ageTermsAggs.subAggregation(balanceAvgAggs);
            sourceBuilder.aggregation(ageTermsAggs);
        searchRequest.source(sourceBuilder);
        System.out.println("檢索語句："+sourceBuilder.toString());

        //發送請求
        SearchResponse resp = client.search(searchRequest, ElasticSearchConfig.COMMON_OPTIONS);
        
        //處理請求結果
        RestStatus status = resp.status();
        System.out.println("檢索結果status："+status.toString());
        //搜索結果
        SearchHits hits = resp.getHits();
        for (SearchHit hit : hits) {
            String sourceString = hit.getSourceAsString();
            BankData bankData = JSONObject.parseObject(sourceString, BankData.class);
            System.out.println("檢索結果body："+bankData);
        }
        //聚合結果
        Aggregations aggs = resp.getAggregations();
        Terms ageTermsAggsResult = aggs.get("ageTermsAggs");
        for (Terms.Bucket bucket : ageTermsAggsResult.getBuckets()) {
            String key = bucket.getKey().toString();
            long docCount = bucket.getDocCount();
            System.out.println("檢索結果aggs   年齡分佈："+key+" "+docCount);
            Aggregations subAggs = bucket.getAggregations();
            Avg balanceAvgAggsResult = subAggs.get("balanceAvgAggs");
            System.out.println("檢索結果aggs    所屬年齡的 平均薪資："+balanceAvgAggsResult.getValue());
        }


    }

打印內容：

檢索語句：{"query":{"match":{"address":{"query":"mill","operator":"OR","prefix_length":0,"max_expansions":50,"fuzzy_transpositions":true,"lenient":false,"zero_terms_query":"NONE","auto_generate_synonyms_phrase_query":true,"boost":1.0}}},"aggregations":{"ageTermsAggs":{"terms":{"field":"age","size":10,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"_count":"desc"},{"_key":"asc"}]},"aggregations":{"balanceAvgAggs":{"avg":{"field":"balance"}}}}}}
檢索結果status：OK
檢索結果body：GulimallSearchApplicationTests.BankData(account_number=970, balance=19648, firstname=Forbes, lastname=Wallace, age=28, gender=M, address=990 Mill Road, employer=Pheast, [email protected], city=Lopezo, state=AK)
檢索結果body：GulimallSearchApplicationTests.BankData(account_number=136, balance=45801, firstname=Winnie, lastname=Holland, age=38, gender=M, address=198 Mill Lane, employer=Neteria, [email protected], city=Urie, state=IL)
檢索結果body：GulimallSearchApplicationTests.BankData(account_number=345, balance=9812, firstname=Parker, lastname=Hines, age=38, gender=M, address=715 Mill Avenue, employer=Baluba, [email protected], city=Blackgum, state=KY)
檢索結果body：GulimallSearchApplicationTests.BankData(account_number=472, balance=25571, firstname=Lee, lastname=Long, age=32, gender=F, address=288 Mill Street, employer=Comverges, [email protected], city=Movico, state=MT)
檢索結果aggs   年齡分佈：38 2
檢索結果aggs    所屬年齡的 平均薪資：27806.5
檢索結果aggs   年齡分佈：28 1
檢索結果aggs    所屬年齡的 平均薪資：19648.0
檢索結果aggs   年齡分佈：32 1
檢索結果aggs    所屬年齡的 平均薪資：25571.0

二、ES數組的扁平化處理

注：數組裏邊是對象的時候需要注意，若數組裏邊是基礎數據沒關係

Elasticsearch 鼓勵你在創建索引的時候就 扁平化 你的數據，這樣做可以獲取最好的搜索性能。在每一篇文檔裏面冗餘一些數據可以避免join操作。

PUT users/_doc/1
{
  "group" : "fans",
  "user" : [ 
    {
      "first" : "John",
      "last" :  "Smith"
    },
    {
      "first" : "Alice",
      "last" :  "White"
    }
  ]
}

##ES會存儲成如下結構：
{
  "group" :        "fans",
  "user.first" : [ "alice", "john" ],
  "user.last" :  [ "smith", "white" ]
}

##然後我們搜索的時候回出現 alice smith的結果。
GET users/_search
{
  "query": {
    "bool": {
      "must": [
        { "match": { "user.first": "Alice" }},
        { "match": { "user.last":  "Smith" }}
      ]
    }
  }
}

爲了避免數據被扁平化：

##爲避免帶有Object的數組 在存儲的時候被扁平化，我們需要添加Nested
PUT users
{
  "mappings": {
    "properties": {
      "user": {
        "type": "nested" 
      }
    }
  }
}

PUT users/_doc/1
{
  "group" : "fans",
  "user" : [
    {
      "first" : "John",
      "last" :  "Smith"
    },
    {
      "first" : "Alice",
      "last" :  "White"
    }
  ]
}

三、商城中ES商品模型

商城中，商品上架SKU在ES中的模型

ES存儲模型：

PUT product
{
  "mappings": {
    "properties": {
      "skuId": {
        "type": "long"
      },
      "spuId": {
        "type": "keyword"
      },
      "skuTitle": {
        "type": "text",
        "analyzer": "ik_smart"
      },
      "skuPrice": {
        "type": "keyword"
      },
      "skuImg": {
        "type": "keyword",
        "index": false,
        "doc_values": false
      },
      "saleCount": {
        "type": "long"
      },
      "hasStock": {
        "type": "boolean"
      },
      "hotScore": {
        "type": "long"
      },
      "brandId": {
        "type": "long"
      },
      "catelogId": {
        "type": "long"
      },
      "brandName": {
        "type": "keyword",
        "index": false,
        "doc_values": false
      },
      "brandImg": {
        "type": "keyword",
        "index": false,
        "doc_values": false
      },
      "catelogName": {
        "type": "keyword",
        "index": false,
        "doc_values": false
      },
      "attrs": {
        "type": "nested", ##嵌套,防止數組扁平化
        "properties": {
          "attrId": {
            "type": "long"
          },
          "attrName": {
            "type": "keyword",
            "index": false,
            "doc_values": false
          },
          "attrValue": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

對應的JavaBean：

@Data
public class SkuEsModel {

    //商品ID
    private Long spuId;
    //sku_id
    private Long skuId;
    //標題
    private String skuTitle;
    //價格
    private BigDecimal skuPrice;
    //圖片
    private String skuImg;
    //銷售量
    private Long saleCount;
    //是否還有庫存
    private Boolean hasStock;
    //熱度評分
    private Long hotScore;
    //品牌ID
    private Long brandId;
    //品牌名
    private String brandName;
    //品牌圖片
    private String brandImg;
    //分類ID
    private Long catalogId;
    //分類名
    private String catalogName;
    //屬性
    private List<AttrsEsModel> attrs;
}


@Data
public class AttrsEsModel {
    //屬性ID
    private Long attrId;
    //屬性名
    private String attrName;
    //屬性值
    private String attrValue;
}