灰灰商城-整合RestHighLevelClient筆記（尚硅谷穀粒商城2020）夠豪橫！

---

灰灰商城-分佈式高級篇-2

整合 RestHighLevelClient

導入es的rest-high-level-client

<dependency>
	<groupId>org.elasticsearch.client</groupId>    
	<artifactId>elasticsearch-rest-high-level-client</artifactId>  
	<version>7.4.2</version>
</dependency>

• 注意Spring Boot（2.2.5版本）中默認整合es版本爲6.8.7

需要在pom中指定

<properties>      
    <elasticsearch.version>7.4.2</elasticsearch.version>
</properties>

添加配置類

• 給容器中注入一個RestHighLevelClient

@Configuration
public class GreymallElasticSearchConfig {

    public static final RequestOptions COMMON_OPTIONS;
    static {
        RequestOptions.Builder builder = RequestOptions.DEFAULT.toBuilder();
//        builder.addHeader("Authorization", "Bearer " + TOKEN);
//        builder.setHttpAsyncResponseConsumerFactory(
//                new HttpAsyncResponseConsumerFactory
//                        .HeapBufferedResponseConsumerFactory(30 * 1024 * 1024 * 1024));
        COMMON_OPTIONS = builder.build();
    }

    /**
     * 配置ElasticSearch RestHighLevelClient
     * @return
     */
    @Bean
    public RestHighLevelClient esRestClient(){

        return new RestHighLevelClient(
                RestClient.builder(
                        new HttpHost("ES部署的ip", 9200, "http")
                ));
    }
}

保存索引

 // 測試保存數據到es
 @Test
 void indexData() throws IOException {
     IndexRequest request = new IndexRequest("users");
     request.id("1");
     User user = new User();
     user.setUserName("wei-xhh");
     user.setAge(20);
     user.setGender("男");
     String jsonString = JSON.toJSONString(user);
     request.source(jsonString, XContentType.JSON); // 保存的內容

     IndexResponse index = client.index(request, GreymallElasticSearchConfig.COMMON_OPTIONS);

 }

更多可以參考文檔

查詢文檔

    // 測試查詢數據
    @Test
    void searchData2() throws IOException {
//        GET /bank/_search
//        {
//            "query": {
//            "term": {
//                "address": {
//                    "value": "mill"
//                }
//            }
//        },
//            "aggs": {
//            "aggAvg": {
//                "avg": {
//                    "field":"age"
//                }
//            },
//            "balanceAgg":{
//                "terms": {
//                    "field": "balance",
//                            "size": 10
//                }
//            }
//        }
//        }
        SearchRequest searchRequest = new SearchRequest();
        searchRequest.indices("bank");

        SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
        searchSourceBuilder.query(QueryBuilders.termQuery("address", "mill"));

        // 按照薪資的值進行分佈
        TermsAggregationBuilder balance = AggregationBuilders.terms("balanceAgg")
                .field("balance");
        searchSourceBuilder.aggregation(balance);

        // 計算平均年齡
        AvgAggregationBuilder age = AggregationBuilders.avg("ageAvg").field("age");
        searchSourceBuilder.aggregation(age);

        searchRequest.source(searchSourceBuilder);
        System.out.println(searchSourceBuilder.toString());

        SearchResponse search = client.search(searchRequest, GreymallElasticSearchConfig.COMMON_OPTIONS);

        System.out.println(search.toString());

        // 封裝結果
        SearchHits hits = search.getHits();
        SearchHit[] data = hits.getHits();
        for (SearchHit datum : data) {
            String sourceAsString = datum.getSourceAsString();
            ResultData resultData = JSON.parseObject(sourceAsString, ResultData.class);
            System.out.println(resultData);
        }

        Aggregations aggregations = search.getAggregations();
//        List<Aggregation> aggregationsData = aggregations.asList();
//        for (Aggregation aggregationsDatum : aggregationsData) {
//            String name = aggregationsDatum.getName();
//            System.out.println(name);
//        }
        Terms balanceAgg = aggregations.get("balanceAgg");
        for (Terms.Bucket bucket : balanceAgg.getBuckets()) {
            String keyAsString = bucket.getKeyAsString();
            System.out.println("薪資" + keyAsString);
        }

        Avg aggAvg = aggregations.get("ageAvg");
        double value = aggAvg.getValue();
        System.out.println("平均年齡" + value);

    }

更多可以參考文檔

商品上架

分析應該怎麼存數據到ES

• 按照Sku的信息檢索
• 按照品牌檢索
• 按照當前分類下的產品檢索
• 按照Spu的信息檢索

保存的文檔應該怎樣呢？

情況1：

保存sku信息和屬性信息（都在一個索引下）
如：

{
    "skuId":1,
    "skuTitle":"華爲xx",
    "price":998,
    "saleCount":99,
    "attrs":[
        {"尺寸":5寸},
        {"CPU":"高通945"}，
        {"分辨率":"全高清"}
    ]
}

後果：產生冗餘字段

假設
100萬個產品，有20個屬性（20個屬性假設有2kb）
1000000 * 2kb = 2000MB = 2G 內存

---

情況2：

保存有用的信息（不同索引）

如：

sku索引

{
    "skuId":1,
    "spuId":11,
    "xxx":"xxx"
}

attr索引, 可以看到屬性只存了一次

{
    "spuId":11,
    "attrs":[
        {"尺寸":5寸},
        {"CPU":"高通945"}，
        {"分辨率":"全高清"}
    ]
}

沒有像第一種情況的冗餘了。

但實際項目中需要根據attrs動態查詢到sku：

如 搜索 小米；

假設帶小米的商品有10000個，涉及到4000個spu

需要做分步查詢：查出4000個spu對應的所有可能屬性。

一個請求：
esClient : spuId:[4000個spuId] 4000 * 8個字節 = 32kb

10000個人請求：

32kb * 10000 = 32000Mb = 32GB

只能選第一個：空間換時間

---

商品的映射信息

在kibana中執行

PUT product

{
  "mappings": {
    "skuId": {
      "type": "long"
    },
    "spuId": {
      "type": "keyword"
    },
    "skuTitle": {
      "type": "text",
      "analyzer":"ik_smart"
    },
    "skuPrice":{
      "type":"keyword"
    },
    "skuImg":{
      "type":"keyword",
      "index":false,
      "doc_values":false
    },
    "saleCount":{
      "type":"long"
    },
    "hasStock":{
      "type":"boolean"
    },
    "hotScore":{
      "type":"long"
    },
    "brandId":{
      "type":"long"
    },
    "catelogId":{
      "type":"long"
    },
    "brandName":{
      "type":"keyword",
      "index":false,
      "doc_values":false
    },
    "brandImg":{
      "type":"keyword",
      "index":false,
      "doc_values":false
    },
    "catalogName":{
      "type":"keyword",
      "index":false,
      "doc_values":false
    },
    "attrs":{
      "type":"nested",
      "properties":{
        "attrId":{
          "type":"long"
        },
        "attrName":{
          "type":"keyword",
          "index":false,
          "doc_values":false
        },
        "attrValue":{
          "type":"keyword"
        }
      }
    }
  }
}

nested的數據類型介紹

[https://www.elastic.co/guide/en/elasticsearch/reference/7.x/nested.html]

扁平化處理

如ES開發文檔的例子

PUT my_index/_doc/1

{
  "group" : "fans",
  "user" : [ 
    {
      "first" : "John",
      "last" :  "Smith"
    },
    {
      "first" : "Alice",
      "last" :  "White"
    }
  ]
}

保存了上述數據後 ，會被扁平化處理

上述數據：存入了兩個用戶，有first和last屬性
保存被扁平化成這樣：

{
  "group" :        "fans",
  "user.first" : [ "alice", "john" ],
  "user.last" :  [ "smith", "white" ]
}

假設檢索了Alice,Smith他們對應着不同的用戶，但卻可以被檢索到

GET my_index/_search

{
  "query": {
    "bool": {
      "must": [
        { "match": { "user.first": "Alice" }},
        { "match": { "user.last":  "Smith" }}
      ]
    }
  }
}

所以使用nested嵌入式屬性，使用後就不會出現扁平化。

商品在ES中保存的數據模型

@Data
public class SkuEsModel {
    private Long skuId;

    private Long spuId;

    private String skuTitle;

    private BigDecimal skuPrice;

    private String skuImg;

    private Long saleCount;

    private Boolean hasStock;

    private Long hotScore;

    private Long brandId;

    private Long catelogId;

    private String brandName;

    private String brandImg;

    private String catalogName;

    private List<Attrs> attrs;

    @Data
    public static class Attrs{

        private Long attrId;

        private String attrName;

        private String attrValue;

    }
}