ElasticSearch-2.3.5學習筆記

一、下載elasticSearch

下載地址：

https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/5.0.0-alpha5/elasticsearch-5.0.0-alpha5.tar.gz

二、安裝elasticSearch

1）安裝head插件；進入~/elasticsearch-2.3.5/bin，執行如下命令：

./plugin install mobz/elasticsearch-head

2）啓動；進入~/elasticsearch-2.3.5/bin，執行如下命令：

./bin/elasticsearch

3) 啓動日誌

[2016-08-20 11:33:33,712][INFO ][node                     ] [Spoilsport] version[2.3.5], pid[28820], build[90f439f/2016-07-27T10:36:52Z]
[2016-08-20 11:33:33,713][INFO ][node                     ] [Spoilsport] initializing ...
[2016-08-20 11:33:34,621][INFO ][plugins                  ] [Spoilsport] modules [reindex, lang-expression, lang-groovy], plugins [head], sites [head]
[2016-08-20 11:33:34,654][INFO ][env                      ] [Spoilsport] using [1] data paths, mounts [[/ (/dev/disk1)]], net usable_space [31.1gb], net total_space [111.8gb], spins? [unknown], types [hfs]
[2016-08-20 11:33:34,654][INFO ][env                      ] [Spoilsport] heap size [990.7mb], compressed ordinary object pointers [true]
[2016-08-20 11:33:34,655][WARN ][env                      ] [Spoilsport] max file descriptors [10240] for elasticsearch process likely too low, consider increasing to at least [65536]
[2016-08-20 11:33:37,750][INFO ][node                     ] [Spoilsport] initialized
[2016-08-20 11:33:37,751][INFO ][node                     ] [Spoilsport] starting ...
[2016-08-20 11:33:37,862][INFO ][transport                ] [Spoilsport] publish_address {127.0.0.1:9300}, bound_addresses {[fe80::1]:9300}, {[::1]:9300}, {127.0.0.1:9300}
[2016-08-20 11:33:37,868][INFO ][discovery                ] [Spoilsport] elasticsearch/8rCs0kfJRcS-B0CEveqqvg
[2016-08-20 11:33:40,941][INFO ][cluster.service          ] [Spoilsport] new_master {Spoilsport}{8rCs0kfJRcS-B0CEveqqvg}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2016-08-20 11:33:40,959][INFO ][http                     ] [Spoilsport] publish_address {127.0.0.1:9200}, bound_addresses {[fe80::1]:9200}, {[::1]:9200}, {127.0.0.1:9200}
[2016-08-20 11:33:40,959][INFO ][node                     ] [Spoilsport] started
[2016-08-20 11:33:41,032][INFO ][gateway                  ] [Spoilsport] recovered [1] indices into cluster_state
[2016-08-20 11:33:41,929][INFO ][cluster.routing.allocation] [Spoilsport] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[zhengyong][2], [zhengyong][4], [zhengyong][2], [zhengyong][4]] ...]).

三、驗證是否安裝成功

1）執行 curl ‘http://localhost:9200/?pretty’ ,顯示如下成功

{
  "name" : "Spoilsport",
  "cluster_name" : "elasticsearch",
  "version" : {
    "number" : "2.3.5",
    "build_hash" : "90f439ff60a3c0f497f91663701e64ccd01edbb4",
    "build_timestamp" : "2016-07-27T10:36:52Z",
    "build_snapshot" : false,
    "lucene_version" : "5.5.0"
  },
  "tagline" : "You Know, for Search"
}

2）訪問 http://localhost:9200/_plugin/head/ 能打開成功

四、創建分片

POST http://localhost:9200/indexName/
{
   "settings" : {
      "number_of_shards" : 3,
      "number_of_replicas" : 1
   }
}

五、創建索引

elasticsearch默認中文是分詞的，如果不手動設置”index”: “not_analyzed”，中文詞組就不能正常檢索

POST http://localhost:9200/indexName/_mapping/typeName?pretty

{
  "properties": {
    "@timestamp": {
      "format": "strict_date_optional_time||epoch_millis",
      "type": "date",
      "fields": {
        "raw": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    },
    "price": {
      "type": "string",
      "index": "not_analyzed"
    },
    "name": {
      "type": "string",
      "fields": {
        "raw": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    },
    "houseType": {
      "type": "string",
      "fields": {
        "raw": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    },
    "building": {
      "type": "string",
      "index": "not_analyzed"
    },
    "floorNumber": {
      "type": "string",
      "index": "not_analyzed"
    },
    "unit": {
      "type": "string",
      "index": "not_analyzed"
    },
    "doorNumber": {
      "type": "string",
      "index": "not_analyzed"
    },
    "area": {
      "type": "string",
      "fields": {
        "raw": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    },
    "location": {
      "type": "string",
      "fields": {
        "raw": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    },
    "region": {
      "type": "string",
      "fields": {
        "raw": {
          "index": "not_analyzed",
          "type": "string"
        }
      }
    },
    "url": {
      "type": "string",
      "index": "not_analyzed"
    }
  }
}

六、使用PUT命令插入數據

curl -XPUT http://localhost:9200/indexName/typeName/1 -d '{"first_name":"Jane","last_name":"Smith","age":32,"about":"I like to collect rock albums"}'

返回如下數據成功：

{
  "_index": "indexName",
  "_type": "typeName",
  "_id": "1",
  "_version": 1,
  "_shards": {
    "total": 3,
    "successful": 1,
    "failed": 0
  },
  "created": true
}

七、集羣配置

1）集羣節點elasticsearch1配置elasticsearch.yml

node.name: "esNode-1"
cluster.name: myCluster
transport.tcp.port: 9300
http.port: 9200
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300", "127.0.0.1:9301"]

2）集羣節點elasticsearch2配置elasticsearch.yml

node.name: "esNode-2"
cluster.name: myCluster
transport.tcp.port: 9301
http.port: 9201
discovery.zen.ping.unicast.hosts: ["127.0.0.1:9300", "127.0.0.1:9301"]

八、高級配置

修改elasticsearch.yml文件配置

node.name: "mynode"
cluster.name: myCluster
path.data: /home/admin/elasticsearch/data
path.work: /home/admin/elasticsearch/work
path.logs: /home/admin/output/elasticsearch/logs
bootstrap.mlockall: true
# 防止腦裂，要求可用節點必須大於quorum
discovery.zen.minimum_master_nodes: 3
discovery.zen.ping.multicast.enabled: false
discovery.zen.ping.unicast.hosts: ["192.168.47.82:9300", "192.168.47.11:9300", "172.31.238.19:9300", "192.168.47.28:9300", "192.168.47.29:9300"]
index:
  analysis:
    analyzer:
      ik:
        alias: [ik_analyzer]
        type: org.elasticsearch.index.analysis.IkAnalyzerProvider
      ik_max_word:
        type: ik
        use_smart: false
      ik_smart:
        type: ik
        use_smart: true

# fielddata緩存
indices.fielddata.cache.size: "4G"
indices.cache.filter.size: "1G"

# 穩定性：增強集羣穩定性
discovery.zen.ping.timeout: 30s

# 安全性：防止刪除所有索引
action.disable_delete_all_indices: true
script.groovy.sandbox.enabled: false
action.auto_create_index: false

# 慢查詢日誌
index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
#index.search.slowlog.threshold.query.trace: 500ms
index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
#index.search.slowlog.threshold.fetch.trace: 200ms

elasticsearch-2.3.5示列代碼

一、簡介

該示列是基於elasticsearch-2.3.5版本開發，參考地址：https://www.elastic.co/guide/en/elasticsearch/client/java-api/2.3/index.html

二、Maven依賴

<dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>2.3.5</version>
</dependency>
<dependency>
    <groupId>com.alibaba</groupId>
    <artifactId>fastjson</artifactId>
    <version>1.1.35</version>
</dependency>

三、Java代碼

package easticsearch;

import com.alibaba.fastjson.JSON;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.InetSocketTransportAddress;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;

import java.net.InetAddress;
import java.util.ArrayList;
import java.util.Date;
import java.util.List;
import java.util.Map;

/**
 * Created by zhengyong on 16/11/22.
 */
public class Elasticsearch {

    private static TransportClient client   = null;

    /**
     * es index
     */
    private static final String    ES_INDEX = "database_test";
    /**
     * es type
     */
    private static final String    ES_TYPE  = "user";

    public static void main(String[] args) throws Exception {

        createClient();

        String _id = createIndex();

        getDataResponse(_id);

        updateData(_id);

        getDataResponse(_id);

        QueryBuilder builder = QueryBuilders.matchAllQuery();
        // QueryBuilder builder = QueryBuilders.termQuery("username", "zhengyong");
        queryDataList(builder);

        // deleteDataResponse(_id);

        shutdown();
    }

    private static void createClient() throws Exception {
        if (client == null) {
            synchronized (Elasticsearch.class) {
                Settings settings = Settings.settingsBuilder().put("cluster.name", "myCluster").build();
                client = TransportClient.builder().settings(settings).build().addTransportAddress(new InetSocketTransportAddress(InetAddress.getByName("localhost"),
                                                                                                                                 9300));
            }
        }
    }

    /**
     * 創建es庫並導入數據
     * 
     * @return _id
     */
    private static String createIndex() {
        String json = "{" + "\"username\":\"zhengyong\"," + "\"postDate\":\"" + new Date().toLocaleString() + "\","
                      + "\"message\":\"create\"" + "}";

        IndexResponse response = client.prepareIndex(ES_INDEX, ES_TYPE).setSource(json).get();

        System.out.println(String.format("create index response: %s", response.toString()));

        return response.getId();
    }

    /**
     * 根據_id獲取數據
     * 
     * @param _id 唯一標識
     * @return GetResponse
     */
    private static GetResponse getDataResponse(String _id) {
        GetResponse response = client.prepareGet(ES_INDEX, ES_TYPE, _id).get();
        System.out.println(String.format("get data response: %s", JSON.toJSONString(response.getSource())));
        return response;
    }

    /**
     * 根據查詢條件查詢結果集
     * 
     * @param queryBuilder 查詢條件
     * @return List
     */
    private static List<String> queryDataList(QueryBuilder queryBuilder) {
        SearchResponse sResponse = client.prepareSearch(ES_INDEX).setTypes(ES_TYPE).setQuery(queryBuilder).setSize(1000).execute().actionGet();
        SearchHits hits = sResponse.getHits();

        List<String> list = new ArrayList<>();
        SearchHit[] hitArray = hits.hits();
        for (SearchHit hit : hitArray) {
            Map<String, Object> map = hit.getSource();

            String username = (String) map.get("username");
            String postDate = (String) map.get("postDate");
            String message = (String) map.get("message");

            StringBuilder br = new StringBuilder();
            br.append(username).append("_").append(message).append("_").append(postDate);
            list.add(br.toString());
        }

        System.out.println(String.format("query data count=%s, list : %s", list.size(), JSON.toJSONString(list)));

        return list;
    }

    /**
     * 根據_id刪除數據
     * 
     * @param _id 唯一標識
     * @return DeleteResponse
     */
    private static DeleteResponse deleteDataResponse(String _id) {
        DeleteResponse response = client.prepareDelete(ES_INDEX, ES_TYPE, _id).get();
        System.out.println(String.format("delete data response: %s", JSON.toJSONString(response)));
        return response;
    }

    /**
     * 根據_id跟下數據
     * 
     * @param _id 唯一標識
     * @throws Exception
     */
    private static void updateData(String _id) throws Exception {
        UpdateRequest updateRequest = new UpdateRequest();
        updateRequest.index(ES_INDEX);
        updateRequest.type(ES_TYPE);
        updateRequest.id(_id);

        String json = "{" + "\"username\":\"lisi\"," + "\"postDate\":\"" + new Date().toLocaleString() + "\","
                      + "\"message\":\"update\"" + "}";

        updateRequest.doc(json);

        client.update(updateRequest).get();
    }

    private static void shutdown() {
        if (client != null) {
            client.close();
        }
    }

}

四、異常信息

開發時由於maven依賴elasticsearch版本與安裝包版本不一致，出現以下異常：

Exception in thread "main" NoNodeAvailableException[None of the configured nodes are available: []]
    at org.elasticsearch.client.transport.TransportClientNodesService.ensureNodesAreAvailable(TransportClientNodesService.java:314)
    at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:228)
    at org.elasticsearch.client.transport.support.TransportProxyClient.execute(TransportProxyClient.java:63)
    at org.elasticsearch.client.transport.TransportClient.doExecute(TransportClient.java:326)
    at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:80)
    at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:54)
    at org.elasticsearch.action.ActionRequestBuilder.get(ActionRequestBuilder.java:62)
    at easticsearch.Elasticsearch.createIndex(Elasticsearch.java:51)
    at easticsearch.Elasticsearch.main(Elasticsearch.java:23)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:497)
    at com.intellij.rt.execution.application.AppMain.main(AppMain.java:144)

五、解決辦法

需要elasticsearch-2.3.5與client的jar版本一致

ElasticSearch-2.3.5學習筆記

ElasticSearch-2.3.5學習筆記

一、下載elasticSearch

二、安裝elasticSearch

三、驗證是否安裝成功

四、創建分片

五、創建索引

六、使用PUT命令插入數據

七、集羣配置

八、高級配置

elasticsearch-2.3.5示列代碼

一、簡介

二、Maven依賴

三、Java代碼

四、異常信息

五、解決辦法

六、運行效果

jmeter使用指南

ElasticSearch-2.3.5學習筆記

kibana 4.5.0安裝

cassandra學習筆記

zookeeper實現分佈式鎖

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結