es 的一個特點,版本更新快,且各個大版本 不兼容,並且 查詢語法變化也特別多
並且需要注意的分詞器要和es的版本完全一致纔可以 ,否則報錯
es 最新版本爲 7,安裝版本與公司一致 6.7.1
mac 安裝 使用 6.3.0
brew install elasticsearch
然後修改配置文件
network.host: 0.0.0.0 ##千萬不可以是127.0.0.1 localhost 否則無法被其他電腦訪問
http.port: 9200
bootstrap.memory_lock: false
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
#cluster.name: my-application
然後安裝 中文分詞器插件
./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.3.0/elasticsearch-analysis-ik-6.3.0.zip
最後運行即可
centos7安裝 es 6.7.1
下載地址
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.7.1.rpm
使用yum 離線安裝
yum install ./elasticsearch-6.7.1.rpm
修改配置文件
cd /etc/elasticsearch/
vi elasticsearch.yml
network.host: 0.0.0.0
http.port: 9200
bootstrap.memory_lock: false
path.data: /var/lib/elasticsearch
path.logs: /var/log/elasticsearch
#cluster.name: my-application
安裝中文分詞器
./elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.3.0/elasticsearch-analysis-ik-6.3.0.zip
centos7 安裝 kibana
wget [https://artifacts.elastic.co/downloads/kibana/kibana-6.7.1-x86_64.rpm](https://artifacts.elastic.co/downloads/kibana/kibana-6.7.1-x86_64.rpm)
Kibana 需要修改自己的 public host 和 es host 和port
cd cd /etc/kibana/
vi kibana.yml
elasticsearch.hosts: ["http://0.0.0.0:9200"]
server.host: "0.0.0.0"
server.port: 5601
es kibana 目錄都加入到 環境變量
vi /etc/profile
export ELASTICSEARCH_HOME=/usr/share/elasticsearch
export KIBANA_HOME=/usr/share/kibana
export PATH=$PATH:$ELASTICSEARCH_HOME/bin:$KIBANA_HOME/bin
分別啓動他們 先前臺 啓動觀察 ,最後後臺啓動
切換用戶 su elasticsearch 這裏會報一個小錯【This account is currently not available]
cd /usr/share/elasticsearch/bin
./elasticsearch 觀察
./elasticsearch -d
/usr/share/kibana/bin
./kibana 觀察
nohup /usr/share/kibana/bin/kibana & >> /dev/null 後臺啓動
安裝完elasticsearch之後系統會自動創建一個elasticsearch用戶,在啓動elasticsearch的時候需要切換該用戶啓動,但是在我切換到elasticsearch用戶的時候顯示This account is currently not available。
查資料發現是因爲該用戶的shell不可用,查看/etc/passwd文件發現此用戶的shell是/sbin/nologin需修改成/bin/bash
修改完後就可以切換到elasticsearch用戶了。
/etc/elasticsearch/
編輯 /etc/security/limits.conf,追加以下內容;
* soft nofile 65536
* hard nofile 65536
此文件修改後需要重新登錄用戶,纔會生效
[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]
解決辦法:
編輯 /etc/sysctl.conf,追加以下內容:
vm.max_map_count=655360
保存後,執行:
sysctl -p
java 調用 es
http://www.cnblogs.com/xyddm/p/10417371.html
程序中遇到 【# [ElasticsearchStatusException contains unrecognized parameter: [ccs_minimize_roundtrips]]]](https://stackoverflow.com/questions/55602377/elasticsearchstatusexception-contains-unrecognized-parameter-ccs-minimize-roun)
這個主要是是 es jar 版本的問題 ,切換和服務器相同的版本即可
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>6.7.1</version>
</dependency>
package com.hhz.es.searchhhz;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.termvectors.TermVectorsFilter;
import org.elasticsearch.client.*;
import org.elasticsearch.client.transport.TransportClient;
import org.elasticsearch.index.query.BoolQueryBuilder;
import org.elasticsearch.index.query.QueryBuilders;
import org.elasticsearch.index.query.TermQueryBuilder;
import org.apache.http.HttpHost;
import org.apache.http.client.config.RequestConfig.Builder;
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder;
import org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback;
import org.elasticsearch.client.RestClientBuilder.RequestConfigCallback;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.transport.TransportAddress;
import org.elasticsearch.script.ScriptType;
import org.elasticsearch.script.mustache.SearchTemplateRequestBuilder;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.SearchHits;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.elasticsearch.transport.client.PreBuiltTransportClient;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestClientBuilder;
import java.io.IOException;
import java.net.InetAddress;
import java.net.UnknownHostException;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import org.elasticsearch.index.search.MultiMatchQuery;
import org.springframework.data.elasticsearch.core.ElasticsearchTemplate;
import org.springframework.data.elasticsearch.core.query.NativeSearchQueryBuilder;
import org.springframework.data.elasticsearch.core.query.SearchQuery;
import org.springframework.data.elasticsearch.core.query.NativeSearchQuery;
public class searchadpt {
private static TransportClient client;
private ElasticsearchClient esclient ;
private RestHighLevelClient esrestclient;
// public Client getClient(){
//
// esclient =
// }
private static String hosts = "192.168.199.102";// "0.0.0.0";//"172.16.8.3"; // 集羣地址,多個用,隔開
private static int port = 9200; // 使用的端口號
private static String schema = "http"; // 使用的協議
private static ArrayList<HttpHost> hostList = null;
private static int connectTimeOut = 1000; // 連接超時時間
private static int socketTimeOut = 30000; // 連接超時時間
private static int connectionRequestTimeOut = 500; // 獲取連接的超時時間
private static int maxConnectNum = 100; // 最大連接數
private static int maxConnectPerRoute = 100; // 最大路由連接數
static {
hostList = new ArrayList<>();
String[] hostStrs = hosts.split(",");
for (String host : hostStrs) {
hostList.add(new HttpHost(host, port, schema));
}
}
public RestHighLevelClient initRestClient() {
RestClientBuilder builder = RestClient.builder(hostList.toArray(new HttpHost[0]));
// 異步httpclient連接延時配置
builder.setRequestConfigCallback(new RequestConfigCallback() {
@Override
public Builder customizeRequestConfig(Builder requestConfigBuilder) {
requestConfigBuilder.setConnectTimeout(connectTimeOut);
requestConfigBuilder.setSocketTimeout(socketTimeOut);
requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeOut);
return requestConfigBuilder;
}
});
// 異步httpclient連接數配置
builder.setHttpClientConfigCallback(new HttpClientConfigCallback() {
@Override
public HttpAsyncClientBuilder customizeHttpClient(HttpAsyncClientBuilder httpClientBuilder) {
httpClientBuilder.setMaxConnTotal(maxConnectNum);
httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute);
return httpClientBuilder;
}
});
// RestClient client = builder.build(); // new RestClient(builder);
RestHighLevelClient client =new RestHighLevelClient(builder);
if(client != null)
{
// client.performRequest()
System.out.println("連接成功!"+client.cluster());
}
return client;
}
public Client initESClient() {
// 配置你的es,如果你的集羣名稱不是默認的elasticsearch,需要以下這步
Settings settings = Settings.builder().put("cluster.name","elasticsearch_apio")
.put("client.transport.ignore_cluster_name",false)
.put("client.transport.sniff",true)
.build();
// 這裏可以同時連接集羣的服務器,可以多個,並且連接服務是可訪問的 172.16.8.3
try {
// 創建client
client = new PreBuiltTransportClient(settings).addTransportAddress(new TransportAddress(InetAddress.getByName("172.16.8.3"),9200));
} catch (Exception e) {
e.printStackTrace();
}finally
{
if(client != null)
{
System.out.println("連接成功!");
}
}
return client;
}
public void closeESClient() {
client.close();
System.out.println("連接關閉!");
}
// private TermVectorsFilter tf=null;
// private ElasticsearchClient client =null;
// private ElasticsearchTemplate et= null;
//
// SearchQuery searchQuery = new NativeSearchQueryBuilder().withFilter(must()).build();
// TermQueryBuilder termQueryBuilderfilter=TermQueryBuilder.fromXContent(null);
// termQueryBuilderfilter.
// BoolQueryBuilder bbf
// MultiMatchQuery.QueryBuilder
public void search(String index, String type, String name) throws IOException {
BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();
boolBuilder.must(QueryBuilders.matchQuery("admin_tag", name)); // 這裏可以根據字段進行搜索,must表示符合條件的,相反的mustnot表示不符合條件的
// boolBuilder.must(QueryBuilders.matchQuery("id", tests.getId().toString()));
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
sourceBuilder.query(boolBuilder);
sourceBuilder.from(0);
sourceBuilder.size(100); // 獲取記錄數,默認10
sourceBuilder.fetchSource(new String[] { "id", "name" }, new String[] {}); // 第一個是獲取字段,第二個是過濾的字段,默認獲取全部
SearchRequest searchRequest = new SearchRequest(index);
searchRequest.types(type);
searchRequest.source(sourceBuilder);
SearchResponse response = esrestclient.search(searchRequest, RequestOptions.DEFAULT);
// System.out.println("search: " + JSON.toJSONString(response));
SearchHits hits = response.getHits();
SearchHit[] searchHits = hits.getHits();
for (SearchHit hit : searchHits) {
System.out.println("search -> " + hit.getSourceAsString());
}
}
public List<Map<String,Object>> searchZd(String query, String msg) {
Map<String, Object> template_params = new HashMap<String, Object>();
template_params.put("param_gender", msg);
template_params.put("param_query", query);
final List<Map<String, Object>> list = new ArrayList<Map<String, Object>>();
//模板查詢
SearchResponse sr = new SearchTemplateRequestBuilder(client)
.setScript("{\n" +
" \"query\" : {\n" +
" \"match\" : {\n" +
" \"{{param_query}}\" : \"{{param_gender}}\"\n" +
" }\n" +
" }\n" +
"}")
.setScriptType(ScriptType.INLINE)
.setScriptParams(template_params)
.setRequest(new SearchRequest())
.get()
.getResponse();
long totalHits1 = sr.getHits().totalHits; //命中個數
for (SearchHit searchHit : sr.getHits().getHits()) {
final Map<String, Object> sourceAsMap = searchHit.getSourceAsMap();
list.add(sourceAsMap);
}
System.out.println(totalHits1 + "---" + sr.toString());
return list;
}
public static void main(String[] args)throws IOException{
searchadpt se=new searchadpt();
se.esrestclient= se.initRestClient();
se.search("goods","goods_type","裝修記錄");
// se.initESClient();
// List<Map<String,Object>> result=se.searchZd("admin_tag","裝修記錄");
//
// se.closeESClient();
}
}
scala
import org.apache.http.HttpHost
import org.apache.http.client.config.RequestConfig.Builder
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder
import org.elasticsearch.action.search.SearchRequest
import org.elasticsearch.action.search.SearchResponse
import org.elasticsearch.client._
import org.elasticsearch.client.RestClientBuilder.HttpClientConfigCallback
import org.elasticsearch.client.RestClientBuilder.RequestConfigCallback
import org.elasticsearch.client.transport.TransportClient
import org.elasticsearch.common.settings.Settings
import org.elasticsearch.common.transport.TransportAddress
import org.elasticsearch.index.query.BoolQueryBuilder
import org.elasticsearch.index.query.QueryBuilders
import org.elasticsearch.script.ScriptType
import org.elasticsearch.script.mustache.SearchTemplateRequestBuilder
import org.elasticsearch.search.SearchHit
import org.elasticsearch.search.SearchHits
import org.elasticsearch.search.builder.SearchSourceBuilder
import org.elasticsearch.transport.client.PreBuiltTransportClient
import java.io.IOException
import java.net.InetAddress
import java.util.ArrayList
import java.util.HashMap
import java.util.List
import java.util.Map
import org.apache.http.client.config.RequestConfig
import org.apache.http.impl.client.BasicCredentialsProvider
import org.elasticsearch.action.index.IndexRequest
object searchadpt {
private var client = null
// public Client getClient(){
//
// esclient =
// }
// private TermVectorsFilter tf=null;
// private ElasticsearchClient client =null;
// private ElasticsearchTemplate et= null;
// SearchQuery searchQuery = new NativeSearchQueryBuilder().withFilter(must()).build();
// TermQueryBuilder termQueryBuilderfilter=TermQueryBuilder.fromXContent(null);
// termQueryBuilderfilter.
// BoolQueryBuilder bbf
// MultiMatchQuery.QueryBuilder
@throws[IOException]
def main(args: Array[String]): Unit = {
val eshosts = "0.0.0.0"//"172.16.8.3"; // 集羣地址,多個用,隔開
val esport = 9200// 使用的端口號
val schema = "http"// 使用的協議
var hostList = null
val connectTimeOut = 1000// 連接超時時間
val socketTimeOut = 30000
val connectionRequestTimeOut = 500// 獲取連接的超時時間
val maxConnectNum = 100// 最大連接數
val maxConnectPerRoute = 100// 最大路由連接數
val builder = RestClient.builder(new HttpHost(eshosts, esport, "http"))
// 異步httpclient連接延時配置
builder.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
override def customizeRequestConfig(requestConfigBuilder: RequestConfig.Builder): RequestConfig.Builder = {
requestConfigBuilder.setConnectTimeout(connectTimeOut)
requestConfigBuilder.setSocketTimeout(socketTimeOut)
requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeOut)
requestConfigBuilder
}
})
// 異步httpclient連接數配置
builder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
httpClientBuilder.setMaxConnTotal(maxConnectNum)
httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute)
httpClientBuilder
}
})
// RestClient client = builder.build(); // new RestClient(builder);
val client = new RestHighLevelClient(builder)
if (client != null) { // client.performRequest()
System.out.println("連接成功!" + client.cluster)
}
val index= "goods"
val `type` ="goods_type"
val name ="裝修記錄"
val indexreq= new IndexRequest(index,`type`,name)
val res = client.index(indexreq,RequestOptions.DEFAULT)
print(res)
// .index(index=self.index_name, doc_type=self.index_type, body=doc)
val boolBuilder = QueryBuilders.boolQuery
boolBuilder.must(QueryBuilders.matchQuery("admin_tag", name))// 這裏可以根據字段進行搜索,must表示符合條件的,相反的mustnot表示不符合條件的
// boolBuilder.must(QueryBuilders.matchQuery("id", tests.getId().toString()));
val sourceBuilder = new SearchSourceBuilder
sourceBuilder.query(boolBuilder)
sourceBuilder.from(0)
sourceBuilder.size(100)// 獲取記錄數,默認10
sourceBuilder.fetchSource(Array[String]("id", "name"), Array[String]())// 第一個是獲取字段,第二個是過濾的字段,默認獲取全部
val searchRequest = new SearchRequest(index)
searchRequest.types(`type`)
searchRequest.source(sourceBuilder)
// searchRequest.setCcsMinimizeRoundtrips(false)
val response = client.search(searchRequest, RequestOptions.DEFAULT)
// System.out.println("search: " + JSON.toJSONString(response));
val hits = response.getHits
val searchHits = hits.getHits
for (hit <- searchHits) {
System.out.println("search -> " + hit.getSourceAsString)
}
// val credentialsProvider = new BasicCredentialsProvider()
// val srcClient :RestHighLevelClient= new RestHighLevelClient(RestClient.builder(new HttpHost(eshosts, esport, "http")).setHttpClientConfigCallback((httpClientBuilder: HttpAsyncClientBuilder) => httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)))
// val se = new searchadpt
// se.esrestclient = se.srcClient
// se.search("topic_activity", "doc", "裝修記錄")
// se.initESClient();
// List<Map<String,Object>> result=se.searchZd("admin_tag","裝修記錄");
// se.closeESClient();
}
// try hostList = new ArrayList[HttpHost]
// val hostStrs = hosts.split(",")
// for (host <- hostStrs) {
// hostList.add(new HttpHost(host, port, schema))
// }
}
class searchadpt {
private val esclient = null
private var esrestclient = null
private val eshosts = "0.0.0.0"//"172.16.8.3"; // 集羣地址,多個用,隔開
private val esport = 9200// 使用的端口號
private val schema = "http"// 使用的協議
private var hostList = null
private val connectTimeOut = 1000// 連接超時時間
private val socketTimeOut = 30000
private val connectionRequestTimeOut = 500// 獲取連接的超時時間
private val maxConnectNum = 100// 最大連接數
private val maxConnectPerRoute = 100// 最大路由連接數
val credentialsProvider = new BasicCredentialsProvider()
val srcClient :RestHighLevelClient= new RestHighLevelClient(RestClient.builder(new HttpHost(eshosts, esport, "http")).setHttpClientConfigCallback((httpClientBuilder: HttpAsyncClientBuilder) => httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)))
def initRestClient: RestHighLevelClient = {
val builder = RestClient.builder(new HttpHost(eshosts, esport, "http"))
// 異步httpclient連接延時配置
builder.setRequestConfigCallback(new RestClientBuilder.RequestConfigCallback() {
override def customizeRequestConfig(requestConfigBuilder: RequestConfig.Builder): RequestConfig.Builder = {
requestConfigBuilder.setConnectTimeout(connectTimeOut)
requestConfigBuilder.setSocketTimeout(socketTimeOut)
requestConfigBuilder.setConnectionRequestTimeout(connectionRequestTimeOut)
requestConfigBuilder
}
})
// 異步httpclient連接數配置
builder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
httpClientBuilder.setMaxConnTotal(maxConnectNum)
httpClientBuilder.setMaxConnPerRoute(maxConnectPerRoute)
httpClientBuilder
}
})
// RestClient client = builder.build(); // new RestClient(builder);
val client = new RestHighLevelClient(builder)
if (client != null) { // client.performRequest()
System.out.println("連接成功!" + client.cluster)
}
return client
}
// def initESClient: Client = { // 配置你的es,如果你的集羣名稱不是默認的elasticsearch,需要以下這步
// val settings = Settings.builder.put("cluster.name", "elasticsearch_apio").put("client.transport.ignore_cluster_name", false).put("client.transport.sniff", true).build
//// 這裏可以同時連接集羣的服務器,可以多個,並且連接服務是可訪問的 172.16.8.3
// try // 創建client
// searchadpt.client = new PreBuiltTransportClient(settings).addTransportAddress(new TransportAddress(InetAddress.getByName("172.16.8.3"), 9200))
// catch {
// case e: Exception =>
// e.printStackTrace()
// } finally if (searchadpt.client != null) System.out.println("連接成功!")
// searchadpt.client
// }
//
// def closeESClient(): Unit = {
// searchadpt.client.close()
// System.out.println("連接關閉!")
// }
@throws[IOException]
def search(index: String, `type`: String, name: String): Unit = {
val boolBuilder = QueryBuilders.boolQuery
boolBuilder.must(QueryBuilders.matchQuery("admin_tag", name))// 這裏可以根據字段進行搜索,must表示符合條件的,相反的mustnot表示不符合條件的
// boolBuilder.must(QueryBuilders.matchQuery("id", tests.getId().toString()));
val sourceBuilder = new SearchSourceBuilder
sourceBuilder.query(boolBuilder)
sourceBuilder.from(0)
sourceBuilder.size(100)// 獲取記錄數,默認10
sourceBuilder.fetchSource(Array[String]("id", "name"), Array[String]())// 第一個是獲取字段,第二個是過濾的字段,默認獲取全部
val searchRequest = new SearchRequest(index)
searchRequest.types(`type`)
searchRequest.source(sourceBuilder)
val response = srcClient.search(searchRequest, RequestOptions.DEFAULT)
// System.out.println("search: " + JSON.toJSONString(response));
val hits = response.getHits
val searchHits = hits.getHits
for (hit <- searchHits) {
System.out.println("search -> " + hit.getSourceAsString)
}
}
// def searchZd(query: String, msg: String): List[Map[String, AnyRef]] = {
// val template_params = new HashMap[String, AnyRef]
// template_params.put("param_gender", msg)
// template_params.put("param_query", query)
// val list = new ArrayList[Map[String, AnyRef]]
// //模板查詢
// val sr = new SearchTemplateRequestBuilder(searchadpt.client).setScript("{\n" + " \"query\" : {\n" + " \"match\" : {\n" + " \"{{param_query}}\" : \"{{param_gender}}\"\n" + " }\n" + " }\n" + "}").setScriptType(ScriptType.INLINE).setScriptParams(template_params).setRequest(new SearchRequest).get.getResponse
// val totalHits1 = sr.getHits.totalHits//命中個數
// for (searchHit <- sr.getHits.getHits) {
// val sourceAsMap = searchHit.getSourceAsMap
// list.add(sourceAsMap)
// }
// System.out.println(totalHits1 + "---" + sr.toString)
// list
// }
}
python
pip install elasticsearch
from elasticsearch import Elasticsearch
ip=['0.0.0.0']
port =9200
es=Elasticsearch(ip,port=port)
es.cluster.__dict__
_index_mappings = {
"mappings": {
'goods_type': {
"properties": {
"title": {
"type": "text",
"index": True,
"analyzer": "ik_max_word",
"search_analyzer": "ik_max_word"
},
"date": {
"type": "text",
"index": True
},
"keyword": {
"type": "text",
"index": False
},
"source": {
"type": "text",
"index": False
},
"link": {
"type": "text",
"index": False
}
}
}
}
}
es.indices.create(index='goods', body=_index_mappings)