Elasticsearch 查詢索引結果時,用於分頁的兩個屬性 from和size。用於查詢時,默認from=0、size=0。而在生產環境時,查詢的數據要大很多。有時是幾百G,甚至P級。
在設置size時,超過10000。會出現報錯。
Result window is too large, from + size must be less than or equal to:[10000] but was
[12000]. See the scroll api for a more efficient way to requestlarge data sets. This
limit can be set by changing the[index.max_result_window] index level parameter
說是size 不能超過10000! 查閱資料,顯示可以通過設置,改變:
PUT _settings
{
"index": {
"max_result_window": "10000000"
}
}
經過測試,還是不可以。原因是我的Elasticsearch是集羣環境(部署了三個master兩個data節點)。上面的方式只能改變單節點的size。
我想要不就通過yaml文件,不是環境變量。重新部署吧。大膽猜想,然後實踐。因爲是kubernetes部署的elasticsearch集羣。所以直接改他們的yaml文件,重新部署就行啦。
加上配置:
- name: "index.max_result_window"
value: "10000000"
然後部署:
kubectl apply -f es.yaml
但是查看狀態的時候,出錯。查看日誌:
*******************************************************************************
Found index level settings on node level configuration.
Since elasticsearch 5.x index level settings can NOT be set on the nodes
configuration like the elasticsearch.yaml, in system properties or command line
arguments.In order to upgrade all indices the settings must be updated via the
/${index}/_settings API. Unless all settings are dynamic all indices must be closed
in order to apply the upgradeIndices created in the future should use index templates
to set default values.
Please ensure all required values are updated on all indices by executing:
curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
"index.max_result_window" : "100000"
}'
*******************************************************************************
[2018-11-23T07:03:13,840][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [elasticsearch-master-0] uncaught exception in thread [main]
org.elasticsearch.bootstrap.StartupException: java.lang.IllegalArgumentException: node settings must not contain any index level settings
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.4.jar:5.6.4]
可以看到es 5.X以後,就不可以在yaml文件,系統屬性,以及命令行參數設置索引級別的配置啦。
因此只能通過他給的提示:
curl -XPUT 'http://localhost:9200/_all/_settings?preserve_existing=true' -d '{
"index.max_result_window" : "100000"
}'
設置size長度。