Elasticsearch8/7/6各版本特性 - MyOldTime的個人空間 - OSCHINA - 中文開源技術交流社區
| 版本 | 新特性 | 說明 |
| 8.1 | Doc-values-only search on numeric, date, keyword, ip, and boolean fields | numeric(數字類型), date, keyword, ip, and boolean 等字段，可以執行 term（=）和 range（範圍）的查詢，在該字段只作爲存儲（意思就是不開index，在mappings 中設置這些字段 index爲false）。查詢會變慢，但是，索引、存儲、聚合和排序的效率和速度會提升很多。 |
| 8 | 7.x REST API compatibility | 簡單來說就是ES8 有新得REST API ，但是做了對7.x版本 REST API 得兼容 |
| Security features are enabled and configured by default | 就是默認會開啓一些安全配置，如用戶認證、用戶授權、節點間通信TLS加密，與kibana間通信TLS加密。以前得版本默認是關閉的。 | |
| Better protection for system indices | 對系統索引的保護默認開啓，無法通過API 訪問系統索引。 | |
| New kNN search API | 新的k近鄰算法的支持 | |
| Storage savings for keyword, match_only_text, and text fields | 優化存儲結構，減少空間佔用。match_only_text 字段減少14.4% 。整體磁盤佔用減少3.5% | |
| Faster indexing of geo_point, geo_shape, and range fields | geo_point字段, geo_shape字段 和 範圍(不清是什麼字段)類字段，索引(存儲 入庫)速度快了 10%-15% | |
| PyTorch model support for natural language processing (NLP) | PyTorch 模型、和自然語言處理(NLP) | |
| SQL: Support for cross-cluster search | 支持通過sql跨集羣查詢 | |
| 7.17 | https://www.elastic.co/guide/en/elasticsearch/reference/7.17/release-highlights.html | |
| 7.16 | Search: Improved can-match phase for scalability | 如果搜索命中大量碎片，搜索操作將包括一個稱爲“can-match ”階段的預過濾階段。在此階段，Elasticsearch檢查受影響的碎片是否包含可能與搜索查詢匹配的數據。如果不是，Elasticsearch不會在碎片上運行查詢。 之前，搜索的協調節點向can-match 階段檢查的每個碎片發送了一個單獨的請求。然而，如果搜索需要檢查數千個碎片，那麼協調節點將需要處理數千個請求，從而導致高開銷。 在7.16中，協調節點在can-match 階段向每個數據節點發送一個請求。此請求覆蓋可以匹配檢查節點上所有受影響的碎片，從而顯著減少請求數量和相關開銷 |
| 7.15 | Index disk usage API | 一個新的API，可以支持，查看索引裏面每個字段 以及 索引本身的磁盤佔用率 |
| Search vector tile API | 爲地圖數據生產新的矢量瓦片數據。 | |
| Composite runtime fields | 可以組合多個運行時字段，同時支持 grok 和 dissect模式 | |
| 7.14 | Cross-cluster EQL search | EQL支持跨集羣查詢 |
| Async SQL search | 支持異步sql查詢，針對查詢大數據，可以異步返回結果。 | |
| Transforms: support for top metrics | 支持了一種新的聚合top_metrics，這可以提高按多個字段分組時的性能。以前只能通過script實現。 | |
| Anomaly detection: reset job API | 異常檢測 重置任務API | |
| New field type match_only_text | 新字段類型支持 match_only_text | |
| More memory-efficient composite aggregations | 複合聚合使用更少的內存。字段的複合聚合不在使用 global ordinals | |
| New migrate to data tiers routing API | 遷移數據層路由API | |
| New terms enum API | 新的針對於terms的api | |
| Automatic database updates for the GeoIP processor | 地理位置IP數據庫自動更新功能 | |
| 7.13 | Frozen tier is now GA | 凍結層和共享快照緩存正式可用了。索引生命週期ILM階分爲hot、warm、cold、frozen、delete。Frozen就是這次新加的一個階段。 |
| Index runtime fields | 運行時字段允許您在查詢時從其他字段和文檔屬性動態創建字段。這些查詢時運行時字段針對速度的靈活性進行了優化，允許您隨時更改它們。 | |
| Match IPv4 and IPv6 addresses against CIDR ranges in Painless | 可以通過CIDR API 直接對ip範圍匹配，範圍 掩碼 等。 | |
| New combined_fields query type | 一個新的查詢類型 combined_fields | |
| Faster terms aggregations | 在一定條件下的terms 聚合更快了。哪些條件可以看官方文檔。 | |
| Data frame analytics and inference are generally available | 在7.13中，可以訓練異常值檢測、迴歸和分類模型，然後使用這些模型根據輸入數據進行推斷。 | |
| Trained model aliases | 引入訓練模型的別名 | |
| 7.12 | Frozen tier and shared snapshot caches | 凍結層和共享快照緩存的實驗性功能。7.13正式可用。 |
| Analyze snapshot repositories | 新增了存儲分析的API | |
| EQL: Case-insensitive in lookups and functionse | EQL支持不區分大小寫的查詢和函數 | |
| EQL: like and regex keywords | EQL引入like 和 regex 關鍵字 | |
| Retention policy for transforms | ||
| Hyperparameter importance | 大概就是數據訓練過程中有一些重點的指標，我們可以指定這些指標 來加速訓練速度 | |
| Search-time runtime fields support for transforms | 大概就是Transforms過程中支持使用runtime_mappings來查詢字段 | |
| 7.11 | Runtime fields | 支持runtime_mappings |
| Speed improvements to the date histogram | 在histogram是最外層聚合 且沒有子聚合時，速度提升了85% | |
| Cross-cluster replication (CCR) now supports data streams | 數據流支持跨集羣複製索引 | |
| New audit record for security configuration changes via API | 爲安配置變化添加了日誌審計的API。 | |
| EQL: Wildcard and list lookup support for the : operator | 模糊查詢支持 : 的操作(可以在一組模糊查詢數據中查詢) | |
| New garbage collection defaults for small heaps | 爲小於8G的堆使用新的gc配置，提升性能。 | |
| Data frame analytics is now beta! | 數據訓練分析的測試版，7.13 正式可用 | |
| Latest document transform | ||
| 7.10 | Indexing speed improvement | 提升索引速度20%。對於全文搜索和其他分析密集型用例，性能提升較低。 |
| More space-efficient indices | 提升壓縮效率 減少存儲空間0-10%。可配置。 | |
| Data tiers | 引入數據層(分hot、warm、cold、frozen 7.13、delete)，可用通過ILM 管理這些層。 | |
| AUC ROC evaluation metrics for classification analysis | ||
| Custom feature processors in data frame analytics | ||
| Points in time (PITs) for search | 引入PITs的查詢方式，感覺和異步查詢類似。 | |
| Request-level circuit breakers on coordinating nodes | ||
| EQL: Case-sensitivity and the : operator | EQL 添加區分大小寫，和 : 操作符 | |
| REST API access to system indices is deprecated | 啓用通過API 訪問 系統索引，和 es8 那個對應上了 | |
| New thread pools for system indices | 線程池 改造，分爲 system_read和system_write 。把讀寫操作分在不同的線程池中。 | |
| 7.9 | Fixed retries for cross-cluster replication | 修復跨集羣複製中重試的問題 |
| Fixed index throttling | 索引限速 | |
| EQL | 一直新的查詢語法支持 | |
| Data streams | 它是索引、模板、rollover、ilm 基於時序性數據的綜合產物。更加方便的去管理實時數據流。大概就是分索引，生命週期，模板，滾動 幾個的統一管理。 | |
| Enable fully concurrent snapshot operations | 快照操作現在可以以完全併發的方式執行。 | |
| Improve speed and memory usage of multi-bucket aggregations | 提高多桶聚合的速度和內存使用率 | |
| Allow index filtering in field capabilities API | 容許field capabilities API中使用 index filtering 。field capabilities就是不指定或指定模糊的索引名，查某個字段的值是xx的結果， index filtering 可以過濾這個結果中查詢的索引範圍 | |
| Support terms and rare_terms aggregations in transforms | ||
| Optimize date_histograms across daylight savings time | 提升 date_histograms 性能 | |
| Improved resilience to network disruption | 增加了一個斷連機制，增強了對網絡中斷的恢復能力 | |
| Wildcard field optimised for wildcard queries | 優化通配符字段 | |
| Indexing metrics and back pressure | 新的API 記錄索引請求的 跟蹤 數據，配置可能會降低負載 | |
| Inference in pipeline aggregations | 好像和，模型訓練預測相關 | |
| 7.8 | Composable index templates | 組合模板，組件化各種配置，使模板的使用更加靈活 |
| Geo improvements | geo 的改進 | |
| Add support for t-test aggregations | 新增一種聚合 t-test | |
| Expose aggregation usage in feature usage API | 提供一個API 獲取從上次啓動到現在，索引 分片等使用次數的統計。 | |
| Support value_count and avg aggregations over histogram fields | histogram 支持，平均值和文檔數的統計 | |
| Reduce aggregation memory consumption | 減少聚合內存使用 | |
| Scalar functions now supported in SQL aggregations | sql聚合中支持 scalar 函數 | |
| Increase the performance and scalability of transforms with throttling | 提升Transforms 的性能 | |
| Better estimates for machine learning model memory usage | 更好地估計機器學習模型的內存使用 | |
| Additional loss functions for regression | ||
| Extended upload limit and explanations for Data Visualizer | 擴展 Data Visualizer 上傳大小的限制 爲1GB | |
| Fixed out-of-memory error when using cross-cluster replication with large documents | 修復大量文本跨集羣複製產生的內存溢出 | |
| 7.7 | Fixed index corruption on shrunk indices | |
| Significant reduction of heap usage of segments | 降低了 打開Lucene segments所需的內存 | |
| Transforms – now in GA! | Transforms 的正式支持。這個玩意感覺類似 insert into select from 。就是通過查詢 轉換，把結果存到另一個索引中。 | |
| Introducing multiclass classification | 預覽版 分類 機器學習 | |
| Feature importance at inference time | 特徵重要性現在可以在推理時計算。 | |
| Finer memory control for bucket aggregations | 對桶聚合內存精細控制 | |
| A new way of searching: asynchronously | 添加一種異步查詢的方式 | |
| Password protection for the keystore | ||
| A new aggregation: top_metrics | 新的聚合 top_metrics | |
| Query speed-up for sorted queries on time-based indices | 加快，基於時間索引的排序查詢 | |
| A new aggregation: boxplot | 新的聚合 boxplot | |
| AArch64 support | 支持AArch64 | |
| 7.6 | New histogram field type | |
| Optimized sorting on long field types | 優化long類型字段的排序 | |
| Simplifying and operationalizing machine learning | ||
| Cross-cluster search in transforms | transforms 可以跨集羣查詢 | |
| 7.5 | Enrich processor | |
| Shape support in SQL | ||
| Snapshot lifecycle management retention | ||
| Pause cross-cluster replication | 添加對索引複製 的暫定 恢復 API | |
| Machine learning classification analysis | ||
| 7.4 | Results pinning | 提升文檔評分排名，置頂查詢結果 |
| New shape field type | 新的字段shape | |
| Aggregations on range fields | 直方圖和日期直方圖聚合現在支持範圍字段類型。 | |
| Cumulative cardinality aggregation | 新的聚合類型 | |
| Snapshot lifecycle management | We’re introducing snapshot lifecycle management (SLM), which allows an administrator to define policies, via API or Kibana UI, that manage when and how often snapshots are taken. You can use SLM to ensure that appropriate, recent backups are ready if disaster strikes or you need to restore Elasticsearch data. | |
| API key management | ||
| TLS settings for email notifications | ||
| Automatic query cancellation | 自動關閉 查詢，當請求終止時 | |
| Support for AdoptOpenJDK | 自帶綁定AdoptOpenJDK 13 | |
| Regression analysis - Experimental | 機器學習 迴歸分析 實驗性 | |
| New vector distance functions for document script scoring - Experimental | ||
| 7.3 | Voting-only master nodes | 新增僅投票的節點 |
| Reloading of search-time synonyms | ||
| New flattened field type | ||
| Functions on vector fields | ||
| Prefix and wildcard support for intervals | intervals 支持模糊匹配 | |
| Rare terms aggregation | 新的聚合類型 | |
| Aliases are replicated via cross-cluster replication | ||
| SQL supports frozen indices | ||
| Fixed memory leak when using templates in document-level security | ||
| More memory-efficient aggregations on keyword fields | Terms aggregations 聚合 keyword fields 使用更少的內存 | |
| Data frames: transform and pivot your streaming data | beta，有關數據預測的 | |
| Discover your most unusual data using outlier detection | ||
| 7.2 | Data frames | 就是那個7.7 的insert into select from 的測試版 |
| Closed indices are now replicated | 容許關閉的索引 複製 | |
| Geo features in SQL | ||
| OpenId Connect authentication realme | 授權相關的 | |
| Search as you type field mapping type | 新增 search_as_you_type 字段類型 | |
| Distance Feature Query | distance_feature 查詢針對時間 距離 ，根據給定的源信息，以距離爲評分點 來對查詢結果評分。 | |
| 7.1 | TLS is now licensed under the Elastic Basic license | |
| RBAC is now licensed under the Elastic Basic license | ||
| 7.0 | Adaptive replica selection enabled by defaul | 對於查詢節點的自適應，查詢時會將請求分給不同節點，以前是循環的方式。現在會根據節點的負載，自動將請求分給負載較少的節點 |
| Skip shard refreshes if a shard is "search idle" | 優化 查詢 和索引刷新 | |
| Default to one shard | 分片數默認從5調整爲1 | |
| Lucene 8 | Lucene 8 | |
| Introduce the ability to minimize round-trips in cross-cluster search | 在跨羣集搜索中引入最小化往返的功能，跨集羣搜索更快了 | |
| New cluster coordination implementation | 集羣協調，新的方案 | |
| Better support for small heaps (the real-memory circuit breaker) | ||
| Cross-cluster replication is production-ready | 跨集羣複製 | |
| Index lifecycle management is production-ready | LIM 正式可用 | |
| SQL is production-ready | sql 正式可用 | |
| High-level REST client is feature-complete | High-level REST client | |
| Support nanosecond timestamps | 支持納秒級時間戳 | |
| Faster retrieval of top hits | ||
| Support for TLS 1.3 | ||
| Bundle JDK in Elasticsearch distribution | 從7.0開始將自帶jdk 並默認綁定自帶jdk。目的是爲了連 es是一個java項目都不知道的人 更方便部署。 | |
| Rank features | ||
| JSON logging | ||
| Script score query (aka function score 2.0) | ||
| https://www.elastic.co/guide/en/elasticsearch/reference/6.8/release-highlights.html | ||
| 6.8.13 | Fixed retries for cross-cluster replication | |
| 6.8.11 | Fixed out-of-memory error when using cross-cluster replication with large documents | |
| 6.8.0 | TLS is now licensed under the Elastic Basic license | |
| RBAC is now licensed under the Elastic Basic license | ||
| 6.7.0 | Cross-cluster replication | |
| Index lifecycle management | ||
| Elasticsearch SQL | ||
| 6.6.0 | Index lifecycle management (Beta) | |
| Frozen indices | ||
| BKD-backed geoshapes | ||
| 6.5.0 | Audit security events in new structured logs | |
| Discover the structure of text files | ||
| Improved machine learning results for partitioned multi-metric jobs | ||
| Find multi-bucket anomalies in machine learning jobs | ||
| Create source-only snapshots | ||
| Apply token filters conditionally | ||
| Use ODBC to connect to Elasticsearch SQL | ||
| Delegate authorization to other realms | ||
| Cross-cluster replication (beta*) | ||
| Monitor Elasticsearch with Metricbeat (beta*) | 可以使用 Metricbeat 收集數據 | |
| 6.4.0 | Analysis | 在文本字段中索引短語的選項-文本字段中添加了一個新的索引短語選項。添加對韓語支持。添加多路複用令牌過濾器 |
| Mappings | 字段可以取別名，添加_ignored meta field | |
| Rank Eval API | ||
| Search | ||
| 6.3.0 | License management and X-Pack code | 默認自帶X-Pack 且免費 |
| SQL | ||
| Rollups | ||
| Java 10 Support | ||