如何对 ElasticSearch 集群进行压力测试

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当 ElasticSearch 的业务量足够大，比如每天都会产生数百 GB 数据的时候，你就会自然而然的需要一个性能更强的 ElasticSearch 集群。特别是当你使用的场景是一些典型的大量数据进入的场景，比如网站日志、用户行为记录、大型电商网站的站内搜索时，一个强劲的 ElasticSearch 是必不可少的组件。在这样的场景下，如何找到一组合适的 ElasticSearch 集群？如何评估 ElasticSearch 集群的性能，就成为了一个十分重要的因素。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"用什么 ElasticSearch 进行压力测试？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于 ElasticSearch 性能测试和压力测试方面，其实有很多不同的方案，比如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://github.com/elastic/rally","title":""},"content":[{"type":"text","text":"Rally"}]},{"type":"text","text":"：Rally 是 Elastic 官方针对于 ElasticSearch 的宏观测试工具。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://github.com/danielfireman/esperf","title":""},"content":[{"type":"text","text":"ESPerf"}]},{"type":"text","text":"：一个基于 Golang 编写的 ElasticSerch 性能测试工具"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://github.com/logzio/elasticsearch-stress-test","title":""},"content":[{"type":"text","text":"Elasticsearch Stress Test"}]},{"type":"text","text":"：由著名的 ElasticSearch 服务提供商 Logzio 开发的性能测试工具"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了这些定制化的工具意外以外，ElasticSearch 也可以借由其 Restful API 来使用Load Runner、JMeter等老牌工具进行测试，这些工具零零散散，各有各的用法和用途，不过，对于广大开发者而言，还是官方出的 Rally 更令人满意。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本篇文章中，将会使用 Rally 来完成 ElasticSearch 集群的压力测试"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rally 作为官方出品的工具，来自官方的信任加成让他成为各家进行压力测试的首选工具。其次， Rally 官方给出了多种默认的数据集（Tracks）。如果你的使用场景覆盖在这些数据集(比如HTTP 访问事件数据、地理名称数据、地理座标点数据、HTTP 请求日志、问答场景、打车记录等场景)中，可以直接使用已有的数据集，来完成测试，而无需制造测试数据。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b6/b6da98cb64c896b41cb80317d030d161.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即使你的场景比较特殊，无法被官方的数据集所覆盖，也依然可以根据自己的线上数据，来创建数据集，确保测试效果的准确。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/da/dae82599ac18e034140dd15b67e830b3.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"如何使用 Rally 进行测试？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在了解了 Rally 后，来具体看一看 Rally 的使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"测试前的准备"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"关于 Rally 的基本安装，这里就不再介绍，总的来说十分简单，在配置好了 JDK 和 Python 环境以后，只需要执行"},{"type":"codeinline","content":[{"type":"text","text":"pip install rally"}]},{"type":"text","text":" 就可以完成安装。如果你的环境复杂，希望以一个更简单的方式来运行，你也可以选择使用 Docker 来运行 Rally ，进行测试。关于更多的安装方式，你可以"},{"type":"link","attrs":{"href":"https://esrally.readthedocs.io/en/latest/install.html#","title":""},"content":[{"type":"text","text":"参考 Rally 的安装文档"}]},{"type":"text","text":"."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在安装完成了 Rally 后，就可以开始进行 ElasticSearch 的测试。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在测试前，你需要先了解一些基本概念"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"race"},{"type":"text","text":"：在 Rally 中，每一次测试都可以称之为 race"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"car"},{"type":"text","text":"： 在 Rally 中，每一个参与测试的集群，都可以称之为 car ，不同的集群就是不同的 car，如果你是在选配置，则可以通过切换 car 来设置不同配置的测试。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"track: 在 Rally 中，每一次测试用的数据，都可以称之为 Track，不同的 Track 意味着不同的测试数据。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"challange： 在 Rally 中，每一个 challange 意味着一个不同的测试场景，具体的场景则代表着 ElasticSearch 所执行的操作。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在了解测试的基本概念后，就可以开始进行压力测试。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"进行压力测试"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"测试设备"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于本次测试实际上是一次选型的过程，在测试之前，我曾阅读了 Elastic 官方的权威指南中的"},{"type":"link","attrs":{"href":"https://www.elastic.co/guide/cn/elasticsearch/guide/current/hardware.html","title":""},"content":[{"type":"text","text":"硬件部分"}]},{"type":"text","text":".在其文档中提供了对于运转 ElasticSearch 集群设备的推荐配置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"64 GB 内存的机器是非常理想的， 但是32 GB 和16 GB 机器也是很常见的。少于8 GB 会适得其反"},{"type":"text","text":"（你最终需要很多很多的小机器），大于64 GB 的机器也会有问题， 我们将在 堆内存:大小和交换 中讨论。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"如果你要在更快的 CPUs 和更多的核心之间选择，选择更多的核心更好"},{"type":"text","text":"。多个内核提供的额外并发远胜过稍微快一点点的时钟频率。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"如果你负担得起 SSD，它将远远超出任何旋转介质（注：机械硬盘，磁带等）。 "},{"type":"text","marks":[{"type":"strong"}],"text":"基于 SSD 的节点，查询和索引性能都有提升"},{"type":"text","text":"。如果你负担得起，SSD 是一个好的选择。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"通常，选择中配或者高配机器更好。避免使用低配机器"},{"type":"text","text":"， 因为你不会希望去管理拥有上千个节点的集群，而且在这些低配机器上运行 Elasticsearch 的开销也是显著的。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此，我选择了原本打算购买的三家厂商（阿里云、腾讯云、UCloud）的设备进行测试，在具体的配置层面，则在各家购买四台 8C64G 100GBSSD磁盘的的云主机来进行测试。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/79/7933c661cbe0cce9c90bbda0fe6341af.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"测试环节"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"1. 构建集群"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在进行测试前，需要先对已有的三台设备搭建集群，并配置集群链接，确保集群正常工作。集群搭建的部分，你可以参考官方文档中的"},{"type":"link","attrs":{"href":"https://www.elastic.co/guide/en/elasticsearch/reference/current/add-elasticsearch-nodes.html","title":""},"content":[{"type":"text","text":"Add and remove nodes in your cluster"}]},{"type":"text","text":"部分。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"2. 执行测试命令"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在完成了集群的建设后，测试就简单很多，只需要执行命令，便可以对已经建设好的集群进行测试，比如:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"bash"},"content":[{"type":"text","text":"esrally --track=pmc --target-hosts=10.5.5.10:9200,10.5.5.11:9200,10.5.5.12:9200 --pipeline=benchmark-only"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"执行完命令后，接下来就是漫长的等待，根据机器的配置和集群等"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"3. 获得测试结果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在执行了测试命令后，测试完成后，你就可以获得相应的测试结果。不过，为了方便进行对比和查看，你可以在测试的命令中加入参数，从而实现将测试结果导出为特定的格式。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"比如,执行这样的测试命令，就可以获得 CSV 格式的测试数据"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"esrally --track=pmc --target-hosts=10.5.5.10:9200,10.5.5.11:9200,10.5.5.12:9200 --pipeline=benchmark-only -report-format=csv --report-file=~/benchmarks/result.csv"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当你得到了 csv 的结果后，就可以对数据进行分析和对比了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"如何查看 Rally 的测试结果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/38/3893e4838217dffbd18fcaf86d899ac2.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当我们对 Rally 执行测试以后，我们会拿到大量的数据，而具体这些数据应该如何来看呢？接下来我们一一来看。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何理解 Rally 的数据"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rally 导出的数据共有 4 列，分别是 "},{"type":"text","marks":[{"type":"strong"}],"text":"Metric（维度）"},{"type":"text","text":"、*"},{"type":"text","marks":[{"type":"italic"}],"text":"Task（任务）"},{"type":"text","text":"*、"},{"type":"text","marks":[{"type":"strong"}],"text":"Unit（单位）"},{"type":"text","text":"和*"},{"type":"text","marks":[{"type":"italic"}],"text":"‌Result（结果）"},{"type":"text","text":"*。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们可以将数据按照 Task 进行切分，你可以得到若干组结果，如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"没有 Task 的宏观数据"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"index-append 组数据"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"index-stats 组数据"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"node-stats 组数据"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"phrase 组数据"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"..."}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不同的组意味着 ElasticSearch 在不同场景下的应用，因此，你在对比时，需要按照分组来看数据。而分组内部的数据，就简单了许多，除了宏观数据以外，其他各组的数据基本上都是一个模式的，具体可以分为以下 14 组数据。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Min/Median/Max"},{"type":"text","text":"：本组测试的最小吞吐率、中位吞吐率和最大吞吐率，单位为 ops/s ,越大越好。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"50th/90th/99th/100th percentile latency"},{"type":"text","text":": 提交请求和收到完整回复之间的时间段，越小越好"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"50th/90th/99th/99.9th/100th percentile service time"},{"type":"text","text":"：请求处理开始和接收完整响应之间的时间段，越小越好"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"error rate"},{"type":"text","text":"：错误率，错误响应相对于响应总数的比例。任何被 Elasticsearch Python 客户端抛出的异常都被认为是错误响应(例如，HTTP 响应码 4xx、5xx或者网络错误，如网络不可达)。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当你能够理解数据的划分方式后，对于 Rally 返回的众多数据就比较好理解了。接下来，我们以实际例子来看 Rally 返回的数据。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何理解 Rally 返回的宏观测试数据"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这里以我测试的数据为例。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ce/ced3d4c30384189c13f4acf8a887642d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"根据每组数据对应的不同操作，我们可以将其数据分为若干组，这里我将数据按照颜色进行了一个基础的划分，从上到下依次为："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"索引时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"索引节流时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合并时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合并节流时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"刷新时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重刷时间"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，先看第一组的索引时间，索引时间共有四个指标:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing time of primary shards: 主分片累计索引时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing time across primary shards：跨分片累计索引时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing throttle time of primary shards：主分片累计节流索引时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative indexing throttle time across primary shards：跨分片累计节流索引时间"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这四个指标说明了 ElasticSearch 在进行数据处理所需要的索引时间，因此，时间越短越好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/17/17ca2914b5c3df500e67c061ba2916e8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/72/720a65b66c6b24fa5de9a5dc229c524f.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本组测试结果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"腾讯云的计算时间耗费最长，阿里云可以在腾讯的基础之上，有约 16 % 的性能提升，UCloud 则在腾讯云的基础之上提升了 24%"},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在测试过程中，会因为机器的性能等因素，而造成实际测试时间的不同。在这三组中，阿里云是最神奇的一个，因为它测试时间达到了6个小时。而 UCloud 和腾讯云则分别在 3 小时和 4 小时左右，实际的测试体验还是有很大的差距。如果你要复现相应的实验，建议你在早上进行，这样出测试结果的时候刚好还在白天。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下来看合并时间的数据"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cumulative merge throttle time of primary shards：主分片累计节流合并时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Min cumulative merge throttle time across primary shards：主分片累计节流合并时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Median cumulative merge throttle time across primary shards：主分片累计节流中位合并时间"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Max cumulative merge throttle time across primary shards：主分片累计节流最大合并时间"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合并时间组结果类似于索引时间组，不同的是测量的数据 Merge 时间。和 index 类似，时间越短越好，合并数量越大越好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/96/96ddccac0e3d015613520c0c9ee87f48.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e1/e1f075ddaa7e795b6a2fca3fe8a4479b.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本组测试结果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"腾讯云的计算时间耗费最长，阿里云可以在腾讯的基础之上，有约 9 % 的性能提升，UCloud 则在腾讯云的基础之上提升了 30%～40%"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其他几组类似的数据，这里就不再一一介绍，大家可以自行下载文章最后的数据进行分析，评估，得出结论。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何理解 Rally 返回的项目测试数据"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"除了宏观数据以外，Rally 返回数据中极大比例的是各种不同场景下的数据，这里，我们也选两组数据进行对比分析，看一看这些数据应该如何理解。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，我们看一下 node-stats 组的结果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"node-stats 组的结果是针对 node-stats 命令的数据分析结果。这里的吞吐量越大越好，时延则越小越好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c4/c4b4742f88d0cf23f3d9f2b7a5f4c1bf.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本组测试结果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"阿里云、腾讯云、UCloud 在性能方面没有较大的差距。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"类似的，我们可以对比 country"},{"type":"text","marks":[{"type":"italic"}],"text":"agg"},{"type":"text","text":"uncached 组的结果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/04/04a135eb7ab56240346fc52784a2b5ac.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本组测试结果中，"},{"type":"text","marks":[{"type":"strong"}],"text":"阿里云、腾讯云、UCloud 在吞吐量方面没有较大的差距，时延方面 UCloud 会更好一点。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"关于更多的数据，我们不在这里一一介绍，我将测试数据已经附在了文章的最后，你可以通过下载数据，自行分析，来练习 Rally 的使用和数据分析。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一些使用 Rally 时的小技巧"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"如何解决 Rally 网络不好的问题？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Rally 在执行时，需要下载相应的数据进行真实场景下的模拟，在这种情况下，Rally 需要从 AWS S3 上下载一些数据包，用于本地的测试。但国内的设备对于 S3 的访问不算太友好，经常会在下载的时候出现问题，因此，你可以选择"},{"type":"text","marks":[{"type":"strong"}],"text":"前置下载数据"},{"type":"text","text":"，这样在测试的时候可以直接使用本地的数据来完成测试，减少失败的可能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"想要前置下载数据也很简单只需要执行如下命令:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"bash"},"content":[{"type":"text","text":"curl -O https://raw.githubusercontent.com/elastic/rally-tracks/master/download.sh\nchmod u+x download.sh\n./download.sh geonames\ncd ~\ntar -xf rally-track-data-geonames.tar"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在执行 download.sh 时加入 track 名，就可以下载对应 track 数据的压缩包到本地，你可以根据自己的实际使用情况，来决定使用什么样的 track 模拟真实场景。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"总结"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在业务真正使用 ElasticSearch 之前，你可以像我一样，借助于 Rally 对备选的 ElasticSearch 集群进行压力测试，并通过对压测结果进行分析，从而获得明确的选型建议。比如，在我这次测试中，显然 UCloud 优于其他两家，是一个性价比 更高的选择。当然，这是一个实验环境下的压测，还是需要根据具体的业务场景做测试来进行选型，更为科学，不过希望压测方法能给予大家参考，也欢迎大家在后续实际的测试过程中给予我反馈。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"附录"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文涉及到的测试数据均可以在"},{"type":"link","attrs":{"href":"https://postimg.aliavv.com/mbp/3tf6y.zip","title":""},"content":[{"type":"text","text":"下载地址"}]},{"type":"text","text":"找到。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}