百億級實時計算系統性能優化–—Elasticsearch篇

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"導語"},{"type":"text","text":" | 隨着業務的發展，系統日益複雜，功能愈發強大，用戶數量級不斷增多，設備cpu、io、帶寬、成本逐漸增加，當發展到某個量級時，這些因素會導致系統變得臃腫不堪，服務質量難以保障，系統穩定性變差，耗費相當的人力成本和服務器資源。這就要求我們：要有勇氣和自信重構服務，提供更先進更優秀的系統。文章作者：劉敏，騰訊基礎架構研發工程師。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自今年三月份以來 "},{"type":"text","marks":[{"type":"strong"}],"text":"天機閣"},{"type":"text","text":" 用戶數快速上漲，業務總體接入數達到1000+，數據進入量更是迎來了爆發式上漲，日均處理量上漲了一個數量級：Trace數據峯值處理量達到340億\/日條，Log日誌數據峯值處理量級達到140億\/日條。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"面對海量數據，老的實時計算系統處理壓力逐漸增加，底層存儲系統無論在磁盤、IO、CPU、還是索引上都面臨巨大的壓力，計算集羣資源利用率不高，系統的調整優化迫在眉睫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/c0\/b8\/c0e16cc448b36fd4ffd2e13f0f1762b8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"業務接入量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/53\/f6\/538fec3d620b6c5fb39c0d029944e5f6.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"同比日均處理量級"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、什麼是天機閣"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在傳統單機系統的使用過程中，如果某個請求響應過慢或是出錯，開發人員可以通過查看日誌快速定位到具體服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}