四種知識圖譜數據庫讀寫性能基準測試

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"知識圖譜數據庫是"},{"type":"text","text":"NoSQL"},{"type":"text","text":"數據庫中增速最快的一個分支,它在大數據和人工智能領域的地位逐漸凸顯。但是目前主流的圖數據庫產品大都屬於海外產品,且售價極其高昂,爲了解各大主流圖數據庫的讀寫性能指標,特將國產的新興圖數據庫"},{"type":"text","text":"AbutionGraph"},{"type":"text","text":"("},{"type":"text","text":"AbutionGDB"},{"type":"text","text":")與"},{"type":"text","text":"Neo4j"},{"type":"text","text":","},{"type":"text","text":"JanusGraph"},{"type":"text","text":","},{"type":"text","text":"TigerGraph"},{"type":"text","text":"等佔據着市場"},{"type":"text","text":"95%"},{"type":"text","text":"份額的主流圖數據庫做了讀寫性能對比測試。特別說明:"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"是唯一面向"},{"type":"text","text":"OLAP"},{"type":"text","text":"(數據分析決策)場景的圖數據倉庫,而其它對比者是面向"},{"type":"text","text":"OLTP"},{"type":"text","text":"(數據增刪改查)的圖數據庫,不過這並不影響讀寫性能的測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"1 測試用數據說明"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/d5\/d4\/d5a0fc61bb13d7fdeb7ba6aa823cc6d4.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖"},{"type":"text","text":"1"},{"type":"text","text":"數據示例"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖中的數據時間戳在測試中進行了調整,進入到系統中的時間戳沒有采用如圖所示的非嚴格的"},{"type":"text","text":"RFC3339"},{"type":"text","text":"格式,而是使用與"},{"type":"text","text":"1970-01-0100:00:00(UTC)"},{"type":"text","text":"時間的差"},{"type":"text","text":"("},{"type":"text","text":"精確到毫秒"},{"type":"text","text":")"},{"type":"text","text":"的時間戳表示方式"},{"type":"text","text":"(8bytes"},{"type":"text","text":"的長整型"},{"type":"text","text":")"},{"type":"text","text":",進入到系統中的金額數據也沒有采用如圖所示的"},{"type":"text","text":"Double"},{"type":"text","text":"標準雙精度浮點數類型表示,而是採用"},{"type":"text","text":"Bigdicemal"},{"type":"text","text":"非基本類型來存儲交易金額,因爲交易金額很大時,與歷史記錄聚合很可能發生損失精度的情況,由於系統錯誤導致的財產損失這是絕對不允許的。爲便於測試,實際數據均由計算機隨機生成,實體數限制爲"},{"type":"text","text":"10w"},{"type":"text","text":"個並按照上述格式和數據庫語法格式寫入各數據庫。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲每款圖數據庫使用的語言和方式都不一樣,本次主要測評過程以圖譜建模的最終結構爲標準,使用的"},{"type":"text","text":"KnowledgeGraph"},{"type":"text","text":"建模如下:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/6c\/3d\/6c766136b3f6a9073e73dc16cdb4373d.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"2 統計結果說明"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於任何一個測試,每個操作過程重複運行"},{"type":"text","text":"5"},{"type":"text","text":"次,且最終的統計結果是"},{"type":"text","text":"5"},{"type":"text","text":"次測試結果的算數平均值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每次測試進行之前均重啓數據庫服務,避免緩存對於之前執行獲得的結果產生影響。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文中所說的延遲"},{"type":"text","text":"(latency)"},{"type":"text","text":"是指全部完成該操作所耗費的時間,例如針對多條記錄的查詢操作,其時間延遲是指完全將結果獲取到客戶端後,相對於請求發出時間之間的間隔。吞吐量"},{"type":"text","text":"(throughput)"},{"type":"text","text":"是單位時間內完整完成該操作的數量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"3 AbutionGDB與其他數據庫單節點對比測試"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3.1測試環境及步驟說明"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"所有數據庫的對比測試在同一臺"},{"type":"text","text":"8H16G"},{"type":"text","text":"的阿里雲服務器上進行,該服務器的詳細配置如下"},{"type":"text","text":":"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"架構:             x86_64"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU:            8"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU運行模式:    32-bit, 64-bit"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Size:            16 GB"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每個核的線程數:  2"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每個座的核數:     4"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"座:               1"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"廠商ID:          GenuineIntel"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPU系列:        6"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"型號:              85"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"型號名稱:          Intel(R)Xeon(R)[email protected]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CPUMHz:          2499.998"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"BogoMIPS:         4999.99"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"超管理器廠商:      KVM"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"虛擬化類型:        完全"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"LSB Version:        core-4.1-amd64:core-4.1-noarch"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Description"},{"type":"text","text":":"},{"type":"text","text":"   "},{"type":"text","text":"     "},{"type":"text","text":"CentOS Linux release 8.2.2004(Core)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本測試將"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"與"},{"type":"text","text":"Neo4j"},{"type":"text","text":","},{"type":"text","text":"JanusGraph"},{"type":"text","text":","},{"type":"text","text":"TigerGraph"},{"type":"text","text":"等圖數據庫進行了對比,所測試的"},{"type":"text","text":"Neo4j"},{"type":"text","text":"版本爲"},{"type":"text","text":"3.5.24"},{"type":"text","text":";"},{"type":"text","text":"JanusGraph"},{"type":"text","text":"的版本爲"},{"type":"text","text":"0.5.2"},{"type":"text","text":"(使用"},{"type":"text","text":"Hbase"},{"type":"text","text":"後端存儲);"},{"type":"text","text":"TigerGraph"},{"type":"text","text":"的版本爲"},{"type":"text","text":"3.0.0"},{"type":"text","text":",而"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"的測試版本爲"},{"type":"text","text":"1.3.0"},{"type":"text","text":",截止至"},{"type":"text","text":"2021"},{"type":"text","text":"年,皆爲"},{"type":"text","text":"jdk-1.8"},{"type":"text","text":"支持下的最新版本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在測試時,"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"、"},{"type":"text","text":"Janasgraph"},{"type":"text","text":"、"},{"type":"text","text":"Neo4j"},{"type":"text","text":"均使用"},{"type":"text","text":"JavaAPI"},{"type":"text","text":"並結合各自的查詢語言進行數據操作,由於"},{"type":"text","text":"TigerGraph"},{"type":"text","text":"只提供了"},{"type":"text","text":"HTTP"},{"type":"text","text":"接口,在測試中我們採用"},{"type":"text","text":"Java"},{"type":"text","text":"語言的"},{"type":"text","text":"Apache HTTP lib"},{"type":"text","text":"庫來編寫測試程序。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AbutionGDB"},{"type":"text","text":"提供更優異的異步接口,支持使用"},{"type":"text","text":"Flink"},{"type":"text","text":"作爲大規模實時數據源寫入數據,但"},{"type":"text","text":"Janasgraph"},{"type":"text","text":"、"},{"type":"text","text":"Neo4j"},{"type":"text","text":"、"},{"type":"text","text":"TigerGraph"},{"type":"text","text":"均不支持。只有"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"和"},{"type":"text","text":"TigerGraph"},{"type":"text","text":"可以使用"},{"type":"text","text":"Kafka"},{"type":"text","text":"作爲實時數據源寫入數據,爲了測試一致性及公平性,僅使用各自推薦的同步接口方式進行讀寫測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本次測試中,"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"、"},{"type":"text","text":"Janasgraph"},{"type":"text","text":"及"},{"type":"text","text":"TigerGraph"},{"type":"text","text":"都採用默認配置,"},{"type":"text","text":"Neo4j"},{"type":"text","text":"則只修改"},{"type":"text","text":"dbms.memory.heap.initial_size"},{"type":"text","text":"和"},{"type":"text","text":"dbms.memory.heap.max_size"},{"type":"text","text":",由默認的"},{"type":"text","text":"512m"},{"type":"text","text":"改到"},{"type":"text","text":"5g"},{"type":"text","text":",以最大程度發揮"},{"type":"text","text":"Neo4j"},{"type":"text","text":"性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲避免網絡延遲的影響,應用與數據庫均在同一臺服務器上運行。爲更進一步控制變量,本測試確保在對每個數據庫進行測試時只有該數據庫運行,因此數據庫所能佔用的"},{"type":"text","text":"CPU"},{"type":"text","text":"資源最高可達"},{"type":"text","text":"100%"},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本測試測試了各個數據庫在不同客戶端連接數同時寫入和讀取不同批量數據的表現,且每種情況均進行至少"},{"type":"text","text":"5"},{"type":"text","text":"次測試,最終結果爲"},{"type":"text","text":"5"},{"type":"text","text":"次測試結果的平均值。同時爲了避免測試與測試之間的干擾,每次測試結束後,都會刪除數據,重啓數據庫服務並且等待"},{"type":"text","text":"5"},{"type":"text","text":"分鐘左右。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3.2 寫入性能對比"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據庫的一個寫入請求可以包含一條或多條記錄,性能結果取每個客戶端每秒內同時寫入數據量的總和,並取多次平均值。總而言之,一次請求裏包含的記錄條數越多,寫入性能就會相應提升。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時,一個數據庫可以支持多個客戶端鏈接,鏈接數增加,其系統總的插入通吐量會相應增加。因此測試中,對於每一個數據庫,都會測試一個客戶端和多個客戶端連接的情況。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"中,可以定義任意多維度的指標列,其中動態聚合的存儲模型是其特有的,其中預計算模型可以大大提高查詢性能,而靜態歷史數據的存儲模型是與"},{"type":"text","text":"Janasgraph"},{"type":"text","text":"、"},{"type":"text","text":"Neo4j"},{"type":"text","text":"、"},{"type":"text","text":"TigerGraph"},{"type":"text","text":"一致的,爲了凸顯寫入事件的速率和查詢性能的公平性,我們定義每一條完整的原始交易數據入庫完成才作爲寫入一條數據,即一條事件數,"},{"type":"text","text":" "},{"type":"text","text":"而不是以每個實體或者每條關係作爲一條數據。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2.1 AbutionGDB批量實時寫入結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此測試中,基於"},{"type":"text","text":"AbutionGDB"},{"type":"text","text":"強大的吞吐量,我們專門把寫入事件數的區間拉大,按照每次請求包含"},{"type":"text","text":"1"},{"type":"text","text":","},{"type":"text","text":"100"},{"type":"text","text":","},{"type":"text","text":"1000"},{"type":"text","text":","},{"type":"text","text":"10000"},{"type":"text","text":","},{"type":"text","text":"100000"},{"type":"text","text":","},{"type":"text","text":"200000"},{"type":"text","text":"條事件記錄各進行了測試,同時也測試了不同客戶端連接數的數據。具體結果如下:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"embedcomp","attrs":{"type":"table","data":{"content":"

批次插入事件數

1client

2client

3client

4client

5client

6client

7client

1

1339

2628

3904

5177

6516

7843

9243

100

24599

50099

71167

87467

99567

112667

136267

1000

44343

84931

119931

145931

166907

195273

235861

10000

58878

95295

130416

160418

190975

199025

236070

100000

99040

129773

172025

196447

232315

267567

251729

200000

187402

229706

207413

233558

263547

264271

267015

"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/9e\/94\/9e9ebec126de5578a24fe31250abdb94.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因本測試將3條圖數據統一歸爲1條事件數統計(每條事件包含2個實體1條關係),按照數據庫寫入性能標準,以上測試結果應該均乘以3即爲真實寫入速率。具體結果如下:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"embedcomp","attrs":{"type":"table","data":{"content":"

批次插入事件數

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章