Apache ShardingSphere：由開源驅動的分佈式數據庫中間件生態

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2021 年 7 月 21 日 2021 亞馬遜雲科技中國峯會現場，SphereEx 聯合創始人、Apache ShardingSphere PMC 潘娟受邀參與此次峯會，以《Apache ShardingSphere 分佈式數據庫中間件開源生態構建》爲主題，圍繞開源理念擴散、社區建設、ShardingSphere 如何踐行 Apache Way 等方面展開了介紹，本文總結自潘娟內容分享。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"01","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"在數據庫之上與業務之下的新生態","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"一層貼近應用，一層貼近 DataBase。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不同的行業、不同的用戶、不同的定位、不同的需求....如今的數據庫面臨着比過去更加複雜的數據應用場景以及愈發個性化和定製化的數據處理需求。愈發苛刻的生產環境，也在推動着不同的數據庫不斷將數據讀寫速度、延時、吞吐量等性能指標發揮到極致。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"久而久之，分工明確的數據應用場景逐漸導致了數據庫市場的碎片化，且難以出現一款能夠完美適配所有場景的數據庫。在不同的業務場景下選擇不同的數據庫，已經成爲一種常見的企業選型方法。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但同樣，這種百花齊放的數據庫形態，也會帶來『百花齊放』的問題。但從宏觀的角度來看，這些問題之間是存在共性的，是可以被抽離出來並形成一套事實標準的。如果能夠在這些百花齊放的數據庫之上構建能夠統一應用管理數據的平臺層，就可以在屏蔽底層數據庫差異的前提下，按照固定標準來進行開發，這種標準化解決方案將會極大縮減用戶管理基礎數據設施的壓力和學習成本。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Apache ShardingSphere 就是位於這一層，通過複用原有數據庫的能力，能夠幫助技術團隊在此之上實現如分片、加密解密等增量能力的開發，且向下不需考慮底層數據庫的配置，向上又能夠屏蔽用戶感知，從而快速構建起面向業務的數據庫直連能力，輕鬆管理大規模的數據集羣。","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/21/217947b70130c36190cd5bdee519adc7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"02","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"如何踐行 Apache Way","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"Sharding","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ShardingSphere 可同時疊加使用多個功能來滿足用戶的多樣化需求。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着業務體量的增大，單體數據庫難以支撐大體量業務時，就有必要對數據庫進行橫向擴展，這就必然要面臨着分佈式管理的問題。ShardingSphere 通過在數據庫之上構建一層熱插拔功能層，並提供傳統數據庫的操作模式，屏蔽使用者對底層數據庫變化的感知，賦予開發者使用單體數據庫的方式來管理大規模數據庫集羣的能力。其中，ShardingSphere 主要包含以下四種應用場景：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"Sharding 策略","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"業務體量增大時，所面臨的數據分片壓力就會隨之增加，所對應的分片策略相應就會被設計的更加複雜。ShardingSphere 能夠以靈活、易擴展的方式，以最低成本協助用戶在原本水平擴展之外做更多的分片策略，同時也支持自定義擴展的能力。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"讀寫分離","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通常情況下實現主從部署能夠有效緩解數據庫的壓力，但如果某一個集羣下的機器或庫表出現問題，無法進行正常讀寫操作，就會對業務造成比較大的影響。爲避免業務不可用，通常需要開發者重新寫一套高可用的策略來實現讀寫庫表的主從切換。ShardingSphere 可以自動探索所有集羣的狀態，在第一時間發現請求不可靠、底層數據庫發生主從切換等問題，並可以在表層用戶沒有產生感知的前提下自動恢復主從狀態。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"Sharding Scaling","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着業務的增長，可能會需要對此前拆分過的數據集羣進行再一次拆分。ShardingSphere 配套的 Scaling 組件，只需一條 SQL 命令就可以啓動任務，並在後臺實時展示運行狀態。通過 Scaling 這種『管道』，使舊的數據庫生態和新的數據庫生態重新連接起來。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/82/825e810b490eab08c46712afff66ef26.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"數據加解密","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在數據庫的應用中，對於關鍵數據的加解密也是非常重要的一部分。如果原有系統監控能力不達標，部分敏感數據可能是以明文的狀態存儲的，後期需要對其進行加密處理，這是許多團隊普遍存在的問題。ShardingSphere 通過對這部分能力進行標準化並集成在中間件生態上，自動化用戶對新、舊業務的數據脫敏以及加解密的過程，整個過程實現了用戶層面的無感知。同時支持多種內置的數據加解密/脫敏算法，用戶也可根據自身情況來自定義擴展相應的數據算法。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"構造數據的接入神經：可插拔的 Database Plus 平臺","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"面對各種各樣的需求以及使用場景，ShardingSphere 爲不同領域的開發者提供了面向 Java 的 JDBC、面向異構的代理端以及面向上雲的 Sidecar 端這三種接入形式，用戶可以按具體需求來做選型，在原有集羣之上來做分片、讀寫分離、數據遷移等相關操作。","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"JDBC 接入：完全以 JDBC 的方式去使用，可以理解爲一款增強的 JDBC 驅動程序，完全兼容 JDBC 和各種 ORM 框架，不需額外的部署和依賴即能夠實現分佈式管理、水平拓展、脫敏等一系列操作；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Proxy 接入：以模擬數據庫服務的形式，通過 Proxy 來管理底層真實的數據庫集羣，基本無需對業務進行改造；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雲上 mesh 接入：爲 ShardingSphere 提供公有云上的部署形式。在雲上，目前 SphereEx 已經加入了亞馬遜雲科技的雲創計劃，後續會在中國區和海外陸續在 Marketplace 與亞馬遜雲科技展開深度合作，爲亞馬遜雲科技上的用戶提供更加強大的 Proxy 鏡像部署能力，共同爲企業應用打造更加成熟的雲上環境。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/cf/cfd46bfd738c0c69eb3c157c2fe405f3.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"03","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"開源，讓個人工作連接到世界","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ShardingSphere 從開源至今，已經在業內產生了相當的影響力，目前國內只要涉及到水平擴展方面的工具或能力時，通常 ShardingSphere 都會出現在候選名單中。這一點當然有項目維護團隊成年累月的貢獻，使 ShardingSphere 的功能愈發完善，另一方面也歸功於國內日益向上的開源氛圍。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"過去幾年在開源社區上，國內用戶大多是扮演程序下載和代碼引用的角色，在社區建設方面卻少有涉及。最近幾年隨着開源理念在國內的推廣，開始湧現出越來越多抱有很強技術情懷的同學，正是有這些同學的加入，才能讓 ShardingSphere 的社區越來越活躍。因爲對於一個好的開源項目而言，評判標準並非只是其理念超前、技術先進等，更多是在技術影響力、開源影響力、生態建設、開發者羣體等多方面所積攢的深厚基礎。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a8/a89aaa50ae1d274f249374213abea4ee.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這也是爲什麼 ShardingSphere 作爲一款 Apache 頂級開源項目，依然在積極號召大家參與到開源社區中來。畢竟大家每天接觸到的只是身邊這羣人，所做的工作也只是辦公室裏的這些事，每天被『侷限』在這個圈子中。而通過開源，則可以讓自己的工作連接到世界，讓自己能夠拋開書本真正投入到項目中來，打開視野，逐漸培養開放、合作的精神，重新發現自己當下所產生的價值。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"來源： SphereEx 潘娟（亞馬遜雲科技中國峯會分享）","attrs":{}}]}]}