腾讯云潘安群：把企业级分布式数据库做到极致

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"数据库是旧市场，也是新市场，是一个既成熟又有新机遇的战场。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"举些例子："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第七次全国人口普查：750万终端、十亿级数据，实时入库、实时统计、实时分析，这是对系统极致弹性、大并发计算以及连续稳定的挑战。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"健康码：覆盖全国10亿人口以上，上线3个月即累计亮码200亿次，考验的是系统对极速流量突增场景的应对。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"平安银行，是行业首例银行核心系统数据库从传统集中式大型机下移至国产分布式架构的案例，打破数十年国外商业数据库的垄断。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这些场景，对传统集中式数据库提出了极大地挑战。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当传统技术架构逐渐难以满足产业互联网时代的需求，转型升级是必然的趋势选择。当前，云计算技术作为当前企业IT架构的不二之选，已走过探索实践阶段，迎来了多样化、规模化的发展时期。同时，随着国家对数据库等基础技术设施的自主可控要求越来越明确，国产化替换成为国家战略。在技术升级和国产替代双重驱动下，国产数据库获得了前所未有的历史新机遇。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"面对历史新机遇，基础技术创新要如何为产业带来新的能力驱动，在近日"},{"type":"link","attrs":{"href":"https:\/\/www.infoq.cn\/theme\/118","title":"xxx","type":null},"content":[{"type":"text","text":"2021腾讯数字生态大会"}]},{"type":"text","text":"上，腾讯云数据库技术负责人潘安群，围绕“将企业级分布式数据库做到极致”的主题，分享了腾讯云数据库TDSQL的一些工作和思考。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"云时代数据库技术趋势"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先随着云计算的发展，整个IT 基础技术发生了翻天覆地的变化："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"l IT设施从零散走向集中化、规模化。过去，每一个企业自建各自的数据中心等 IT 基础设施，包括服务器、网络到操作系统、数据库等，形成企业市场上零散化的 IT 设施模式。而今天基于云计算服务，企业 IT 设施呈现集中化、规模化效应，对效率、性能、成本的要求提升。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"交付方式从软件交付走向服务交付。过去购买商业化软件，或者是使用开源软件产品，基本是通过商业化或开源的方式进行分发，而现在完全变成一个个服务的形式进行交付。这带来的变化是，用户不需要再盘算该购买几台服务器，而是在具有数据库使用需求时，直接云上使用即可，企业IT人员不再需要自己去维护这些数据库软件。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"开发方式从底层（IaaS+PaaS）走向上层（SaaS）。即呈现从过去业务进行非常底层的开发以及调用底层 API 等操作的模式，转向 SaaS 化、Severless 模式的服务。在云上，开发者可以使用各种各样的 SaaS 服务。无论从效率、基础技术能力等方面来说，这都是一个巨大的变化。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"数据形式及应用场景从单一化走向多样化。过去传统行业，比如金融行业，最核心是记账、结算，业务形式比较简单，但现在金融行业所面临的是要做互联网化、数字化的全面升级，包括各行各业都在做数字化升级，数据形式已经不再是过去传统数据库那样简单的模式，而是数据形态、访问模式等会越来越多样化。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基于这样的变化，我们认为未来的数据库应该是：单引擎极致化，加多引擎自治融合的DBaaS服务。那么具体表现为："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"单一引擎极致化。垂直场景在云上的规模化，使得我们必须将单引擎做到极致，才能进一步放大规模效应。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多引擎统一智能融合。而业务形态的多样化，单一数据库包打天下的时代已经过去，多引擎化成为必然趋势。当然我们不希望让用户陷入选择，我们希望对客户来说，多即是一。如何做到多即是一，如何做到在超大规模数据库集群的一体化，智能化是决定这场变革成败的关键因素。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"DBaaS交付。基于多引擎智能融合，做到服务即交付，即交付给客户的是一套完整数据库服务，而不是一堆软件。这是我们在企业级市场的理念。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"TDSQL产品核心能力演进和突破"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在以上理念下，今年TDSQL产品多个引擎持续实现了相应的突破。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"产品演进的驱动力：企业级市场核心场景挑战"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“国产化+数据爆发+复杂度提升”是数据库行业底层驱动力。随着金融政企技术系统对国产化、数字化改造升级的需求加速，国产企业级分布式数据库单引擎极致能力主要受到金融级核心系统应用能力、快速平稳“去O”、新形互联网业务转型等需求和挑战驱动，实现技术突破。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，总结而言，数字化时代金融、政务、互联网、电商等企业对分布式数据库的应用需求体现在以下4个方面："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（1）支持稳态业务的核心应用能力需求持续增强：表现为需要数据库系统在高可用、强一致、稳定可靠的基础上，同时实现高性能、高可扩展，满足业务稳定、快速发展过程中对基础技术要求。与此同时，核心应用关系企业系统最核心的交易，改造起来时间长、投入大，对国产数据库的技术以及服务能力要求进一步提高。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（2）平稳快速“去O”：除了核心应用，金融政企系统中还有大量周边系统或者次核心应用，这些核心应用往往有几百上千套系统，由于历史原因需要短期之内快速“去O”，并基于国产分布式数据库快速上线，如第一个月上线10个系统、第二个月上线20个系统……快速完成“去O”的场景需求，要求国产数据库产品对Oracle兼容具备极高的兼容能力，并能做到高可用、强一致。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（3）自动弹性扩缩容、应对敏态业务需求加大：业务在进行数字化转型过程中，场景化、线上化是大势所趋，这意味着技术系统中的数据吞吐量、业务形态、数据访问和计算管控等也将随之发生巨大变化，变得更加敏捷、多样化，即面临更多的不可预知性，对底层技术系统在灵活性与扩展性方面提出全新的要求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（4）海量数据复杂分析场景需求加大：在数字化业务的融合升级中，数据处理的规模大幅增长，对处理的实时性和效率有更高要求，因此亟需实现对海量数据的复杂分析、实时计算处理能力。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"TDSQL产品核心能力演进和突破"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f3\/f3fd17534bac52ee9a7d1ba73ae2609c.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"针对上述场景需求和挑战，TDSQL从不同应用场景匹配不同引擎，为客户提供融合统一的产品服务，以及从前端管理到底层统一的智能管控体系。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（1）金融级核心交易OLTP引擎"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/73\/73af3406bf78bd723d9ded8b3fff4f71.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"金融级分布式OLTP引擎是TDSQL主打特性体现在强一致、高可用、分布式事务、兼容性等方面，可实现“两地三中心、多地多中心”高可用灵活部署架构的产品引擎，它在跨数据中心强同步复制机制下可实现“RTO<30S、RPO=0”，同时在高性能基础上保持全局一致性。本次引擎全新升级，将为用户带来全局索引、视图、执行计划绑定、计算下推等能力升级，实现系统性能提升50%。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"金融级OLTP引擎是当前大部分TDSQL银行核心系统用户所采用的引擎，具备产品成熟度高、分布式事务能力级别达到行业最高等特性。在互联网银行、区域性银行以及股份制银行、国有大行等领域都有成功案例。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（2）TDSQL 金融级HTAP引擎(Oracle兼容)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/d1\/d15c986a2bcf646042da2d8b5f4edb31.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本次大会，我们也面向行业全新发布TDSQL金融级HTAP引擎（Oracle兼容版）。这款引擎在保险、运营商等行业兼容度高达98%以上，同时在数据的一致性、高可用方面也完全满足金融行业的要求，可以帮助金融机构在极短时间内，几乎零成本改造的情况下，实现国产化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（3）TDSQL全自研新敏态引擎"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/db\/dbbb874339b31c6434b9a24ed113270d.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这也是本次TDSQL带来的全新重磅发布。TDSQL新敏态引擎100%兼容MySQL，计算\/存储资源均可独立全透明弹性扩缩容，实现了PB级存储的Online DDL；计算层每个节点均可读写，轻松支撑千万级QPS流量，可以有效应对业务的变化。针对海量数据存储的场景，实现最高最高20倍压缩率的超高压缩比存储能力，大幅节省资源成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"更关键的是，其独有的数据形态自动感知特性，可以实现数据库全域智能化自适应调度，能够根据业务负载情况自动迁移，打散热点，降低分布式事务比例，获得极致的扩展性和性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"与此同时，TDSQL 新敏态引擎具有对分布式事务完整支持的特性，支撑了上层计算引擎多主读写架构的实现，并与计算引擎结合实现了计算下推、分布式事务一阶段优化等多纬度优化，进一步实现分布式数据库系统性能极致提升，有效适配企业新敏态业务需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在腾讯内部业务实践中，TDSQL新敏态引擎可支撑业务在保持高性能且连续服务的基础上，一个月内完成了高达1000次表结构在线变更。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（4）TDSQL-A 分析型引擎"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/02\/0296d0f1b064da52481c62545d560392.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"今年5月份我们发布了分析引擎TDSQL-A，今天将迎来再次的全面升级。我们在架构上实现了计算与转发双平面的基础上，全新自研了列式存储引擎，搭配全新的智能执行引擎，向量化执行性能有10倍以上的提升，同时凭借业界独有的分布式延迟物化技术，大幅优化了分布式场景下关联查询的计算效率，可以帮助客户进一步挖掘数据价值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（5）智能统一管控"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/0b\/0bf6c74256b6b0edf7531444e9af6423.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这是腾讯云数据库智能统一管控平台，可让数据在不同引擎之间自由流动，更好地支持业务快速发展。具体包括："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以超过769个接口能力，支持系统实现不同应用场景灵活调用、一键运营；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"实现90%常见故障秒级诊断及SQL优化建说议的智能运维体系，大幅降低系统运维复杂度；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基于多源同步工具，实现多引擎数据秒级同步，对业务屏蔽引擎差异；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"实现插件式负载均衡管理，进一步提升可用性。目前支持F5、LVS、TGW、VPCGW等多种……"}]}]}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"TDSQL 应用场景突破"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"   "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，TDSQL已服务近半国内TOP20银行，TOP10银行中服务比例也高达60%。同时，TDSQL在不同金融机构核心系统中的渗透率明显提升，如金融核心系统客户超过20家，率先实现行业突破，包括平安银行、张家港农商行、昆山农商行等银行、保险、券商机构。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而在电信运营商等行业场景，腾讯云数据库TDSQL也实现了率先突破，如在2021年中标某省移动核心系统从传统集中式大型机系列下移至国产数据库项目，是行业首家运营商核心系统数据库分布式国产化改造项目。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"无论是在银行还是保险、政务等行业，TDSQL在关键领域的标杆头部客户已具有丰富的成功案例实践。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基于大量大规模银行核心系统应用实践，TDSQL一路走来经历了最严苛的考验。我们认为，国产数据库即将进入规模化的阶段，我们希望未来五年能够为1000家金融机构实现核心系统数据库国产化改造升级，也希望广大合作伙伴与我们一同实现这个国产数据库的长期事业。感谢！"}]}]}