做好分库分表其实很难之二

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"为什么分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在正式开始之前，菜菜还是要强调一点，你的数据表是否应该分，需要综合考虑很多因素，比如业务的数据量是否到达了必须要切分的数量级，是否可以有其他方案来解决当前问题？我不止一次的见过，有的leader在不考虑综合情况下，盲目的进行表拆分业务，导致的情况就是大家不停的加班，连续几周996，难道leader你不掉头发吗？还有的架构师在一个小小业务初期就进行表拆分，大家为了配合你也是马不停蹄的加班赶进度，上线之后反而发现业务数据量很小，但是代码上却被分表策略牵制了太多。拆表引起的问题在特定的场景下，有时候代价真的很大。 数据库表的拆分解决的问题主要是存储和性能问题，mysql在单表数据量达到一定量级后，性能会急剧下降，相比较于sqlserver和Oracle这些收费DB来说，mysql在某些方面还是处于弱势，但是表的拆分这个策略却适用于几乎所有的关系型数据库。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"数据库进行表拆分不要太盲目"}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"link","attrs":{"href":"#分表策略","title":null}},{"type":"text","text":"分表策略"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"表的拆分和数据库的拆分有相似之处，但是拆分的规则也有不同。以下的拆分规则针对的是拆分一个表。"}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"link","attrs":{"href":"#横向切分","title":null}},{"type":"text","text":"横向切分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"横向切分是诸多业务中最常用的切分方式，本质是把一个表中的数据行按照规则分散到多个表中，比如最常见的按照ID范围，按照业务主键的哈希值等。至于表数据到达什么数量级之后进行切分，这和表中存的数据格式有关，比如一个表只有几列的int字段肯定要比几列text类型的表存储的极限要高。姑且认为这个极限是1000万吧。但是作为一个系统的负责人或者架构师来说，当表的数据量级到达千万级别要引起重视，因为这是一个系统性能瓶颈的隐患。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相对于数据表的横向切分，在符合业务优化的场景下我更倾向于做表分区，按照规则把不同的分区分配到不同的物理磁盘，这样的话，业务里的sql语句几乎可以不用改动。我司的一个sqlserver数据库，某个业务的表做了表分区之后，已经到达几十亿级别的数据量，但是查询和插入速度还是能满足业务的需求（优化一个系统还是要花精力优化业务层面）。"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/87/8774dfa017c82b408e7607618d97b796.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"link","attrs":{"href":"#垂直切分","title":null}},{"type":"text","text":"垂直切分"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"说到垂直拆分，表也可以按照业务来拆分，比如一个数据库中有用户的信息，根据业务可以划分为基础信息和扩展信息，如果对业务有利，完全可以拆分为基础信息表和扩展信息表。当然也可以按照别的规则来拆，比如把访问频繁的信息拆分成一个表，其他不频繁的信息拆分成一个表，具体的拆分规则还是要看当时要解决的问题是什么。垂直拆分可能会引入一定复杂性，比如原来查询一个用户的基础信息和扩展信息可以一次性查询出结果，分表之后需要进行Join操作或者查询两次才能查询出结果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e7/e7d5992499c5b55ddd9f441dc90c5728.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"link","attrs":{"href":"#分表代价","title":null}},{"type":"text","text":"分表代价"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"数据表垂直切分之后，原来一次查询有可能会变为连表的join查询，在一定程度上会有性能损失。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"数据表横向切分需要一定的规则，常用的主要有两种规则：范围切分和哈希值切分。范围切分是指按照某个字段的范围来切分，比如用户表按照用户ID来切分，id为1到10万的位于User表1中，100001到200000万的位于User2中，这样切分的优势是，可以无限的扩容下去，不用考虑数据迁移的问题，劣势就是新表和旧表数据分布不均匀，而且分表的范围选取有一定难度，范围太小会导致表太多，太大会导致问题根本上没有解决的困惑。另外一种分表策略就是把某一列按照哈希值来路由到不同的表中，同样以用户ID为例，假如我们一开始就规划了10个数据库表，路由算法可以简单地用 user_id %10的值来表示数据所属的数据库表编号，ID为985的用户放到编号为 5的子表中，ID为10086的用户放到编号为 6 的字表中。这种切分规则的优势是每个表的数据分布比较均匀，但是后期扩容会设计到部分数据的迁移工作。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"表拆分之后如果遇到有order by 的操作，数据库就无能为力了，只能由业务代码或者数据库中间件来完成了。"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"当有搜索的业务需求的时候，sql语句只能是Join多个表来进行连表查询了，类似的还有统计的需求，例如count的统计操作。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"你在业务中进行过表拆分吗？"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"更多精彩文章"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342955119549267969&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"分布式大并发系列"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342959003139227648&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"架构设计系列"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342962375443529728&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"趣学算法和数据结构系列"}]}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/mp/appmsgalbum?action=getalbum&album_id=1342964237798391808&__biz=MzIwNTc3OTAxOA==#wechat_redirect","title":null},"content":[{"type":"text","text":"设计模式系列"}]}]}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f8/f8af5984765a267892bf1a1272272625.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}