大型互联网应用面对的挑战及应对方案和手段

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作业一："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一个典型的大型互联网应用系统使用了哪些技术方案和手段，主要解决什么问题？请列举描述。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1.大型互联网应用面对的挑战："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"性能挑战：高性能（"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#000000","name":"black"}}],"text":"高并发"},{"type":"text","text":"，高负载）；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可用性挑战：高可用（7*24）；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可靠性挑战：高可靠（数据一致性）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"存储挑战：海量数据存储；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"网络挑战：网络复杂性；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"业务挑战：快速业务发展"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"规模挑战：小应用发展为大应用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"核心挑战：性能挑战，要求"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#F5222D","name":"red"}}],"text":"高并发"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2.高并发方案-垂直伸缩和水平伸缩"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发，需要更多的计算能力和资源需求。为了提供更多计算能力和资源需求，解决方法有两种技术方向：垂直伸缩和水平伸缩。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.1垂直伸缩："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 不断提升单台服务器的性能，来提供更多的计算能力和资源需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 方法：1.使用更快的CPU（提供更多处理器和更多超线程）；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 2.使用更多的内存"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 3.使用RAID磁盘阵列，增加IO吞吐能力"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 4.使用更快的SSD固态硬盘，改善IO访问速度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 5.增加网络接口提高网络吞吐能力等"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 缺点：1.前期性能成本比（性价比）线性上升。升级部分：升级cpu,升级内存，升级磁盘，升级网络"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 2.中后期：过了某个点后，性能提升，导致计算机成本指数级上升。成本提升，收益降低。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 比如：普通服务器-->小型机-->中型机--->大型机--->超级计算机。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 评估：适用于短期伸缩性方案，作为过度手段。短暂目的：避免后期成本急剧上升。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.2水平伸缩："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 不断增加同类型服务器。来提供更多的计算能力和资源需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 目前互联网架构的主要手段：水平伸缩。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 水平伸缩的核心要点：不断增加服务器，增加局部需要的计算能力和资源需求，分摊增长的计算能力和资源需求，将当前增长的高并发，转移到增加的服务器上。以此破除局部瓶颈，提升局部的处理能力，进而增强整个系统架构，提升整体系统的处理能力，来应对业务的发展和高并发用户量的增加。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 优点：1.可以无限增加服务器，提供更多资源"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 2.成本可控，增长较少。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"3.高并发瓶颈及手段"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发主要手段：分而治之。 分治主要手段：集群。集群中每台服务器承担一部分计算和资源需求。集群作为整体对外提供服务。当高并发用户量增长时，可以添加服务器到集群中。以分摊增长的计算能力。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发增长的不同阶段，互联网应用会遇到不同的瓶颈。破除瓶颈，核心手段：建立集群。解决不同的瓶颈，需建立不同的集群。集群伸缩时，可增加移除服务器。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 前提假设：普通应用服务器可应对最高并发为500，数据库系统支持的最高并发为100。 应用，数据，文件部署再同一台服务器。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1瓶颈1：数据库瓶颈"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发用户量增长到500，"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题：1.应用500并发>数据库100并发；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 2.应用服务器速度>>数据库速度（数据库数据访问需要扫描磁盘，磁盘是低速设备）。 "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：访问速度不匹配。数据库服务器低速，满足不了应用服务器高速读写的需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决手段：分布式缓存集群。引入本地缓存和远程缓存。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 本地缓存：速度快，但容量小。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 远程缓存：组建分布式集群，使用更多的内存，解决本地缓存容量不足的问题。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2瓶颈2：应用服务器瓶颈"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发用户量增长到5000. "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题： 单台应用服务器无法满足5000并发。单台最高500，满足不了5000并发"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：单台有并发上限。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决手段：分布式应用服务器集群+负载均衡。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 建立应用服务器集群，5000并发，可分摊到10台服务器，每台分摊500并发。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 负载均衡服务器：负责均衡分发请求到10台服务器。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3瓶颈3：数据库读写瓶颈"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发用户量增长到5000. "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题：缓存穿透，部分读请求和全部写请求压力集中再数据库上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：读写操作争夺数据库服务器资源。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决方法：读写分离。主从分离。主数据库：承担全部写操作。从库：承担读操作"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.4瓶颈4：数据库写瓶颈"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发用户量增长到50000."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题：1.写瓶颈：大量写操作集中在主库，主库只有一个。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 2.海量数据：单一数据库服务器，可能存不下一张表。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：主数据CPU和存储资源不够用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决方法：分布式数据库集群。分布式数据库，将海量数据分摊在多个数据库服务器存储。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.5瓶颈5：负载均衡瓶颈"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发用户量增长到500000（50万}."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题：单一的负载均衡服务器，面对50W并发，响应速度慢，甚至超时。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：单一负载均衡服务器有性能上限。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决方法：CDN集群+反向代理集群+负载均衡集群。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" CDN集群：网络运营商机房的CDN服务器建立集群，主要缓存静态资源。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 拦截并发请求，处理静态资源，直接返回给用户。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 反向代理集群：反向代理整个数据中心，对外提供服务，可缓存部分内容。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 负载均衡集群：下发请求到应用服务器。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.6瓶颈6：查询速度瓶颈"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 高并发用户量增长到500000（50万}."}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题：查询速度慢，特殊场景，关系型数据不适合。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：海量数据，复杂查询，关系性数据库支持不够好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决手段：搜索引擎集群+NoSQL集群。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.7瓶颈7：业务耦合"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 业务庞杂，面对业务的快速变化，多个业务混合在一起，不能很快适应。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题：业务快速变化，系统不能很快上线新业务。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：业务应用耦合程度高。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决方法：业务拆分+消息队列。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 业务拆分：按照业务领域，拆分应用。并将不同应用部署到不同的集群上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 消息队列：解除不同应用之间的耦合。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.8瓶颈8：代码重复开发"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 业务拆分后，不同业务之间有相同或者相似依赖，每个应用各自开发一套"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 问题：重复开发，每个应用开发一套代码，开发维护成本高。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 核心点：代码重复"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决方法：微服务集群+中台化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 微服务集群：剥离公共服务，构建可重复使用，更小业务单元，部署到独立的微服务集群。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 中台化：由多个微服务集群组成中台。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.9瓶颈9：业务分析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" 解决方法：大数据。使用大数据分析业务数据，为业务决策，提供有力支持。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}