11天里13个Apache开源项目宣布退休，Hadoop的时代结束了

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在11天之内，Apache宣布退役了13个与大数据相关的Apache项目，其中包括Sentry、Tajo和Falcon。看起来Hadoop和大数据的黄金年代已经正式结束。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"曾几何时，Apache Hadoop是大数据的代表，但今天谁都知道它已经过时了。而自4月1日起，Apache软件基金会（ASF）宣布将至少19个开源项目撤回到他们的“Attic”，其中13个与大数据相关，10个属于Hadoop生态系统。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"光荣榜"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"单个项目的退役公告可能不算什么，但它们加在一起足以成为一个分水岭。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了帮助从业者和行业观察者充分认识到这次大数据开源项目洗牌的深刻影响，我们应该好好整理一下。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"与大数据相关的Apache退役项目包括："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Apex：基于HadoopYARN的统一大数据流和批处理平台"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Chukwa：基于Hadoop分布式文件系统（HDFS）构建的，用于监视大型分布式系统的数据收集系统"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Crunch，提供了用于编写、测试和运行MapReduce（包括HadoopMapReduce）管道的框架"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Eagle：一种分析解决方案，可在包括Hadoop在内的大数据平台上迅速识别安全和性能问题"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Falcon：针对Hadoop的数据处理和管理解决方案，设计用于数据移动、数据管道协调、生命周期管理和数据发现"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Hama：一种用于大数据分析的框架，运行在Hadoop上，并且基于BulkSynchronousParallel范式"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Lens，提供了统一的分析界面，将Hadoop与传统数据仓库深度集成在一起"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Marmotta：链接数据的开放平台"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Metron：专注于实时大数据安全性"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PredictionIO：一种用于管理和部署生产就绪的预测服务的机器学习服务器"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Sentry：一种用于对ApacheHadoop中的数据和元数据执行细粒度授权的系统"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Tajo：Hadoop上的大数据仓库系统"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Twill，它使用HadoopYARN的分布式功能和类似的编程模型来运行线程"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"房间里的“大象”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面这个列表很长，而完整的列表还包括一些非大数据项目。显然，ASF正在做一些内部清扫工作。此外，由于Cloudera-Hortonworks的合并，与Ranger和Spot项目竞争的Sentry和Metron也被弃用。之前两家公司总共支持四个项目，现在只保留两个就够了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这次合并的背景是大数据市场的整合趋势。而且可以说，这场大数据整合潮流也是上面这些项目“退役”的根本原因。至少可以说，在不到两周的时间内宣布所有这些项目“退役”的确是一件大事。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"官方评论"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我向ASF询问了他们清理大数据项目的解释。ASF市场营销与宣传副总裁Sally Khudairi通过电子邮件回复说：“Apache项目活动在生命周期中往往会起伏不定，这取决于社区的参与。”Khudairi补充说：“我们……从项目管理委员会（PMC）到董事会内部，对多个Apache项目的活动进行了审查和评估，并投票决定将这些项目退回到Attic。”Khudairi还说，ASF的Apache Attic副总裁Hervé Boutemy“最近非常高效地完成了“春季大扫除”，妥善处理了在过去几个月中准备退役的十几个项目。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"尽管ASF断言这次大数据清理工作只是其他常规项目退役的一部分，但很明显，大数据领域的情况已经发生了变化。Hadoop在开源分析技术的主导地位已让给Spark，Hortonworks和老牌的Cloudera之间的相似项目无意义竞争也结束了，这些项目完成了达尔文自然选择过程。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"小心一点吧"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"很明显，在大数据世界中，大量投资于Apache Sentry的供应商和客户现在需要整理他们的损失并继续前进。残酷的现实带来的教训几乎适用于所有技术炒作周期：社区开始兴奋起来，开源技术激增，生态系统逐渐完善。但这些生态系统并不会永存，几乎任何新平台（无论是商业平台还是开源平台）都存在固有的风险。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用ASF的Khudairi的话来说：“每个项目背后的社区才是代码生命力的源泉（'代码不会自动编写出来'），因此社区改变项目步伐的情况并不少见。”换句话说，尖端技术令人兴奋，但早期采用者要小心：它也是很脆弱的。请多加注意，并妥善管理风险。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"原文链接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/www.zdnet.com\/article\/apache-software-foundation-retires-slew-of-hadoop-related-projects\/?fileGuid=RtTCVhcpvRcRdC9c","title":"","type":null},"content":[{"type":"text","text":"https:\/\/www.zdnet.com\/article\/apache-software-foundation-retires-slew-of-hadoop-related-projects\/"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}