Hadoop Committer如何炼成？爱奇艺新晋核心贡献人给出了这份攻略！

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"近日，全球最大的开源基金会Apache基金会的大数据开源社区Hadoop公布了最新一批","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"Committer（核心贡献人）","attrs":{}},{"type":"text","text":"，爱奇艺大数据团队的朱琦同学接受Apache社区邀请，正式成为了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"Hadoop Committer","attrs":{}},{"type":"text","text":"的一员。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"朱琦从2018年开始为Apache 社区贡献，被merge（合并）过","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"超万行代码","attrs":{}},{"type":"text","text":"，这也是","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"爱奇艺首位Hadoop Committer","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3a/3a712e96d95e9a1535ea9a3648258ea8.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"来自Apache的邀请邮件","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"成为一名Hadoop Committer需要哪些品质？又有哪些责任？如何在开源工作和日常工作中寻找平衡？就这些问题，我们和朱琦聊了聊，深入了解了在爱奇艺，一名年轻的Hadoop Committer如何炼成。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"01 Apache Hadoop Committer如何选出？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于不太熟悉的同学，我们可能需要先解决几个问题。什么是Apache Hadoop？Committer又是怎样被选出来的？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"了解大数据领域的读者对Apache Hadoop肯定不会陌生，这款开源软件的分布式应用程序根据谷歌公司发表的MapReduce和Google论文自行实现而成，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"目前已经是全球最受欢迎的的大数据开源软件框架之一。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e2/e268ed72405f100247253986afce95fb.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Apache每个项目会安排","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"相关委员会（PMC）和若干Committer","attrs":{}},{"type":"text","text":"。选举方式和组织结构都比较明确。Committer的主要责任是维护项目持续发展，比如项目新功能的开发，提出新的功能作为发展趋势，除了可以直接提交代码，还可以一票否决任何一个Patch。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Hadoop作为全球","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"最“数据中心的操作系统”","attrs":{}},{"type":"text","text":"，对代码质量要求比较高。因此要成为一名Apache Hadoop Committer，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"你首先必须要对社区已有开源代码和结构设计有深刻理解。","attrs":{}},{"type":"text","text":"因为Hadoop涉及到不少分布式计算里相对复杂的概念，完全理解现有设计和代码已经是一件很不容易的事情了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，成为Committer还要在","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"至少一个大的feature方向上是主力贡献人","attrs":{}},{"type":"text","text":"，这一点主要是通过被项目合并的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"代码数量","attrs":{}},{"type":"text","text":"体现，也就是说，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"对merged代码数量上有一定硬性规定。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Hadoop作为大数据底层的支撑项目，一般改代码量不会很多，这就意味着，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"贡献者不仅要有过硬的代码技能和专业知识，还得真实解决过社区的项目问题","attrs":{}},{"type":"text","text":"，才能被PMC提名，获得入选机会。朱琦告诉我们，他在当选Committer之前，目前已经有超过万行代码被社区合并。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，全球共有","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"230多","attrs":{}},{"type":"text","text":"位Hadoop Committer，其中来自中国的Committer有","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"十余名","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"02 从非计算机专业，到Hadoop Committer","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"虽然还很年轻，但朱琦在开源社区的工作经验已经非常扎实了。朱琦","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"本科主修化学材料","attrs":{}},{"type":"text","text":"，因为选修了一门程序语言设计课程对软件行业产生了浓厚的兴趣，于是在研究生阶段直接转到了计算机相关专业。虽然专业不是大数据方向，但","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"他一直对大数据特别是Hadoop非常感兴趣。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"工作后朱琦开始接触Hadoop项目，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"加入爱奇艺后在智能平台部门承担与大数据相关的工作","attrs":{}},{"type":"text","text":"，也因为工作原因，对一些大项目的底层源码研究很多，日积月累积累了看代码的能力。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a9/a977c3165bd15fe2cc1fb6997376553e.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"朱琦生活照","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"（代码）的输入非常重要","attrs":{}},{"type":"text","text":"，要输出必须要有输入。作为开源社区贡献者的前提是有输入，得理解了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"原先代码的逻辑","attrs":{}},{"type":"text","text":"，才能输入开发改造新的模块，才能有积累。”朱琦表示，“接触开源社区最开始是作为Contributor做一些贡献，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"从发现小问题、流程化的东西开始研究","attrs":{}},{"type":"text","text":"。慢慢把流程搞懂之后，就可以开始深入代码了。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在积极参与开源社区的过程中，朱琦自己也收获了很多。从作为一个参与者可以了解一个顶级项目的发展触类旁通，到现在成为Committer，为社区的其他成员和社区发展解决了不少问题，也研究了很多代码。“","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"现在去其他项目也比较能了解代码逻辑，也可以为公司各个大数据项目做贡献。","attrs":{}},{"type":"text","text":"”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/be/beb2fc7a24ead5775f6681fe037dbcde.png","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在公司中，朱琦也发现，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"越来越多的同学","attrs":{}},{"type":"text","text":"开始参与到了开源项目中，一起深入了解各个项目。“这一方面也能","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"反馈给公司底层部门","attrs":{}},{"type":"text","text":"，比如大数据相关内容，维护能力就变强了，另一方面可以","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提升成本优化和性能优化","attrs":{}},{"type":"text","text":"，减少公司实在支出，提升用户体验，更有利于平台推广。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"03 爱奇艺持续建设开源文化，回馈社区","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这些年，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"爱奇艺一直非常重视开源文化的发展","attrs":{}},{"type":"text","text":"，成立了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"开源工作组","attrs":{}},{"type":"text","text":"，发布了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"规范的开源项目流程","attrs":{}},{"type":"text","text":"，致力于建立","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"规范、有序、安全","attrs":{}},{"type":"text","text":"的开源体系架构，鼓励项目内部、外部开源，增强技术交流，提高研发效率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爱奇艺自己的一些业务实践探索，也会积极回馈到开源社区中，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"解决了不少开源社区中的项目问题","attrs":{}},{"type":"text","text":"。以朱琦所在的大数据团队为例，爱奇艺大数据架构和开源社区始终保持同步良性发展。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“比如爱奇艺最近紧跟社区对Hadoop相关的架构进行升级，针对新的hadoop项目去做测试，这个过程中就会出现一些问题，爱奇艺自己出现的问题就会回馈社区，一方面可以回馈社区，另一方面也解决了社区问题。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基于Hadoop的开源内容，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"爱奇艺自己的业务改进已经有不少合入了社区","attrs":{}},{"type":"text","text":"，已经Merge了Hadoop相关的部分包括：CS调度器统一FS的兼容性，全局调度优化，GPU资源相关的优化，大规模集群事件驱动的优化，弹性伸缩队列，Proxy server优化，Router Based Federation问题修复和优化，等功能。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"和朱琦一样，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"爱奇艺中不少员工也都是开源社区的积极参与者。","attrs":{}},{"type":"text","text":"他们在开源社区中贡献代码、解答其他社区参与者遇到的问题，维持社区的正常、有序运转。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"朱琦对此深有感触，“Apache发展十几年，就是因为有不同的用户和开发者，良性维护生态，共同交流互动。大家一方面在这里交流经验，解决自己遇到的问题，另一方面也可以从别的公司或者开发者提出的相关问题中吸收经验，防范于未然。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"04 从入门到终极，开源进阶攻略！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"亲历了开源文化近些年在国内的发展，朱琦也颇为感慨。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“之前开源其实没有这么火热，大家对开源的理解就是借来用，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"不太愿意贡献和参与其中","attrs":{}},{"type":"text","text":"，但也存在不少问题。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"很多公司没有使用开源项目，是闭源的","attrs":{}},{"type":"text","text":"，相关研发人员离职，代码就有很多历史包袱。现在","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"开源已经是一个大趋势","attrs":{}},{"type":"text","text":"，对公司来说，现在国内外各个互联网公司，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"特别是爱奇艺内部工程师文化建设中","attrs":{}},{"type":"text","text":"，都越来越重视开源文化了，通过开源，不仅公司的项目也会越来越好，贡献者本身也可以从中获得很多。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最后，朱琦也对想要入门开源文化的同学，给出了一些从入门、进阶到终极的建议。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"1.入门","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"熟悉开源项目的贡献流程","attrs":{}},{"type":"text","text":"，各个项目都会有how to contribute的文档。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"小的问题","attrs":{}},{"type":"text","text":"着手开始贡献，例如代码中的拼写错误，项目文档的改进，小的bug的修复。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"2.进阶","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"能够对项目在","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"生产环境","attrs":{}},{"type":"text","text":"产生的问题，或者","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"社区用户遇到的问题","attrs":{}},{"type":"text","text":"进行修复，性能优化等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"3.终极","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于项目的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"重大feature","attrs":{}},{"type":"text","text":"的设计文档进行理解，甚至","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"参与设计文档","attrs":{}},{"type":"text","text":"，然后","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"根据项目架构进行添加子任务进行开发贡献。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“总体来说，开源社区贡献是一件需要","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"持续花费时间和精力","attrs":{}},{"type":"text","text":"的事情。从吃透一个模块开始，逐渐深入，多贡献，也多参与问题讨论和review代码，祝大家都能在开源社区走得更远更好。”","attrs":{}}]}]}