Hadoop Committer如何煉成？愛奇藝新晉核心貢獻人給出了這份攻略！

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"近日，全球最大的開源基金會Apache基金會的大數據開源社區Hadoop公佈了最新一批","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"Committer（核心貢獻人）","attrs":{}},{"type":"text","text":"，愛奇藝大數據團隊的朱琦同學接受Apache社區邀請，正式成爲了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"Hadoop Committer","attrs":{}},{"type":"text","text":"的一員。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"朱琦從2018年開始爲Apache 社區貢獻，被merge（合併）過","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"超萬行代碼","attrs":{}},{"type":"text","text":"，這也是","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"愛奇藝首位Hadoop Committer","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3a/3a712e96d95e9a1535ea9a3648258ea8.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"來自Apache的邀請郵件","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"成爲一名Hadoop Committer需要哪些品質？又有哪些責任？如何在開源工作和日常工作中尋找平衡？就這些問題，我們和朱琦聊了聊，深入瞭解了在愛奇藝，一名年輕的Hadoop Committer如何煉成。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"01 Apache Hadoop Committer如何選出？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於不太熟悉的同學，我們可能需要先解決幾個問題。什麼是Apache Hadoop？Committer又是怎樣被選出來的？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"瞭解大數據領域的讀者對Apache Hadoop肯定不會陌生，這款開源軟件的分佈式應用程序根據谷歌公司發表的MapReduce和Google論文自行實現而成，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"目前已經是全球最受歡迎的的大數據開源軟件框架之一。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e2/e268ed72405f100247253986afce95fb.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Apache每個項目會安排","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"相關委員會（PMC）和若干Committer","attrs":{}},{"type":"text","text":"。選舉方式和組織結構都比較明確。Committer的主要責任是維護項目持續發展，比如項目新功能的開發，提出新的功能作爲發展趨勢，除了可以直接提交代碼，還可以一票否決任何一個Patch。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Hadoop作爲全球","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"最“數據中心的操作系統”","attrs":{}},{"type":"text","text":"，對代碼質量要求比較高。因此要成爲一名Apache Hadoop Committer，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"你首先必須要對社區已有開源代碼和結構設計有深刻理解。","attrs":{}},{"type":"text","text":"因爲Hadoop涉及到不少分佈式計算裏相對複雜的概念，完全理解現有設計和代碼已經是一件很不容易的事情了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，成爲Committer還要在","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"至少一個大的feature方向上是主力貢獻人","attrs":{}},{"type":"text","text":"，這一點主要是通過被項目合併的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"代碼數量","attrs":{}},{"type":"text","text":"體現，也就是說，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"對merged代碼數量上有一定硬性規定。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Hadoop作爲大數據底層的支撐項目，一般改代碼量不會很多，這就意味着，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"貢獻者不僅要有過硬的代碼技能和專業知識，還得真實解決過社區的項目問題","attrs":{}},{"type":"text","text":"，才能被PMC提名，獲得入選機會。朱琦告訴我們，他在當選Committer之前，目前已經有超過萬行代碼被社區合併。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，全球共有","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"230多","attrs":{}},{"type":"text","text":"位Hadoop Committer，其中來自中國的Committer有","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"十餘名","attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"02 從非計算機專業，到Hadoop Committer","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雖然還很年輕，但朱琦在開源社區的工作經驗已經非常紮實了。朱琦","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"本科主修化學材料","attrs":{}},{"type":"text","text":"，因爲選修了一門程序語言設計課程對軟件行業產生了濃厚的興趣，於是在研究生階段直接轉到了計算機相關專業。雖然專業不是大數據方向，但","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"他一直對大數據特別是Hadoop非常感興趣。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"工作後朱琦開始接觸Hadoop項目，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"加入愛奇藝後在智能平臺部門承擔與大數據相關的工作","attrs":{}},{"type":"text","text":"，也因爲工作原因，對一些大項目的底層源碼研究很多，日積月累積累了看代碼的能力。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a9/a977c3165bd15fe2cc1fb6997376553e.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"朱琦生活照","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"（代碼）的輸入非常重要","attrs":{}},{"type":"text","text":"，要輸出必須要有輸入。作爲開源社區貢獻者的前提是有輸入，得理解了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"原先代碼的邏輯","attrs":{}},{"type":"text","text":"，才能輸入開發改造新的模塊，纔能有積累。”朱琦表示，“接觸開源社區最開始是作爲Contributor做一些貢獻，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"從發現小問題、流程化的東西開始研究","attrs":{}},{"type":"text","text":"。慢慢把流程搞懂之後，就可以開始深入代碼了。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在積極參與開源社區的過程中，朱琦自己也收穫了很多。從作爲一個參與者可以瞭解一個頂級項目的發展觸類旁通，到現在成爲Committer，爲社區的其他成員和社區發展解決了不少問題，也研究了很多代碼。“","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"現在去其他項目也比較能瞭解代碼邏輯，也可以爲公司各個大數據項目做貢獻。","attrs":{}},{"type":"text","text":"”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/be/beb2fc7a24ead5775f6681fe037dbcde.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在公司中，朱琦也發現，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"越來越多的同學","attrs":{}},{"type":"text","text":"開始參與到了開源項目中，一起深入瞭解各個項目。“這一方面也能","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"反饋給公司底層部門","attrs":{}},{"type":"text","text":"，比如大數據相關內容，維護能力就變強了，另一方面可以","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"提升成本優化和性能優化","attrs":{}},{"type":"text","text":"，減少公司實在支出，提升用戶體驗，更有利於平臺推廣。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"03 愛奇藝持續建設開源文化，回饋社區","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這些年，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"愛奇藝一直非常重視開源文化的發展","attrs":{}},{"type":"text","text":"，成立了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"開源工作組","attrs":{}},{"type":"text","text":"，發佈了","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"規範的開源項目流程","attrs":{}},{"type":"text","text":"，致力於建立","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"規範、有序、安全","attrs":{}},{"type":"text","text":"的開源體系架構，鼓勵項目內部、外部開源，增強技術交流，提高研發效率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"愛奇藝自己的一些業務實踐探索，也會積極回饋到開源社區中，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"解決了不少開源社區中的項目問題","attrs":{}},{"type":"text","text":"。以朱琦所在的大數據團隊爲例，愛奇藝大數據架構和開源社區始終保持同步良性發展。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“比如愛奇藝最近緊跟社區對Hadoop相關的架構進行升級，針對新的hadoop項目去做測試，這個過程中就會出現一些問題，愛奇藝自己出現的問題就會回饋社區，一方面可以回饋社區，另一方面也解決了社區問題。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於Hadoop的開源內容，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"愛奇藝自己的業務改進已經有不少合入了社區","attrs":{}},{"type":"text","text":"，已經Merge了Hadoop相關的部分包括：CS調度器統一FS的兼容性，全局調度優化，GPU資源相關的優化，大規模集羣事件驅動的優化，彈性伸縮隊列，Proxy server優化，Router Based Federation問題修復和優化，等功能。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"和朱琦一樣，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"愛奇藝中不少員工也都是開源社區的積極參與者。","attrs":{}},{"type":"text","text":"他們在開源社區中貢獻代碼、解答其他社區參與者遇到的問題，維持社區的正常、有序運轉。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"朱琦對此深有感觸，“Apache發展十幾年，就是因爲有不同的用戶和開發者，良性維護生態，共同交流互動。大家一方面在這裏交流經驗，解決自己遇到的問題，另一方面也可以從別的公司或者開發者提出的相關問題中吸收經驗，防範於未然。”","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":14}},{"type":"strong","attrs":{}}],"text":"04 從入門到終極，開源進階攻略！","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"親歷了開源文化近些年在國內的發展，朱琦也頗爲感慨。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“之前開源其實沒有這麼火熱，大家對開源的理解就是借來用，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"不太願意貢獻和參與其中","attrs":{}},{"type":"text","text":"，但也存在不少問題。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"很多公司沒有使用開源項目，是閉源的","attrs":{}},{"type":"text","text":"，相關研發人員離職，代碼就有很多歷史包袱。現在","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"開源已經是一個大趨勢","attrs":{}},{"type":"text","text":"，對公司來說，現在國內外各個互聯網公司，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"特別是愛奇藝內部工程師文化建設中","attrs":{}},{"type":"text","text":"，都越來越重視開源文化了，通過開源，不僅公司的項目也會越來越好，貢獻者本身也可以從中獲得很多。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，朱琦也對想要入門開源文化的同學，給出了一些從入門、進階到終極的建議。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"1.入門","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"熟悉開源項目的貢獻流程","attrs":{}},{"type":"text","text":"，各個項目都會有how to contribute的文檔。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"小的問題","attrs":{}},{"type":"text","text":"着手開始貢獻，例如代碼中的拼寫錯誤，項目文檔的改進，小的bug的修復。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"2.進階","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"能夠對項目在","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"生產環境","attrs":{}},{"type":"text","text":"產生的問題，或者","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"社區用戶遇到的問題","attrs":{}},{"type":"text","text":"進行修復，性能優化等。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"3.終極","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於項目的","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"重大feature","attrs":{}},{"type":"text","text":"的設計文檔進行理解，甚至","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"參與設計文檔","attrs":{}},{"type":"text","text":"，然後","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"根據項目架構進行添加子任務進行開發貢獻。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“總體來說，開源社區貢獻是一件需要","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"持續花費時間和精力","attrs":{}},{"type":"text","text":"的事情。從喫透一個模塊開始，逐漸深入，多貢獻，也多參與問題討論和review代碼，祝大家都能在開源社區走得更遠更好。”","attrs":{}}]}]}