Netflix如何通過GraphQL Federation擴展API?

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/how-netflix-scales-its-api-with-graphql-federation-part-1-ae3557c187e2","title":null,"type":null},"content":[{"type":"text","text":"之前的文章"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"和"},{"type":"link","attrs":{"href":"https:\/\/www.infoq.com\/presentations\/netflix-api-graphql-federation\/","title":null,"type":null},"content":[{"type":"text","text":"QConPlus討論"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"中,我們討論了GraphQL Federation作爲一種解決方案是如何分發我們的GraphQL模式及其實現的。 在這篇文章中,我們會將注意力轉移到成功運行聯邦GraphQL平臺所需的內容上——從它實現的過程到我們汲取的經驗教訓等方面。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/fc\/73\/fc3508b87a37e6624c69fb6b7da3bc73.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"我們迄今爲止的旅程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在過去的一年中,我們已經實現了聯邦GraphQL架構所需的核心基礎架構,正如我們前一篇文章所描述的那樣:"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/90\/90608eb3c84349eeceb723d0a268b0b3.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Studio Edge架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"該平臺上的第一個Graph域服務(Domain Graph Service,DGS)是我們在第一篇文章(Studio API)中討論過的GraphQL單體應用。接下來,我們與其他幾個應用程序團隊合作一起製作了DGS,使其API能與之前的單體應用一起公開。到2019年底,我們有了第一個使用聯邦Graph的Studio應用程序,性能沒有任何下降。我們知道這個架構是可行的之後,就專注於爲它更廣泛的使用做準備了。我們的目標是在2020年4月開放Studio Edge自助服務平臺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"2020年4月是一個動盪的時期,疫情大流行,人們一夜之間就過渡到了遠程辦公。儘管如此,各小組還是開始了成羣結隊地投入到Graph中。很快,我們每天都有數百名工程師直接爲API貢獻力量。而那個曾經是瓶頸的Studio API單體應用呢?我們將Studio API公開的字段遷移到個人擁有的DGS中,而不破壞用戶的API。原單體應用預計將在2020年底完全棄用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這段旅程並非沒有挑戰。最大的挑戰在於需在整個組織內協調這一戰略。最初,有很多人持懷疑態度並有異議;這個概念相當新,需要整個組織高度一致才能成功。我們的團隊花了很多時間來解決不同意見,並根據開發人員的反饋對架構進行了調整。通過我們的原型開發以及與一些關鍵批評聲音的積極合作,我們能夠逐漸增加信心並彌合關鍵差距。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"一旦我們在這個想法上達成了廣泛的一致,我們就需要確保其採用是無縫的。 這就需要構建魯棒性強的核心基礎設施,以確保良好的開發人員體驗,並能解決關鍵的跨域問題。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"核心基礎設施"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們的GraphQL網關(GraphQL Gateway)基於Apollo的參考實現,並且用"},{"type":"link","attrs":{"href":"https:\/\/kotlinlang.org\/","title":null,"type":null},"content":[{"type":"text","text":"Kotlin"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"編寫。這使得我們能夠訪問Netflix的Java生態系統,同時也爲我們提供了魯棒性強的語言特性,比如用於高效並行抓取的"},{"type":"link","attrs":{"href":"https:\/\/kotlinlang.org\/docs\/reference\/coroutines-overview.html","title":null,"type":null},"content":[{"type":"text","text":"協程(coroutines)"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":",以及具有null安全的表述性類型系統(expressive type system)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"模式註冊(Schema Registry)是內部開發的,也是用的Kotlin。爲了存儲模式變更,我們使用了一個"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/scaling-event-sourcing-for-netflix-downloads-episode-2-ce1b54d46eec","title":null,"type":null},"content":[{"type":"text","text":"內部庫"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":",該庫是在"},{"type":"link","attrs":{"href":"https:\/\/cassandra.apache.org\/","title":null,"type":null},"content":[{"type":"text","text":"Cassandra"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"數據庫之上實現的"},{"type":"link","attrs":{"href":"https:\/\/microservices.io\/patterns\/data\/event-sourcing.html","title":null,"type":null},"content":[{"type":"text","text":"事件源模式"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。使用事件源允許我們實現新的開發者體驗特性,比如Schema History視圖。模式註冊還集成了我們的CI\/CD系統,如"},{"type":"link","attrs":{"href":"https:\/\/spinnaker.io\/","title":null,"type":null},"content":[{"type":"text","text":"Spinnaker"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":",可以爲DGS自動設置雲網絡。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"開發人員教育與經驗"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在之前的架構中,只有單體應用Studio API的團隊需要學習GraphQL。在Studio Edge中,每個DGS團隊都需要在GraphQL上積累專業知識。GraphQL有自己的學習曲線,對於像"},{"type":"link","attrs":{"href":"http:\/\/batching","title":null,"type":null},"content":[{"type":"text","text":"批處理(Batching)"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"和"},{"type":"link","attrs":{"href":"https:\/\/www.graphql-java.com\/blog\/deep-dive-data-fetcher-results\/","title":null,"type":null},"content":[{"type":"text","text":"先行斷言(Lookahead)"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這樣的複雜情況,它會變得特別棘手。另外,正如前一篇文章所討論的那樣,理解GraphQL Federation和實現實體解析器(Entity Resolver)也不是一件容易的事。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們與Netflix的開發者體驗(Netflix’s Developer Experience,DevEx)團隊合作,爲開發人員提供文檔、培訓材料和教程。對於一般的GraphQL問題,我們依賴於開源社區,並建立了一個內部GraphQL社區來討論諸如分頁、錯誤處理、可空性和命名約定之類的熱門話題。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"DGS框架和開發者工具"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了方便後端工程師構建GraphQL DGS,DevEx團隊在GraphQL Java和Spring Boot的基礎之上構建了一個“DGS框架”(“DGS Framework”)。該框架解決了運行在生產環境中的GraphQL服務的所有跨域問題,同時也使開發人員能更容易地編寫GraphQL解析器。此外,DevEx還構建了強大的工具,可用於將模式推送到模式註冊Schema Registry中,並構建了一個自助服務UI(Self Service UI),可用於瀏覽各種DGS模式。點擊查看"},{"type":"link","attrs":{"href":"https:\/\/plus.qconferences.com\/plus2020\/presentation\/using-devex-accelerate-graphql-federation-adoption-netflix","title":null,"type":null},"content":[{"type":"text","text":"他們的會議演講"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":",期待我們的同事將來能發表一篇相關的博文。DGS框架計劃於2021年初開源。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"模式治理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Netflix的Studio數據極其豐富和複雜。在早期,我們預期活動的模式管理對模式演進和整體的健康是至關重要的。我們組織中已經有一位Studio數據架構師(Studio Data Architect)了,他專注於跨Studio的數據建模和對齊。我們與他們合作,一起確定了最適用於Studio Engineering需求的Graph模式的最佳實踐。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們的目標是設計一個能夠反映域本身而不是數據庫模型的GraphQL模式。UI開發人員不必構建面向前端的後端(Backends For Frontends,BFF)來根據他們的需要處理數據,相反,他們應該協助一起來塑造模式,以滿足他們的需求。擁抱協作的模式設計方法對於實現這一目標至關重要。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/bb\/5a\/bbde2400dec8d7d3c0620b11c869015a.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Schema設計工作流"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"協作設計過程涉及跨團隊的反饋和審查。爲了簡化模式設計和審查,我們成立了一個模式工作組並開發了一個用於加入聯邦架構的託管技術程序。雖然審查會增加產品開發過程的開銷,但我們相信,優先考慮Graph模型的質量能減少未來的變更及其所需的反工量。審查的級別因受影響的實體而異;對於核心聯邦類型,則需要更嚴格的審查(儘管工具有助於簡化該流程)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們有一個用於改進模式的deprecation工作流。我們利用了GraphQL的"},{"type":"link","attrs":{"href":"https:\/\/spec.graphql.org\/June2018\/#sec--deprecated","title":null,"type":null},"content":[{"type":"text","text":"deprecation"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"特性,並且還跟蹤了模式中每個字段的使用統計信息。一旦統計數據表明已棄用字段不再使用,我們就可以進行向後不兼容的變更,將該字段從模式中刪除。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/57\/20\/57ab79a3d755c8312c802cfedd3fae20.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"顯示已棄用字段使用情況的客戶端"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們採用了模式優先的方法,而不是從現有的模型中(例如gRPC API的Protobuf對象)生成模式。雖然"},{"type":"link","attrs":{"href":"https:\/\/developers.google.com\/protocol-buffers","title":null,"type":null},"content":[{"type":"text","text":"Protobu"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"f和"},{"type":"link","attrs":{"href":"https:\/\/grpc.io\/docs\/what-is-grpc\/introduction\/","title":null,"type":null},"content":[{"type":"text","text":"gRPC"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"是構建服務API的優秀解決方案,但我們更喜歡將GraphQL模式與這些層解耦,以實現更清晰的Graph設計和獨立的可擴展性。在某些場景中,我們實現了從GraphQL解析器到gRPC調用的通用映射編碼,但是額外的樣板對GraphQL API的長期靈活性來說是值得的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們使用的方法是基於“上下文高於控制”(“context over control”)的,這是"},{"type":"link","attrs":{"href":"https:\/\/jobs.netflix.com\/culture","title":null,"type":null},"content":[{"type":"text","text":"Netflix文化的一個重要原則"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。我們沒有試圖嚴格控制整個Graph,而是爲產品團隊提供指導和上下文,這樣他們就可以應用自己的領域知識爲自己的領域創建靈活的API。隨着該架構的成熟,我們將繼續監控模式運行情況,並在需要的時候開發新的工具、流程和最佳實踐。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"可觀測性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在我們之前的架構中,可觀測性是通過人工分析和API團隊的路由來實現的,它的可伸縮性很差。對於我們的聯邦架構,我們以一種更具可伸縮性的方式來優先解決可觀測性的需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們優先考慮如下三個方面:"}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"報警——當發生錯誤時"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"(when)"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"報告"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"發現——輕鬆確定哪些"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"(what)"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"是不工作的"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"診斷——調試某些東西不工作的原因"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"(why)"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們在這一領域的指導度量指標是平均修復時間(MTTR)以及服務等級目標和服務等級指標(SLO\/SLI)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們與Netflix遙測團隊的專家合作。我們將網關和DGS架構組件與"},{"type":"link","attrs":{"href":"https:\/\/zipkin.io\/","title":null,"type":null},"content":[{"type":"text","text":"Zipkin"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"、內部分佈式跟蹤工具"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/edgar-solving-mysteries-faster-with-observability-e1a76302c71f","title":null,"type":null},"content":[{"type":"text","text":"Edgar"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"和應用程序監控工具"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/telltale-netflix-application-monitoring-simplified-5c08bfa780ba","title":null,"type":null},"content":[{"type":"text","text":"TellTal"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"e集成在一起。在GraphQL中,幾乎每個響應都是200,錯誤塊中包含了自定義的錯誤。我們從響應中檢查這些自定義的錯誤代碼,並將它們發送到我們的度量服務"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/introducing-atlas-netflixs-primary-telemetry-platform-bd31f4d8ed9a","title":null,"type":null},"content":[{"type":"text","text":"Atlas"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"中。這些集成爲GraphQL API的使用者和開發人員奠定了豐富的可見性和洞察力的良好基礎。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/83\/b9\/83a304e4150aceb8a411ec831dbe9cb9.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"監控聯邦請求生命週期的Edgar Trace"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/d2\/d29ab922234b9834563a68a6d3397c06.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"聯邦請求的時間軸視圖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"分佈式日誌關聯(Distributed Log Correlation)有助於調試更復雜的服務問題。通過顯示處理請求所涉及的所有系統的應用級日誌詳細信息,我們可以更深入地瞭解堆棧中發生的事情。開發人員可以很容易地看到與給定請求同時發生的事情,以排查可能影響交互的周邊因素。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/5c\/5c6aeec5c8eae500fb8d79930993c3dc.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"一個跨多個服務記錄的聯邦請求的日誌"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了解決“我應該問誰("},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"who"},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":")……”的路由問題,我們集成了從GraphQL類型和字段到它們所屬團隊的支持(support)渠道的深度鏈接。現在,尋找support只需單擊跟蹤中的鏈接,這有助於縮短MTTR並能減少網關團隊所需的參與次數。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"保護聯邦Graph的安全"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 我們的目標是在整個聯邦架構中實現可靠且一致的安全實踐。爲了實現這一點,我們與Netflix的安全專家合作,將安全性構建到Graph中。讓我們來看下我們安全解決方案的兩個基本部分:AuthN和AuthZ。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"身份認證"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們在Studio空間中的所有產品體驗都需要一個經過認證的帳戶,因此我們將GraphQL網關的訪問權限限制爲只允許受信任的經過身份驗證的調用者。此外,"},{"type":"link","attrs":{"href":"https:\/\/graphql.org\/learn\/introspection\/","title":null,"type":null},"content":[{"type":"text","text":"Graph Introspection"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"僅限於Netflix的內部開發人員。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"授權"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在Studio Edge之前,授權邏輯在各個團隊之間是分散的。一些團隊在他們的BFF中實現了授權,一些在微服務中實現了授權,而另一些團隊則在這兩處兼而有之。結果往往是,由於用戶訪問的UI不同,對於給定的數據,授權也不同。UI團隊還發現他們需要對每個新的前端實現(以及重新實現)進行授權檢查。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在Studio Edge中,我們將授權責任委託給DGS的所有者。這就實現了在不同應用程序中對同一用戶進行一致的授權。此外,產品經理、工程師和安全團隊可以很容易地瞭解誰可以訪問哪種數據類型以及如何訪問。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們在Netflix內部提供了多種授權服務:從基於用戶身份授予訪問權限的簡單系統,到引入角色和功能概念的更細粒度的系統。DGS開發人員可以根據自己的需要選擇解決方案。然後,他們只需使用@Secured註解對解析器進行註解,並將其配置爲使用一個可用的系統即可。如果需要,可以在解析器或下游系統中實現更復雜的授權。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"授權的未來"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們目前正在構建一個支持GraphQL授權解決方案的原型。註冊模式時,Schema Registry會自動爲每個字段及其相應類型生成訪問控制組(Access Control Groups,ACGs)。產品經理和DGS工程師可以決策這些生成的ACG的成員資格和規則。由於ACG映射到GraphQL中的某個字段,因此DGS框架會在執行期間自動應用與ACG關聯的規則。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"面向失敗的架構設計"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"GraphQL網關是所有請求的單一入口點;網關上的故障可能會導致嚴重的中斷。根據Netflix工程的最佳實踐,我們假設會發生故障,並設計方法來減輕這些故障的影響。以下是我們用來確保網關層具有彈性的設計原則:"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"單一用途"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"無狀態服務"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"需求控制"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"多區域"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"按功能分片"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"首先,我們將網關層的職責集中在一個用途上:解析客戶端查詢,然後構建和執行查詢計劃。通過縮小範圍,我們限制了可能發生的問題的範圍。我們的目標是即時執行日誌記錄和度量指標之外的任何其他資源密集型操作。在網關層中添加其他不相關的邏輯可能會增加在這一關鍵層中出現故障的表面積。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"其次,我們運行網關服務的多個無狀態實例。任何網關實例都能夠爲任何請求生成和執行查詢計劃。當我們對網關層進行代碼變更時,我們會在正式投入生產之前對它們進行嚴格的測試。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"第三,我們通過應用需求控制來平衡每個請求消耗的資源。我們限制調用方的速率,以避免底層數據庫過載,這些數據庫是大多數領域元素的來源。我們還對所有傳入的查詢運行靜態查詢開銷計算,並拒絕昂貴的查詢,以避免網關和DGS資源的阻塞。我們的合作伙伴瞭解這些折衷方案,並與我們一起來滿足這些要求,重新處理昂貴的查詢並減少大的調用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"第四,我們在世界各地的多個AWS區域中部署了網關層。這使我們能夠限制爆炸半徑以應對發生不可避免的問題。當問題發生時,我們可以將故障轉移到另一個區域,以確保我們的客戶受到的影響最小。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"最後,我們部署多個網關層的功能分片(Shard),每個分片中的代碼均相同,傳入的請求根據類別進行路由。例如,GraphQL訂閱(Subscription)通常會導致長時間的連接,而查詢(Query)和突變(Mutation)則是短暫的。我們使用一個單獨的實例組來處理訂閱,因此“連接耗盡”也不會影響查詢和突變的可用性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們還可以做更多的事情來提高彈性。我們計劃對網關部署以及最終的模"},{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/automated-canary-analysis-at-netflix-with-kayenta-3260bc7acc69","title":null,"type":null},"content":[{"type":"text","text":"式變更進行金絲雀(Canary)部署和分析"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"。今天,我們的網關通過輪詢模式註冊來動態更新其模式。我們正在通過將聯邦配置存儲在一個版本化的S3存儲桶中來實現這些功能的解耦,從而使網關能夠抵禦模式註冊的故障。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"結束語"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"GraphQL和Federation已成爲Studio應用程序的效率倍增器。基於此,我們最近使用GraphQL Federation爲iOS和Android上的Netflix消費者應用程序搜索頁面創建了原型。爲了做到這一點,我們創建了三個DGS來爲消費者Graph的最小部分提供數據。我們正在將一小部分用戶切流到這個替代技術棧中,並測量其高層度量指標。我們很高興看到結果,並進一步探討其在Netflix消費者領域的適用空間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"儘管我們擁有積極的經驗,但GraphQL Federation尚處於成熟期的早期,可能對每個團隊和組織來說並不都是最合適的。學習GraphQL和DGS開發、運行聯邦層並進行遷移需要合作團隊的高度投入以及無縫的跨功能協作。如果你正在考慮朝這個方向發展,我們建議你查看Apollo爲"},{"type":"link","attrs":{"href":"https:\/\/www.apollographql.com\/docs\/federation\/","title":null,"type":null},"content":[{"type":"text","text":"Federation提供的SaaS產品"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":",以及許多學習GraphQL相關的在線資源。對於像我們這樣擁有大量的微服務並需要其聚合在一起的生態系統來說,開發速度和可操作性的提高使得這種轉變是值得的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"最後,我們想聽聽你的意見!如果你已經實現了Federation,或者嘗試用另一種方法來解決這個問題,我們很樂意向你瞭解更多信息。分享知識是我們這個行業快速學習和提高的方式之一。最後,如果你想參與解決諸如Netflix彈性相關的複雜而有趣的問題,請查看我們的"},{"type":"link","attrs":{"href":"https:\/\/jobs.netflix.com\/","title":null,"type":null},"content":[{"type":"text","text":"招聘頁面"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"或直接聯繫我們。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#222222","name":"user"}},{"type":"strong"}],"text":"英文原文鏈接:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/netflixtechblog.com\/how-netflix-scales-its-api-with-graphql-federation-part-2-bbe71aaec44a","title":null,"type":null},"content":[{"type":"text","text":"https:\/\/netflixtechblog.com\/how-netflix-scales-its-api-with-graphql-federation-part-2-bbe71aaec44a"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]}]}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章