Netflix實用API設計(上)

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"gRPC如今被很多公司應用在大規模生產環境中，很多時候我們並不需要通過RPC請求所有數據，而只關心響應數據中的部分字段，Protobuf FieldMask就可以幫助我們實現這一目的。本文介紹了Netflix基於FieldMask設計更高效健壯的API的實踐，全文分兩個部分，這是第一部分。原文：Practical API Design at Netflix, Part 1: Using Protobuf FieldMask","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[1]","attrs":{}}],"marks":[{"type":"italic"}],"attrs":{}}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"背景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Netflix，我們大量使用gRPC","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[2]","attrs":{}}],"attrs":{}},{"type":"text","text":"進行後端通信。處理請求的時候，如果能夠知道調用者對哪些字段感興趣、哪些字段可以忽略，會非常有用。有些響應字段的計算代價可能很高，有些字段可能需要對其他服務進行遠程調用。遠程調用需要付出額外的開銷：額外的延遲，更高的出錯概率，並且消耗了網絡帶寬。我們怎麼樣才能理解哪些字段不需要在響應中提供給調用者，從而避免進行不必要的計算以及遠程調用？在GraphQL中，可以通過字段選擇器來實現。在JSON:API標準中，有一個類似的被稱爲Sparse Fieldsets","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[3]","attrs":{}}],"attrs":{}},{"type":"text","text":"的技術。我們在設計gRPC API的時候，能不能實現類似的功能？在Netflix Studio Engineering","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[4]","attrs":{}}],"attrs":{}},{"type":"text","text":"，我們提供的解決方案是protobuf FieldMask","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[5]","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e3/e368cf5ec84e7b7eb9336043b803b13c.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"《紙鈔屋》(La casa de papel) / Netflix","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"Protobuf FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Protocol Buffers","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[6]","attrs":{}}],"attrs":{}},{"type":"text","text":"，簡稱protobuf，是一種數據序列化機制。默認情況下，gRPC使用protobuf作爲它的IDL（接口定義語言）和數據序列化協議。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"FieldMask是一個protobuf消息，有許多實用工具和約定用來處理RPC請求中包含的FieldMask。FieldMask消息包含一個名爲","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"paths","attrs":{}}],"attrs":{}},{"type":"text","text":"的字段，用於指定應該由讀操作返回或由更新操作修改的字段。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"message FieldMask {\n // The set of field mask paths.\n repeated string paths = 1;\n}","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"示例：Netflix工作室內容製作","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a6/a668ffe2bf6b2472b944378ecce01132.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"《紙鈔屋》(La casa de papel) / Netflix","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們假設有一個Production服務，可以用來管理工作室內容的生產（Studio Content Productions，在電影和電視行業中，術語production","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[7]","attrs":{}}],"attrs":{}},{"type":"text","text":"指的是製作電影的過程，而不是運行軟件的環境）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// Contains Production-related information \nmessage Production {\n string id = 1;\n string title = 2;\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n // ... more fields\n}\n\nservice ProductionService {\n // returns Production by ID\n rpc GetProduction (GetProductionRequest) returns (GetProductionResponse);\n}\n\nmessage GetProductionRequest {\n string production_id = 1;\n}\n\nmessage GetProductionResponse {\n Production production = 1;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"GetProduction","attrs":{}}],"attrs":{}},{"type":"text","text":"通過其唯一ID返回一個生產消息。一個作品包含多個字段，如:標題、格式、日程日期、腳本（又稱劇本）、預算、情節等，不過我們將重點放在過濾日程日期和腳本上，這樣可以讓例子簡單一點。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"讀取製作細節","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假設我們想要使用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"GetProduction","attrs":{}}],"attrs":{}},{"type":"text","text":" API獲取特定產品（如“紙鈔屋”）的製作信息。雖然產品有很多字段，但有些字段是從其他服務獲取的，比如Schedule服務的返回的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"schedule","attrs":{}}],"attrs":{}},{"type":"text","text":"，或者Script服務返回的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scripts","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9d/9da9b1b8ce6b7580ec3c8b13748e2c71.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即使客戶端忽略響應中的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"schedule","attrs":{}}],"attrs":{}},{"type":"text","text":"和","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scripts","attrs":{}}],"attrs":{}},{"type":"text","text":"字段，Production服務仍然需要在每次調用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"GetProduction","attrs":{}}],"attrs":{}},{"type":"text","text":"時爲Schedule和Script服務生成RPC。如上所述，遠程調用是有成本的。如果服務知道調用者真正關心哪些字段，就可以做出明智的決策，決定是否進行昂貴的調用、啓動資源繁重的計算和/或調用數據庫。在本例中，如果調用者只需要產品標題和產品格式，那麼Production服務就可以避免對Schedule和Script服務進行遠程調用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，請求大量字段會使響應負載變得很大，而這對於帶寬有限的移動應用來說，可能會造成問題。在這些情況下，消費者只請求他們需要的字段是一個很好的實踐。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ba/ba65cf6acc7af3daaaff215aca4bb2a1.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"《紙鈔屋》(La casa de papel) / Netflix","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一個醜陋的解決這些問題的方法是添加額外的請求參數，如","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"includeSchedule","attrs":{}}],"attrs":{}},{"type":"text","text":"和","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"includeScripts","attrs":{}}],"attrs":{}},{"type":"text","text":"：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// Request with one-off \"include\" fields, not recommended\nmessage GetProductionRequest {\n string production_id = 1;\n bool include_format = 2;\n bool include_schedule = 3;\n bool include_scripts = 4;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不過這種方法需要爲每個開銷較大的響應字段添加自定義的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"includeXXX","attrs":{}}],"attrs":{}},{"type":"text","text":"字段，對於嵌套字段就不太適用，而且還增加了請求的複雜性，最終使得維護和支持更困難。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"在請求消息裏添加FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"API設計者可以在請求消息中添加","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"field_mask","attrs":{}}],"attrs":{}},{"type":"text","text":"字段，而不是創建一次性的“include”字段：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"import \"google/protobuf/field_mask.proto\";\n\nmessage GetProductionRequest {\n string production_id = 1;\n google.protobuf.FieldMask field_mask = 2;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"消費者可以爲希望在響應中接收的字段設置路徑，如果消費者只對產品標題和格式感興趣，他們可以設置路徑爲“title”和“format”的FieldMask：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"FieldMask fieldMask = FieldMask.newBuilder()\n .addPaths(\"title\")\n .addPaths(\"format\")\n .build();\n\nGetProductionRequest request = GetProductionRequest.newBuilder()\n .setProductionId(LA_CASA_DE_PAPEL_PRODUCTION_ID)\n .setFieldMask(fieldMask)\n .build();","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/48/48cd541d5a4f669af1b1068341789c59.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"Masking fields","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"請注意，儘管本文代碼示例是基於Java的，但所演示的概念適用於protocol buffers支持的任何語言。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果消費者只需要上一個更新日程的人的標題和電子郵件，他們可以設置一個不同的字段掩碼：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"FieldMask fieldMask = FieldMask.newBuilder()\n .addPaths(\"title\")\n .addPaths(\"schedule.last_updated_by.email\")\n .build();\n\nGetProductionRequest request = GetProductionRequest.newBuilder()\n .setProductionId(LA_CASA_DE_PAPEL_PRODUCTION_ID)\n .setFieldMask(fieldMask)\n .build();","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按照慣例，如果請求中沒有包含FieldMask，則應該返回所有字段。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Protobuf字段名 vs 字段號","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"你可能已經注意到，FieldMask中的路徑是基於字段名指定的，但實際上，編碼後的protocol buffers消息只包含字段號，而不包含字段名，因此（以及其他技術，比如用於簽名類型編碼的ZigZag","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[8]","attrs":{}}],"attrs":{}},{"type":"text","text":"）protobuf消息的空間效率更高。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了理解字段號和字段名之間的區別，讓我們詳細瞭解一下protobuf是如何編碼和解碼消息的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Production消息的protobuf消息定義（.proto文件）含有五個字段，每個字段都有一個類型、名稱和數字。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// Message with Production-related information \nmessage Production {\n string id = 1;\n string title = 2;\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當protobuf編譯器（protoc）編譯這個消息定義時，會根據我們選擇的語言（示例中是Java）創建代碼。生成的代碼包含用於定義消息的類，以及消息和字段描述符。描述符包含將消息編碼和解碼成二進制格式所需的所有信息。例如，它們包含字段編號、名稱和類型。消息生成程序使用描述符將消息編碼爲傳輸格式。爲了提高效率，二進制消息裏只包含字段號和對應的值，不包括字段名。當使用者接收到消息時，它通過引用已編譯的消息定義將字節流解碼爲對象（例如，Java對象）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e9/e9816d385509b6c7dea8803f03dbc17c.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上所述，FieldMask列出的是字段名，而不是數字。而在Netflix，我們使用字段號，並通過FieldMaskUtil.fromFieldNumbers()","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[9]","attrs":{}}],"attrs":{}},{"type":"text","text":"輔助程序轉換爲字段名。fromFieldNumbers方法利用已編譯的消息定義將字段號轉換爲字段名，並創建FieldMask。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"FieldMask fieldMask = FieldMaskUtil.fromFieldNumbers(Production.class,\n Production.TITLE_FIELD_NUMBER,\n Production.FORMAT_FIELD_NUMBER);\n\nGetProductionRequest request = GetProductionRequest.newBuilder()\n .setProductionId(LA_CASA_DE_PAPEL_PRODUCTION_ID)\n .setFieldMask(fieldMask)\n .build();","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是，有一個很容易被忽略的限制：使用FieldMask會限制我們重命名消息字段的能力。重命名消息字段通常被認爲是安全的操作，因爲如上所述，字段名並沒有被編碼發送，而是基於消費者端的消息定義產生的。要是使用FieldMask，字段名就會編碼在消息的有效負載中被髮送出去（在","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"paths","attrs":{}}],"attrs":{}},{"type":"text","text":"字段值中）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假設我們想將字段","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title","attrs":{}}],"attrs":{}},{"type":"text","text":"重命名爲","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title_name","attrs":{}}],"attrs":{}},{"type":"text","text":"，併發布消息定義的2.0版：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// version 2.0, with title field renamed to title_name\nmessage Production {\n string id = 1;\n string title_name = 2; // this field used to be \"title\"\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6f/6fcd5e0ed9a96bc56cadb7d9e1e6ef0b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這個圖中，生產者（服務器端）使用了新的描述符，字段2名爲","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title_name","attrs":{}}],"attrs":{}},{"type":"text","text":"。通過網絡發送的二進制消息包含字段號及其值。消費者仍然使用原始的描述符，其中字段號2名爲","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title","attrs":{}}],"attrs":{}},{"type":"text","text":"，仍然能夠通過字段號解碼消息。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果消費者不使用FieldMask請求字段，這種方法仍然可以工作的很好。不過一旦消費者使用FieldMask字段中的“title”路徑進行調用，生產者將無法找到該字段。生產者的描述符中沒有名爲title的字段，所以不知道消費者要求的字段號是2。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8c/8cca684f14f0988e3995bb03bea70a38.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上所見，如果一個字段被重命名，後臺應該能夠支持新的和舊的字段名，直到所有調用者都遷移到新的字段名（向後兼容）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有多種解決方案可以處理這個問題：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用FieldMask時，永遠不要重命名字段。這是最簡單的解決方案，但並不總是可行。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要求後端支持所有舊字段名。這解決了向後兼容性問題，但需要在後端添加額外的代碼來跟蹤所有歷史字段名。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"棄用舊字段並創建一個新字段而不是重命名。在我們的示例中，我們將創建新的title_name，並設置字段號爲6。與前一個選項相比，這個選項的優點在於：允許生產者繼續使用生成的描述符而不是自定義轉換器；另外，可以讓消費者很快察覺到某個字段已經被棄用了。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"message Production {\n string id = 1;\n string title = 2 [deprecated = true]; // use \"title_name\" field instead\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n string title_name = 6;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不管用哪個解決方案，最重要的是要記住，FieldMask使字段名稱成爲API合約的一個組成部分。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"在生產者（服務器）端使用FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在生產者（服務器）端，可以使用FieldMaskUtil.merge()","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[10]","attrs":{}}],"attrs":{}},{"type":"text","text":"方法從響應負載中刪除不必要的字段（第8、9行）：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"@Override\npublic void getProduction(GetProductionRequest request,\n\t\t\t\t\t\t\t\t\t\t\t\t\tStreamObserver response) {\n\t\n Production production = fetchProduction(request.getProductionId());\n\tFieldMask fieldMask = request.getFieldMask();\n\n\tProduction.Builder productionWithMaskedFields = Production.newBuilder();\n\tFieldMaskUtil.merge(fieldMask, production, productionWithMaskedFields);\n\n\tGetProductionResponse response = GetProductionResponse.newBuilder()\n .setProduction(productionWithMaskedFields).build();\n\tresponseObserver.onNext(response);\n\tresponseObserver.onCompleted();\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果服務器代碼需要知道哪些字段被請求，以避免進行外部調用、數據庫查詢或昂貴的計算，可以從FieldMask paths字段獲得相關信息：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"private static final String FIELD_SEPARATOR_REGEX = \"\\.\";\nprivate static final String MAX_FIELD_NESTING = 2;\nprivate static final String SCHEDULE_FIELD_NAME = // (1)\n\tProduction.getDescriptor().\n findFieldByNumber(Production.SCHEDULE_FIELD_NUMBER).getName();\n\n@Override\npublic void getProduction(GetProductionRequest request,\n\t\t\t\t\t\t\t\t\t\t\t\t\tStreamObserver response) {\n \n\tFieldMask canonicalFieldMask = \n FieldMaskUtil.normalize(request.getFieldMask()); // (2) \n\n\tboolean scheduleFieldRequested = // (3)\n canonicalFieldMask.getPathsList().stream()\n .map(path -> path.split(FIELD_SEPARATOR_REGEX, MAX_FIELD_NESTING)[0])\n .anyMatch(SCHEDULE_FIELD_NAME::equals);\n\n\tif (scheduleFieldRequested) {\n ProductionSchedule schedule = \n makeExpensiveCallToScheduleService(request.getProductionId()); // (4)\n ...\n\t}\n\n\t...\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這段代碼只在請求","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"schedule","attrs":{}}],"attrs":{}},{"type":"text","text":"字段時調用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"makeExpensiveCallToScheduleServicemethod","attrs":{}}],"attrs":{}},{"type":"text","text":"（第21行），讓我們仔細看一下這段代碼。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（1）","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"SCHEDULE_FIELD_NAME","attrs":{}}],"attrs":{}},{"type":"text","text":"常量包含字段名。在示例代碼中使用消息類型Descriptor","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[11]","attrs":{}}],"attrs":{}},{"type":"text","text":"和FieldDescriptor","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[12]","attrs":{}}],"attrs":{}},{"type":"text","text":"按字段編號查找字段名。protobuf字段名和字段號之間的區別請參考前面的介紹。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（2）FieldMaskUtil.normalize()","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[13]","attrs":{}}],"attrs":{}},{"type":"text","text":"返回按字母順序排序並刪除了重複數據的FieldMask（又稱規範形式）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（3）生成","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scheduleFieldRequested","attrs":{}}],"attrs":{}},{"type":"text","text":"的表達式（第14 - 17行）接受FieldMask路徑流，將其映射爲頂級字段流，如果頂級字段中包含","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"SCHEDULE_FIELD_NAME","attrs":{}}],"attrs":{}},{"type":"text","text":"常量的值，則返回","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"true","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（4）只有當","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scheduleFieldRequested","attrs":{}}],"attrs":{}},{"type":"text","text":"爲","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"true","attrs":{}}],"attrs":{}},{"type":"text","text":"時，纔會檢索","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"ProductionSchedule","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果需要爲不同的消息和字段應用FieldMask，可以考慮創建可重用的helper方法。例如，可以基於FieldMask和FieldDescriptor返回所有頂級字段的方法，判斷字段是否出現在FieldMask中的方法，等等。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"使用預構建的FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有些訪問模式可能比其他模式更常見，如果多個消費者都對同一個字段子集感興趣，API生產商可以爲最常用的字段組合提供預構建的FieldMask客戶端庫。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"public class ProductionFieldMasks {\n\t/**\n * Can be used in {@link GetProductionRequest} to query\n * production title and format\n */\n\tpublic static final FieldMask TITLE_AND_FORMAT_FIELD_MASK =\n \tFieldMaskUtil.fromFieldNumbers(Production.class,\n \tProduction.TITLE_FIELD_NUMBER, Production.FORMAT_FIELD_NUMBER);\n \n /**\n * Can be used in {@link GetProductionRequest} to query \n * production title and schedule\n */\n\tpublic static final FieldMask TITLE_AND_SCHEDULE_FIELD_MASK = \n FieldMaskUtil.fromFieldNumbers(Production.class,\n Production.TITLE_FIELD_NUMBER, \n Production.SCHEDULE_FIELD_NUMBER);\n\n /**\n * Can be used in {@link GetProductionRequest} to query \n * production title and scripts\n */\n public static final FieldMask TITLE_AND_SCRIPTS_FIELD_MASK = \n FieldMaskUtil.fromFieldNumbers(Production.class,\n Production.TITLE_FIELD_NUMBER, Production.SCRIPTS_FIELD_NUMBER);\n\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供預構建字段掩碼簡化了大多數常見場景的API使用，並讓消費者可以靈活的爲更具體的用例構建自己的字段掩碼。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"限制","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用FieldMask限制了重命名消息字段的能力","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重複字段只允許出現在路徑字符串的最後一個位置，這意味着不能在列表中選擇（屏蔽）單個子字段。在可預見的未來，這種情況可能會改變，因爲最近批准的谷歌API改進建議AIP-161 Field masks","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[14]","attrs":{}}],"attrs":{}},{"type":"text","text":"包含了對重複字段的通配符支持。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"最後","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/56/567da6a1f7d898cd505dc28ca1d26a25.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Protobuf FieldMask是一個簡單而強大的概念，有助於使API更健壯，服務實現更高效。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文介紹了Netflix Studio Engineering如何以及爲什麼使用FieldMask來讀取數據，","attrs":{}},{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/0e7e09acbfc5610aac3c0a6e0","title":"","type":null},"content":[{"type":"text","text":"下一篇文章","attrs":{}}]},{"type":"text","text":"將介紹如何使用FieldMask進行更新和刪除操作。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"References:","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[1] ","attrs":{}},{"type":"link","attrs":{"href":"https://netflixtechblog.com/practical-api-design-at-netflix-part-1-using-protobuf-fieldmask-35cfdc606518","title":"","type":null},"content":[{"type":"text","text":"https://netflixtechblog.com/practical-api-design-at-netflix-part-1-using-protobuf-fieldmask-35cfdc606518","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[2] ","attrs":{}},{"type":"link","attrs":{"href":"https://grpc.io/","title":"","type":null},"content":[{"type":"text","text":"https://grpc.io/","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[3] ","attrs":{}},{"type":"link","attrs":{"href":"https://jsonapi.org/format/#fetching-sparse-fieldsets","title":"","type":null},"content":[{"type":"text","text":"https://jsonapi.org/format/#fetching-sparse-fieldsets","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[4] ","attrs":{}},{"type":"link","attrs":{"href":"https://netflixtechblog.com/netflix-studio-engineering-overview-ed60afcfa0ce","title":"","type":null},"content":[{"type":"text","text":"https://netflixtechblog.com/netflix-studio-engineering-overview-ed60afcfa0ce","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[5] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/field-mask","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/field-mask","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[6] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[7] ","attrs":{}},{"type":"link","attrs":{"href":"https://en.wikipedia.org/wiki/Filmmaking","title":"","type":null},"content":[{"type":"text","text":"https://en.wikipedia.org/wiki/Filmmaking","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[8] ","attrs":{}},{"type":"link","attrs":{"href":"https://en.wikipedia.org/wiki/Variable-length_quantity#Zigzag_encoding","title":"","type":null},"content":[{"type":"text","text":"https://en.wikipedia.org/wiki/Variable-length_quantity#Zigzag_encoding","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[9] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#fromFieldNumbers-java.lang.Class-int...-","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#fromFieldNumbers-java.lang.Class-int...-","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[10] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#merge-com.google.protobuf.FieldMask-com.google.protobuf.Message-com.google.protobuf.Message.Builder-","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#merge-com.google.protobuf.FieldMask-com.google.protobuf.Message-com.google.protobuf.Message.Builder-","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[11] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[12] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors.FieldDescriptor.html","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors.FieldDescriptor.html","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[13] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#normalize-com.google.protobuf.FieldMask-","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#normalize-com.google.protobuf.FieldMask-","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[14] ","attrs":{}},{"type":"link","attrs":{"href":"https://google.aip.dev/161","title":"","type":null},"content":[{"type":"text","text":"https://google.aip.dev/161","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"你好，我是俞凡，在Motorola做過研發，現在在Mavenir做技術工作，對通信、網絡、後端架構、雲原生、DevOps、CICD、區塊鏈、AI等技術始終保持着濃厚的興趣，平時喜歡閱讀、思考，相信持續學習、終身成長，歡迎一起交流學習。微信公衆號：DeepNoMind","attrs":{}}]}],"attrs":{}}]}