Netflix实用API设计(上)

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic","attrs":{}}],"text":"gRPC如今被很多公司应用在大规模生产环境中，很多时候我们并不需要通过RPC请求所有数据，而只关心响应数据中的部分字段，Protobuf FieldMask就可以帮助我们实现这一目的。本文介绍了Netflix基于FieldMask设计更高效健壮的API的实践，全文分两个部分，这是第一部分。原文：Practical API Design at Netflix, Part 1: Using Protobuf FieldMask","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[1]","attrs":{}}],"marks":[{"type":"italic"}],"attrs":{}}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"背景","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Netflix，我们大量使用gRPC","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[2]","attrs":{}}],"attrs":{}},{"type":"text","text":"进行后端通信。处理请求的时候，如果能够知道调用者对哪些字段感兴趣、哪些字段可以忽略，会非常有用。有些响应字段的计算代价可能很高，有些字段可能需要对其他服务进行远程调用。远程调用需要付出额外的开销：额外的延迟，更高的出错概率，并且消耗了网络带宽。我们怎么样才能理解哪些字段不需要在响应中提供给调用者，从而避免进行不必要的计算以及远程调用？在GraphQL中，可以通过字段选择器来实现。在JSON:API标准中，有一个类似的被称为Sparse Fieldsets","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[3]","attrs":{}}],"attrs":{}},{"type":"text","text":"的技术。我们在设计gRPC API的时候，能不能实现类似的功能？在Netflix Studio Engineering","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[4]","attrs":{}}],"attrs":{}},{"type":"text","text":"，我们提供的解决方案是protobuf FieldMask","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[5]","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e3/e368cf5ec84e7b7eb9336043b803b13c.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"《纸钞屋》(La casa de papel) / Netflix","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"Protobuf FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Protocol Buffers","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[6]","attrs":{}}],"attrs":{}},{"type":"text","text":"，简称protobuf，是一种数据序列化机制。默认情况下，gRPC使用protobuf作为它的IDL（接口定义语言）和数据序列化协议。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"FieldMask是一个protobuf消息，有许多实用工具和约定用来处理RPC请求中包含的FieldMask。FieldMask消息包含一个名为","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"paths","attrs":{}}],"attrs":{}},{"type":"text","text":"的字段，用于指定应该由读操作返回或由更新操作修改的字段。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"message FieldMask {\n // The set of field mask paths.\n repeated string paths = 1;\n}","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"示例：Netflix工作室内容制作","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a6/a668ffe2bf6b2472b944378ecce01132.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"《纸钞屋》(La casa de papel) / Netflix","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们假设有一个Production服务，可以用来管理工作室内容的生产（Studio Content Productions，在电影和电视行业中，术语production","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[7]","attrs":{}}],"attrs":{}},{"type":"text","text":"指的是制作电影的过程，而不是运行软件的环境）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// Contains Production-related information \nmessage Production {\n string id = 1;\n string title = 2;\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n // ... more fields\n}\n\nservice ProductionService {\n // returns Production by ID\n rpc GetProduction (GetProductionRequest) returns (GetProductionResponse);\n}\n\nmessage GetProductionRequest {\n string production_id = 1;\n}\n\nmessage GetProductionResponse {\n Production production = 1;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"GetProduction","attrs":{}}],"attrs":{}},{"type":"text","text":"通过其唯一ID返回一个生产消息。一个作品包含多个字段，如:标题、格式、日程日期、脚本（又称剧本）、预算、情节等，不过我们将重点放在过滤日程日期和脚本上，这样可以让例子简单一点。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"读取制作细节","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假设我们想要使用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"GetProduction","attrs":{}}],"attrs":{}},{"type":"text","text":" API获取特定产品（如“纸钞屋”）的制作信息。虽然产品有很多字段，但有些字段是从其他服务获取的，比如Schedule服务的返回的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"schedule","attrs":{}}],"attrs":{}},{"type":"text","text":"，或者Script服务返回的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scripts","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9d/9da9b1b8ce6b7580ec3c8b13748e2c71.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即使客户端忽略响应中的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"schedule","attrs":{}}],"attrs":{}},{"type":"text","text":"和","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scripts","attrs":{}}],"attrs":{}},{"type":"text","text":"字段，Production服务仍然需要在每次调用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"GetProduction","attrs":{}}],"attrs":{}},{"type":"text","text":"时为Schedule和Script服务生成RPC。如上所述，远程调用是有成本的。如果服务知道调用者真正关心哪些字段，就可以做出明智的决策，决定是否进行昂贵的调用、启动资源繁重的计算和/或调用数据库。在本例中，如果调用者只需要产品标题和产品格式，那么Production服务就可以避免对Schedule和Script服务进行远程调用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"此外，请求大量字段会使响应负载变得很大，而这对于带宽有限的移动应用来说，可能会造成问题。在这些情况下，消费者只请求他们需要的字段是一个很好的实践。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ba/ba65cf6acc7af3daaaff215aca4bb2a1.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"《纸钞屋》(La casa de papel) / Netflix","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一个丑陋的解决这些问题的方法是添加额外的请求参数，如","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"includeSchedule","attrs":{}}],"attrs":{}},{"type":"text","text":"和","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"includeScripts","attrs":{}}],"attrs":{}},{"type":"text","text":"：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// Request with one-off \"include\" fields, not recommended\nmessage GetProductionRequest {\n string production_id = 1;\n bool include_format = 2;\n bool include_schedule = 3;\n bool include_scripts = 4;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不过这种方法需要为每个开销较大的响应字段添加自定义的","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"includeXXX","attrs":{}}],"attrs":{}},{"type":"text","text":"字段，对于嵌套字段就不太适用，而且还增加了请求的复杂性，最终使得维护和支持更困难。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"在请求消息里添加FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"API设计者可以在请求消息中添加","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"field_mask","attrs":{}}],"attrs":{}},{"type":"text","text":"字段，而不是创建一次性的“include”字段：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"import \"google/protobuf/field_mask.proto\";\n\nmessage GetProductionRequest {\n string production_id = 1;\n google.protobuf.FieldMask field_mask = 2;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"消费者可以为希望在响应中接收的字段设置路径，如果消费者只对产品标题和格式感兴趣，他们可以设置路径为“title”和“format”的FieldMask：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"FieldMask fieldMask = FieldMask.newBuilder()\n .addPaths(\"title\")\n .addPaths(\"format\")\n .build();\n\nGetProductionRequest request = GetProductionRequest.newBuilder()\n .setProductionId(LA_CASA_DE_PAPEL_PRODUCTION_ID)\n .setFieldMask(fieldMask)\n .build();","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/48/48cd541d5a4f669af1b1068341789c59.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":9}}],"text":"Masking fields","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"请注意，尽管本文代码示例是基于Java的，但所演示的概念适用于protocol buffers支持的任何语言。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果消费者只需要上一个更新日程的人的标题和电子邮件，他们可以设置一个不同的字段掩码：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"FieldMask fieldMask = FieldMask.newBuilder()\n .addPaths(\"title\")\n .addPaths(\"schedule.last_updated_by.email\")\n .build();\n\nGetProductionRequest request = GetProductionRequest.newBuilder()\n .setProductionId(LA_CASA_DE_PAPEL_PRODUCTION_ID)\n .setFieldMask(fieldMask)\n .build();","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按照惯例，如果请求中没有包含FieldMask，则应该返回所有字段。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"Protobuf字段名 vs 字段号","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"你可能已经注意到，FieldMask中的路径是基于字段名指定的，但实际上，编码后的protocol buffers消息只包含字段号，而不包含字段名，因此（以及其他技术，比如用于签名类型编码的ZigZag","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[8]","attrs":{}}],"attrs":{}},{"type":"text","text":"）protobuf消息的空间效率更高。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了理解字段号和字段名之间的区别，让我们详细了解一下protobuf是如何编码和解码消息的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Production消息的protobuf消息定义（.proto文件）含有五个字段，每个字段都有一个类型、名称和数字。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// Message with Production-related information \nmessage Production {\n string id = 1;\n string title = 2;\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当protobuf编译器（protoc）编译这个消息定义时，会根据我们选择的语言（示例中是Java）创建代码。生成的代码包含用于定义消息的类，以及消息和字段描述符。描述符包含将消息编码和解码成二进制格式所需的所有信息。例如，它们包含字段编号、名称和类型。消息生成程序使用描述符将消息编码为传输格式。为了提高效率，二进制消息里只包含字段号和对应的值，不包括字段名。当使用者接收到消息时，它通过引用已编译的消息定义将字节流解码为对象（例如，Java对象）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e9/e9816d385509b6c7dea8803f03dbc17c.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上所述，FieldMask列出的是字段名，而不是数字。而在Netflix，我们使用字段号，并通过FieldMaskUtil.fromFieldNumbers()","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[9]","attrs":{}}],"attrs":{}},{"type":"text","text":"辅助程序转换为字段名。fromFieldNumbers方法利用已编译的消息定义将字段号转换为字段名，并创建FieldMask。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"FieldMask fieldMask = FieldMaskUtil.fromFieldNumbers(Production.class,\n Production.TITLE_FIELD_NUMBER,\n Production.FORMAT_FIELD_NUMBER);\n\nGetProductionRequest request = GetProductionRequest.newBuilder()\n .setProductionId(LA_CASA_DE_PAPEL_PRODUCTION_ID)\n .setFieldMask(fieldMask)\n .build();","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是，有一个很容易被忽略的限制：使用FieldMask会限制我们重命名消息字段的能力。重命名消息字段通常被认为是安全的操作，因为如上所述，字段名并没有被编码发送，而是基于消费者端的消息定义产生的。要是使用FieldMask，字段名就会编码在消息的有效负载中被发送出去（在","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"paths","attrs":{}}],"attrs":{}},{"type":"text","text":"字段值中）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"假设我们想将字段","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title","attrs":{}}],"attrs":{}},{"type":"text","text":"重命名为","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title_name","attrs":{}}],"attrs":{}},{"type":"text","text":"，并发布消息定义的2.0版：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"// version 2.0, with title field renamed to title_name\nmessage Production {\n string id = 1;\n string title_name = 2; // this field used to be \"title\"\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6f/6fcd5e0ed9a96bc56cadb7d9e1e6ef0b.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在这个图中，生产者（服务器端）使用了新的描述符，字段2名为","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title_name","attrs":{}}],"attrs":{}},{"type":"text","text":"。通过网络发送的二进制消息包含字段号及其值。消费者仍然使用原始的描述符，其中字段号2名为","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"title","attrs":{}}],"attrs":{}},{"type":"text","text":"，仍然能够通过字段号解码消息。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果消费者不使用FieldMask请求字段，这种方法仍然可以工作的很好。不过一旦消费者使用FieldMask字段中的“title”路径进行调用，生产者将无法找到该字段。生产者的描述符中没有名为title的字段，所以不知道消费者要求的字段号是2。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8c/8cca684f14f0988e3995bb03bea70a38.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如上所见，如果一个字段被重命名，后台应该能够支持新的和旧的字段名，直到所有调用者都迁移到新的字段名（向后兼容）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有多种解决方案可以处理这个问题：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用FieldMask时，永远不要重命名字段。这是最简单的解决方案，但并不总是可行。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要求后端支持所有旧字段名。这解决了向后兼容性问题，但需要在后端添加额外的代码来跟踪所有历史字段名。","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"弃用旧字段并创建一个新字段而不是重命名。在我们的示例中，我们将创建新的title_name，并设置字段号为6。与前一个选项相比，这个选项的优点在于：允许生产者继续使用生成的描述符而不是自定义转换器；另外，可以让消费者很快察觉到某个字段已经被弃用了。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":null},"content":[{"type":"text","text":"message Production {\n string id = 1;\n string title = 2 [deprecated = true]; // use \"title_name\" field instead\n ProductionFormat format = 3;\n repeated ProductionScript scripts = 4;\n ProductionSchedule schedule = 5;\n string title_name = 6;\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不管用哪个解决方案，最重要的是要记住，FieldMask使字段名称成为API合约的一个组成部分。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"在生产者（服务器）端使用FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在生产者（服务器）端，可以使用FieldMaskUtil.merge()","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[10]","attrs":{}}],"attrs":{}},{"type":"text","text":"方法从响应负载中删除不必要的字段（第8、9行）：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"@Override\npublic void getProduction(GetProductionRequest request,\n\t\t\t\t\t\t\t\t\t\t\t\t\tStreamObserver response) {\n\t\n Production production = fetchProduction(request.getProductionId());\n\tFieldMask fieldMask = request.getFieldMask();\n\n\tProduction.Builder productionWithMaskedFields = Production.newBuilder();\n\tFieldMaskUtil.merge(fieldMask, production, productionWithMaskedFields);\n\n\tGetProductionResponse response = GetProductionResponse.newBuilder()\n .setProduction(productionWithMaskedFields).build();\n\tresponseObserver.onNext(response);\n\tresponseObserver.onCompleted();\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果服务器代码需要知道哪些字段被请求，以避免进行外部调用、数据库查询或昂贵的计算，可以从FieldMask paths字段获得相关信息：","attrs":{}}]},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"private static final String FIELD_SEPARATOR_REGEX = \"\\.\";\nprivate static final String MAX_FIELD_NESTING = 2;\nprivate static final String SCHEDULE_FIELD_NAME = // (1)\n\tProduction.getDescriptor().\n findFieldByNumber(Production.SCHEDULE_FIELD_NUMBER).getName();\n\n@Override\npublic void getProduction(GetProductionRequest request,\n\t\t\t\t\t\t\t\t\t\t\t\t\tStreamObserver response) {\n \n\tFieldMask canonicalFieldMask = \n FieldMaskUtil.normalize(request.getFieldMask()); // (2) \n\n\tboolean scheduleFieldRequested = // (3)\n canonicalFieldMask.getPathsList().stream()\n .map(path -> path.split(FIELD_SEPARATOR_REGEX, MAX_FIELD_NESTING)[0])\n .anyMatch(SCHEDULE_FIELD_NAME::equals);\n\n\tif (scheduleFieldRequested) {\n ProductionSchedule schedule = \n makeExpensiveCallToScheduleService(request.getProductionId()); // (4)\n ...\n\t}\n\n\t...\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这段代码只在请求","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"schedule","attrs":{}}],"attrs":{}},{"type":"text","text":"字段时调用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"makeExpensiveCallToScheduleServicemethod","attrs":{}}],"attrs":{}},{"type":"text","text":"（第21行），让我们仔细看一下这段代码。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（1）","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"SCHEDULE_FIELD_NAME","attrs":{}}],"attrs":{}},{"type":"text","text":"常量包含字段名。在示例代码中使用消息类型Descriptor","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[11]","attrs":{}}],"attrs":{}},{"type":"text","text":"和FieldDescriptor","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[12]","attrs":{}}],"attrs":{}},{"type":"text","text":"按字段编号查找字段名。protobuf字段名和字段号之间的区别请参考前面的介绍。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（2）FieldMaskUtil.normalize()","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[13]","attrs":{}}],"attrs":{}},{"type":"text","text":"返回按字母顺序排序并删除了重复数据的FieldMask（又称规范形式）。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（3）生成","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scheduleFieldRequested","attrs":{}}],"attrs":{}},{"type":"text","text":"的表达式（第14 - 17行）接受FieldMask路径流，将其映射为顶级字段流，如果顶级字段中包含","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"SCHEDULE_FIELD_NAME","attrs":{}}],"attrs":{}},{"type":"text","text":"常量的值，则返回","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"true","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（4）只有当","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"scheduleFieldRequested","attrs":{}}],"attrs":{}},{"type":"text","text":"为","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"true","attrs":{}}],"attrs":{}},{"type":"text","text":"时，才会检索","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"ProductionSchedule","attrs":{}}],"attrs":{}},{"type":"text","text":"。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果需要为不同的消息和字段应用FieldMask，可以考虑创建可重用的helper方法。例如，可以基于FieldMask和FieldDescriptor返回所有顶级字段的方法，判断字段是否出现在FieldMask中的方法，等等。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"使用预构建的FieldMask","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有些访问模式可能比其他模式更常见，如果多个消费者都对同一个字段子集感兴趣，API生产商可以为最常用的字段组合提供预构建的FieldMask客户端库。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"public class ProductionFieldMasks {\n\t/**\n * Can be used in {@link GetProductionRequest} to query\n * production title and format\n */\n\tpublic static final FieldMask TITLE_AND_FORMAT_FIELD_MASK =\n \tFieldMaskUtil.fromFieldNumbers(Production.class,\n \tProduction.TITLE_FIELD_NUMBER, Production.FORMAT_FIELD_NUMBER);\n \n /**\n * Can be used in {@link GetProductionRequest} to query \n * production title and schedule\n */\n\tpublic static final FieldMask TITLE_AND_SCHEDULE_FIELD_MASK = \n FieldMaskUtil.fromFieldNumbers(Production.class,\n Production.TITLE_FIELD_NUMBER, \n Production.SCHEDULE_FIELD_NUMBER);\n\n /**\n * Can be used in {@link GetProductionRequest} to query \n * production title and scripts\n */\n public static final FieldMask TITLE_AND_SCRIPTS_FIELD_MASK = \n FieldMaskUtil.fromFieldNumbers(Production.class,\n Production.TITLE_FIELD_NUMBER, Production.SCRIPTS_FIELD_NUMBER);\n\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供预构建字段掩码简化了大多数常见场景的API使用，并让消费者可以灵活的为更具体的用例构建自己的字段掩码。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"限制","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用FieldMask限制了重命名消息字段的能力","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"重复字段只允许出现在路径字符串的最后一个位置，这意味着不能在列表中选择（屏蔽）单个子字段。在可预见的未来，这种情况可能会改变，因为最近批准的谷歌API改进建议AIP-161 Field masks","attrs":{}},{"type":"sup","content":[{"type":"text","text":"[14]","attrs":{}}],"attrs":{}},{"type":"text","text":"包含了对重复字段的通配符支持。","attrs":{}}]}]}],"attrs":{}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"最后","attrs":{}}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/56/567da6a1f7d898cd505dc28ca1d26a25.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Protobuf FieldMask是一个简单而强大的概念，有助于使API更健壮，服务实现更高效。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文介绍了Netflix Studio Engineering如何以及为什么使用FieldMask来读取数据，","attrs":{}},{"type":"link","attrs":{"href":"https://xie.infoq.cn/article/0e7e09acbfc5610aac3c0a6e0","title":"","type":null},"content":[{"type":"text","text":"下一篇文章","attrs":{}}]},{"type":"text","text":"将介绍如何使用FieldMask进行更新和删除操作。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"References:","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[1] ","attrs":{}},{"type":"link","attrs":{"href":"https://netflixtechblog.com/practical-api-design-at-netflix-part-1-using-protobuf-fieldmask-35cfdc606518","title":"","type":null},"content":[{"type":"text","text":"https://netflixtechblog.com/practical-api-design-at-netflix-part-1-using-protobuf-fieldmask-35cfdc606518","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[2] ","attrs":{}},{"type":"link","attrs":{"href":"https://grpc.io/","title":"","type":null},"content":[{"type":"text","text":"https://grpc.io/","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[3] ","attrs":{}},{"type":"link","attrs":{"href":"https://jsonapi.org/format/#fetching-sparse-fieldsets","title":"","type":null},"content":[{"type":"text","text":"https://jsonapi.org/format/#fetching-sparse-fieldsets","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[4] ","attrs":{}},{"type":"link","attrs":{"href":"https://netflixtechblog.com/netflix-studio-engineering-overview-ed60afcfa0ce","title":"","type":null},"content":[{"type":"text","text":"https://netflixtechblog.com/netflix-studio-engineering-overview-ed60afcfa0ce","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[5] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/field-mask","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/csharp/class/google/protobuf/well-known-types/field-mask","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[6] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[7] ","attrs":{}},{"type":"link","attrs":{"href":"https://en.wikipedia.org/wiki/Filmmaking","title":"","type":null},"content":[{"type":"text","text":"https://en.wikipedia.org/wiki/Filmmaking","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[8] ","attrs":{}},{"type":"link","attrs":{"href":"https://en.wikipedia.org/wiki/Variable-length_quantity#Zigzag_encoding","title":"","type":null},"content":[{"type":"text","text":"https://en.wikipedia.org/wiki/Variable-length_quantity#Zigzag_encoding","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[9] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#fromFieldNumbers-java.lang.Class-int...-","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#fromFieldNumbers-java.lang.Class-int...-","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[10] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#merge-com.google.protobuf.FieldMask-com.google.protobuf.Message-com.google.protobuf.Message.Builder-","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#merge-com.google.protobuf.FieldMask-com.google.protobuf.Message-com.google.protobuf.Message.Builder-","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[11] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[12] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors.FieldDescriptor.html","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/Descriptors.FieldDescriptor.html","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[13] ","attrs":{}},{"type":"link","attrs":{"href":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#normalize-com.google.protobuf.FieldMask-","title":"","type":null},"content":[{"type":"text","text":"https://developers.google.com/protocol-buffers/docs/reference/java/com/google/protobuf/util/FieldMaskUtil.html#normalize-com.google.protobuf.FieldMask-","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[14] ","attrs":{}},{"type":"link","attrs":{"href":"https://google.aip.dev/161","title":"","type":null},"content":[{"type":"text","text":"https://google.aip.dev/161","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"你好，我是俞凡，在Motorola做过研发，现在在Mavenir做技术工作，对通信、网络、后端架构、云原生、DevOps、CICD、区块链、AI等技术始终保持着浓厚的兴趣，平时喜欢阅读、思考，相信持续学习、终身成长，欢迎一起交流学习。微信公众号：DeepNoMind","attrs":{}}]}],"attrs":{}}]}