如果您最近有學習 gRPC 如何使用的打算,那麼您第一步其實應該學習 Protocol Buffers。
廢話不多說,我們開始學習:
一、定義消息
1、首先看一個簡單的例子:
1 syntax = "proto3"; 2 3 message SearchRequest { 4 string query = 1; 5 int32 page_number = 2; 6 int32 result_per_page = 3; 7 }
第一行 syntax 用於聲明版本,如果不聲明則默認使用版本2。
第三行 message 用於聲明消息結構體。
第四到第六行每個字段後面都有一個數值,用於標識消息在二進制格式中的位置。值從1到15採用1個字節編碼。
值從16到2047採用兩個字節編碼,最小的值是1,最大的值是2的29次方減1,另外19000到19999你也不可以使用,這些是保留值。
1 message SearchRequest { 2 string query = 1; 3 int32 page_number = 2; 4 int32 result_per_page = 3; 5 } 6 7 message SearchResponse { 8 ... 9 }
另外,多個消息類型是可以定義在一個以 .proto 爲後綴的文件的,如果你想要添加註釋的話,同註釋代碼一樣。
值得注意的是,如果您要修改消息類型,比如刪除字段,或者註釋字段,那麼以後有可能有人在你刪除字段的位置添加了一個新字段,並使用原刪除字段相同的數值編號。如果此時有別的服務還在使用老版本的話,那麼會導致數據被破壞。此時你可以使用 reserved 關鍵字來預佔編號或者預佔字段名稱,如下所示:
message Foo { reserved 2, 15, 9 to 11, 40 to max; reserved "foo", "bar"; }
另外,在同一個 reserved 語句中不能同時預佔編號和字段名稱。
2、Protocol Buffers 同樣支持枚舉與結構嵌套。
enum Corpus { CORPUS_UNSPECIFIED = 0; CORPUS_UNIVERSAL = 1; CORPUS_WEB = 2; CORPUS_IMAGES = 3; CORPUS_LOCAL = 4; CORPUS_NEWS = 5; CORPUS_PRODUCTS = 6; CORPUS_VIDEO = 7; } message SearchRequest { string query = 1; int32 page_number = 2; int32 result_per_page = 3; Corpus corpus = 4; }
枚舉的第一個值爲0,另外也支持鍵值對類型 map<key_type, value_type> map_field = N;
3、再來看另外一個語句
message SearchResponse { repeated Result results = 1; } message Result { optional string url = 1; string title = 2; repeated string snippets = 3; }
optional 關鍵字
字面意思是可選的意思,具體protobuf裏面怎麼處理這個字段呢,就是protobuf處理的時候另外加了一個bool的變量,用來標記這個optional字段是否有值,發送方在發送的時候,如果這個字段有值,那麼就給bool變量標記爲true,否則就標記爲false,接收方在收到這個字段的同時,也會收到發送方同時發送的bool變量,拿着bool變量就知道這個字段是否有值了,這就是option的意思。
repeated 關鍵字
字面意思大概是重複的意思,其實protobuf處理這個字段的時候,也是optional字段一樣,另外加了一個count計數變量,用於標明這個字段有多少個,這樣發送方發送的時候,同時發送了count計數變量和這個字段的起始地址,接收方在接受到數據之後,按照count來解析對應的數據即可。
4、導入
如果你想導入別的 proto 文件裏的消息類型,同樣也可以使用 import 導入:
import "myproject/other_protos.proto";
默認情況下,只能使用直接導入的 .proto 文件中的定義。然而,有時您可能需要將 .proto 文件移動到新的位置。您可以在舊位置放置一個佔位符的.proto文件,使用 import public 將所有導入轉發到新位置,而不是直接移動 .proto 文件並在一次更改中更新所有調用站點。
5、包
您可以向.proto文件添加可選的包說明符,以防止協議消息類型之間的名稱衝突。
package foo.bar; message Open { ... }
message Foo { ... foo.bar.Open open = 1; ... }
爲了生成Go代碼,可以爲提供編譯後的輸出路徑。
option go_package = "example.com/project/protos/fizz";
二、定義服務
如果您想在RPC(遠程過程調用)系統中使用您的消息類型,您可以在 .proto 文件中定義RPC服務接口,protocol buffer compiler將用您選擇的語言生成服務接口代碼。因此,如果你想用一個方法定義一個RPC服務,它接受你的SearchRequest並返回一個SearchResponse,你可以在你的 .proto 文件中定義它,如下所示:
service SearchService {
rpc Search(SearchRequest) returns (SearchResponse);
}
三、編譯 protocol buffer
現在運行編譯器,指定源目錄(應用程序源代碼所在的目錄-如果不提供值,則使用當前目錄)和目標目錄(您希望生成的代碼存放的目錄;通常與$SRC_DIR相同),以及你的 .proto 的路徑。在這種情況下,您將調用:
protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto
假如您輸入以下命令進行編譯,並且 proto 文件使用了 option go_package = "/path2";
protoc --go_out=./path1 ./test.proto
那麼輸出目錄就在當前目錄的 /path1/paht2 目錄下。
舉個例子:
syntax = "proto3"; package test; option go_package = "order/service"; message SearchRequest { string requestParam1 = 1; string requestParam2 = 2; int32 requestParam3 = 3; } message SearchResponse { int32 code = 1; string msg = 2; } service SearchService { rpc SearchOrder(SearchRequest) returns (SearchResponse); }
使用如下命令進行編譯:
protoc --go_out=./gen ./test.proto
編譯之後的文件在 ./gen/order/service/ 目錄下,文件名爲 test.pb.go:
// Code generated by protoc-gen-go. DO NOT EDIT. // versions: // protoc-gen-go v1.28.1 // protoc v4.22.0 // source: test.proto package service import ( protoreflect "google.golang.org/protobuf/reflect/protoreflect" protoimpl "google.golang.org/protobuf/runtime/protoimpl" reflect "reflect" sync "sync" ) const ( // Verify that this generated code is sufficiently up-to-date. _ = protoimpl.EnforceVersion(20 - protoimpl.MinVersion) // Verify that runtime/protoimpl is sufficiently up-to-date. _ = protoimpl.EnforceVersion(protoimpl.MaxVersion - 20) ) type SearchRequest struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields RequestParam1 string `protobuf:"bytes,1,opt,name=requestParam1,proto3" json:"requestParam1,omitempty"` RequestParam2 string `protobuf:"bytes,2,opt,name=requestParam2,proto3" json:"requestParam2,omitempty"` RequestParam3 int32 `protobuf:"varint,3,opt,name=requestParam3,proto3" json:"requestParam3,omitempty"` } func (x *SearchRequest) Reset() { *x = SearchRequest{} if protoimpl.UnsafeEnabled { mi := &file_test_proto_msgTypes[0] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } } func (x *SearchRequest) String() string { return protoimpl.X.MessageStringOf(x) } func (*SearchRequest) ProtoMessage() {} func (x *SearchRequest) ProtoReflect() protoreflect.Message { mi := &file_test_proto_msgTypes[0] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) } return ms } return mi.MessageOf(x) } // Deprecated: Use SearchRequest.ProtoReflect.Descriptor instead. func (*SearchRequest) Descriptor() ([]byte, []int) { return file_test_proto_rawDescGZIP(), []int{0} } func (x *SearchRequest) GetRequestParam1() string { if x != nil { return x.RequestParam1 } return "" } func (x *SearchRequest) GetRequestParam2() string { if x != nil { return x.RequestParam2 } return "" } func (x *SearchRequest) GetRequestParam3() int32 { if x != nil { return x.RequestParam3 } return 0 } type SearchResponse struct { state protoimpl.MessageState sizeCache protoimpl.SizeCache unknownFields protoimpl.UnknownFields Code int32 `protobuf:"varint,1,opt,name=code,proto3" json:"code,omitempty"` Msg string `protobuf:"bytes,2,opt,name=msg,proto3" json:"msg,omitempty"` } func (x *SearchResponse) Reset() { *x = SearchResponse{} if protoimpl.UnsafeEnabled { mi := &file_test_proto_msgTypes[1] ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) ms.StoreMessageInfo(mi) } } func (x *SearchResponse) String() string { return protoimpl.X.MessageStringOf(x) } func (*SearchResponse) ProtoMessage() {} func (x *SearchResponse) ProtoReflect() protoreflect.Message { mi := &file_test_proto_msgTypes[1] if protoimpl.UnsafeEnabled && x != nil { ms := protoimpl.X.MessageStateOf(protoimpl.Pointer(x)) if ms.LoadMessageInfo() == nil { ms.StoreMessageInfo(mi) } return ms } return mi.MessageOf(x) } // Deprecated: Use SearchResponse.ProtoReflect.Descriptor instead. func (*SearchResponse) Descriptor() ([]byte, []int) { return file_test_proto_rawDescGZIP(), []int{1} } func (x *SearchResponse) GetCode() int32 { if x != nil { return x.Code } return 0 } func (x *SearchResponse) GetMsg() string { if x != nil { return x.Msg } return "" } var File_test_proto protoreflect.FileDescriptor var file_test_proto_rawDesc = []byte{ 0x0a, 0x0a, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x12, 0x04, 0x74, 0x65, 0x73, 0x74, 0x22, 0x81, 0x01, 0x0a, 0x0d, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x12, 0x24, 0x0a, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x31, 0x18, 0x01, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x31, 0x12, 0x24, 0x0a, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x32, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x32, 0x12, 0x24, 0x0a, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x33, 0x18, 0x03, 0x20, 0x01, 0x28, 0x05, 0x52, 0x0d, 0x72, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x50, 0x61, 0x72, 0x61, 0x6d, 0x33, 0x22, 0x36, 0x0a, 0x0e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x12, 0x12, 0x0a, 0x04, 0x63, 0x6f, 0x64, 0x65, 0x18, 0x01, 0x20, 0x01, 0x28, 0x05, 0x52, 0x04, 0x63, 0x6f, 0x64, 0x65, 0x12, 0x10, 0x0a, 0x03, 0x6d, 0x73, 0x67, 0x18, 0x02, 0x20, 0x01, 0x28, 0x09, 0x52, 0x03, 0x6d, 0x73, 0x67, 0x32, 0x49, 0x0a, 0x0d, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x53, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x12, 0x38, 0x0a, 0x0b, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x4f, 0x72, 0x64, 0x65, 0x72, 0x12, 0x13, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x71, 0x75, 0x65, 0x73, 0x74, 0x1a, 0x14, 0x2e, 0x74, 0x65, 0x73, 0x74, 0x2e, 0x53, 0x65, 0x61, 0x72, 0x63, 0x68, 0x52, 0x65, 0x73, 0x70, 0x6f, 0x6e, 0x73, 0x65, 0x42, 0x0f, 0x5a, 0x0d, 0x6f, 0x72, 0x64, 0x65, 0x72, 0x2f, 0x73, 0x65, 0x72, 0x76, 0x69, 0x63, 0x65, 0x62, 0x06, 0x70, 0x72, 0x6f, 0x74, 0x6f, 0x33, } var ( file_test_proto_rawDescOnce sync.Once file_test_proto_rawDescData = file_test_proto_rawDesc ) func file_test_proto_rawDescGZIP() []byte { file_test_proto_rawDescOnce.Do(func() { file_test_proto_rawDescData = protoimpl.X.CompressGZIP(file_test_proto_rawDescData) }) return file_test_proto_rawDescData } var file_test_proto_msgTypes = make([]protoimpl.MessageInfo, 2) var file_test_proto_goTypes = []interface{}{ (*SearchRequest)(nil), // 0: test.SearchRequest (*SearchResponse)(nil), // 1: test.SearchResponse } var file_test_proto_depIdxs = []int32{ 0, // 0: test.SearchService.SearchOrder:input_type -> test.SearchRequest 1, // 1: test.SearchService.SearchOrder:output_type -> test.SearchResponse 1, // [1:2] is the sub-list for method output_type 0, // [0:1] is the sub-list for method input_type 0, // [0:0] is the sub-list for extension type_name 0, // [0:0] is the sub-list for extension extendee 0, // [0:0] is the sub-list for field type_name } func init() { file_test_proto_init() } func file_test_proto_init() { if File_test_proto != nil { return } if !protoimpl.UnsafeEnabled { file_test_proto_msgTypes[0].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*SearchRequest); i { case 0: return &v.state case 1: return &v.sizeCache case 2: return &v.unknownFields default: return nil } } file_test_proto_msgTypes[1].Exporter = func(v interface{}, i int) interface{} { switch v := v.(*SearchResponse); i { case 0: return &v.state case 1: return &v.sizeCache case 2: return &v.unknownFields default: return nil } } } type x struct{} out := protoimpl.TypeBuilder{ File: protoimpl.DescBuilder{ GoPackagePath: reflect.TypeOf(x{}).PkgPath(), RawDescriptor: file_test_proto_rawDesc, NumEnums: 0, NumMessages: 2, NumExtensions: 0, NumServices: 1, }, GoTypes: file_test_proto_goTypes, DependencyIndexes: file_test_proto_depIdxs, MessageInfos: file_test_proto_msgTypes, }.Build() File_test_proto = out.File file_test_proto_rawDesc = nil file_test_proto_goTypes = nil file_test_proto_depIdxs = nil }
如果你要使用的消息結構的話,可以看如下示例:
package main import ( "fmt" "github.com/golang/protobuf/proto" pb "study.com/study-user/api/proto/gen/order/service" ) func main() { request := &pb.SearchRequest{ RequestParam1: "aaa", RequestParam2: "bbb", RequestParam3: 0, } fmt.Printf("編碼前: %v \n", request) marshal, err := proto.Marshal(request) fmt.Printf("編碼後: %v \n", marshal) if err == nil { newRequest := &pb.SearchRequest{} proto.Unmarshal(marshal, newRequest) fmt.Printf("解碼後: %v \n", newRequest) } }
細心的你可能發現,我沒有調用函數,畢竟也沒有定義函數,你就算用 pb. 後面也找不到函數名,可以先看看gRPC,這裏會告訴你答案。
四、gRPC介紹
當你來到gRPC官網Go語言時,他會要求你提前準備如下:
1、安裝 Go
3、安裝編譯插件
$ go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28 $ go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2
4、配置 Protocol Buffer 環境變量
$ export PATH="$PATH:$(go env GOPATH)/bin"
5、下載測試用例
git clone -b v1.53.0 --depth 1 https://github.com/grpc/grpc-go
6、運行示例
go run greeter_server/main.go
go run greeter_client/main.go
6、生成 gRPC代碼
$ protoc --go_out=. --go_opt=paths=source_relative \ --go-grpc_out=. --go-grpc_opt=paths=source_relative \ helloworld/helloworld.proto
這個我們拿之前的例子來舉例,go_opt 選項你可以在源文件中使用 option go_package 來替代。
protoc --go_out=./gen --go-grpc_out=./gen ./test.proto
這時可以看到生成了兩個文件:test.pb.go、test_grpc.pb.go,我們來看一下生成的 grpc 代碼:
// Code generated by protoc-gen-go-grpc. DO NOT EDIT. // versions: // - protoc-gen-go-grpc v1.2.0 // - protoc v4.22.0 // source: test.proto package service import ( context "context" grpc "google.golang.org/grpc" codes "google.golang.org/grpc/codes" status "google.golang.org/grpc/status" ) // This is a compile-time assertion to ensure that this generated file // is compatible with the grpc package it is being compiled against. // Requires gRPC-Go v1.32.0 or later. const _ = grpc.SupportPackageIsVersion7 // SearchServiceClient is the client API for SearchService service. // // For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream. type SearchServiceClient interface { SearchOrder(ctx context.Context, in *SearchRequest, opts ...grpc.CallOption) (*SearchResponse, error) } type searchServiceClient struct { cc grpc.ClientConnInterface } func NewSearchServiceClient(cc grpc.ClientConnInterface) SearchServiceClient { return &searchServiceClient{cc} } func (c *searchServiceClient) SearchOrder(ctx context.Context, in *SearchRequest, opts ...grpc.CallOption) (*SearchResponse, error) { out := new(SearchResponse) err := c.cc.Invoke(ctx, "/test.SearchService/SearchOrder", in, out, opts...) if err != nil { return nil, err } return out, nil } // SearchServiceServer is the server API for SearchService service. // All implementations must embed UnimplementedSearchServiceServer // for forward compatibility type SearchServiceServer interface { SearchOrder(context.Context, *SearchRequest) (*SearchResponse, error) mustEmbedUnimplementedSearchServiceServer() } // UnimplementedSearchServiceServer must be embedded to have forward compatible implementations. type UnimplementedSearchServiceServer struct { } func (UnimplementedSearchServiceServer) SearchOrder(context.Context, *SearchRequest) (*SearchResponse, error) { return nil, status.Errorf(codes.Unimplemented, "method SearchOrder not implemented") } func (UnimplementedSearchServiceServer) mustEmbedUnimplementedSearchServiceServer() {} // UnsafeSearchServiceServer may be embedded to opt out of forward compatibility for this service. // Use of this interface is not recommended, as added methods to SearchServiceServer will // result in compilation errors. type UnsafeSearchServiceServer interface { mustEmbedUnimplementedSearchServiceServer() } func RegisterSearchServiceServer(s grpc.ServiceRegistrar, srv SearchServiceServer) { s.RegisterService(&SearchService_ServiceDesc, srv) } func _SearchService_SearchOrder_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { in := new(SearchRequest) if err := dec(in); err != nil { return nil, err } if interceptor == nil { return srv.(SearchServiceServer).SearchOrder(ctx, in) } info := &grpc.UnaryServerInfo{ Server: srv, FullMethod: "/test.SearchService/SearchOrder", } handler := func(ctx context.Context, req interface{}) (interface{}, error) { return srv.(SearchServiceServer).SearchOrder(ctx, req.(*SearchRequest)) } return interceptor(ctx, in, info, handler) } // SearchService_ServiceDesc is the grpc.ServiceDesc for SearchService service. // It's only intended for direct use with grpc.RegisterService, // and not to be introspected or modified (even as a copy) var SearchService_ServiceDesc = grpc.ServiceDesc{ ServiceName: "test.SearchService", HandlerType: (*SearchServiceServer)(nil), Methods: []grpc.MethodDesc{ { MethodName: "SearchOrder", Handler: _SearchService_SearchOrder_Handler, }, }, Streams: []grpc.StreamDesc{}, Metadata: "test.proto", }
我們重點關注如下:
// SearchServiceServer is the server API for SearchService service. // All implementations must embed UnimplementedSearchServiceServer // for forward compatibility type SearchServiceServer interface { SearchOrder(context.Context, *SearchRequest) (*SearchResponse, error) mustEmbedUnimplementedSearchServiceServer() }
SearchServiceServer 是 SearchService 服務的 api,所有的實現必須組合 mustEmbedUnimplementedSearchServiceServer 爲了向前兼容。
7、官方helloworld使用示例
proto 文件:
syntax = "proto3"; option go_package = "google.golang.org/grpc/examples/helloworld/helloworld"; option java_multiple_files = true; option java_package = "io.grpc.examples.helloworld"; option java_outer_classname = "HelloWorldProto"; package helloworld; // The greeting service definition. service Greeter { // Sends a greeting rpc SayHello (HelloRequest) returns (HelloReply) {} } // The request message containing the user's name. message HelloRequest { string name = 1; } // The response message containing the greetings message HelloReply { string message = 1; }
服務端:
package main import ( "context" "flag" "fmt" "log" "net" "google.golang.org/grpc" pb "google.golang.org/grpc/examples/helloworld/helloworld" ) var ( port = flag.Int("port", 50051, "The server port") ) // server is used to implement helloworld.GreeterServer. type server struct { pb.UnimplementedGreeterServer } // SayHello implements helloworld.GreeterServer func (s *server) SayHello(ctx context.Context, in *pb.HelloRequest) (*pb.HelloReply, error) { log.Printf("Received: %v", in.GetName()) return &pb.HelloReply{Message: "Hello " + in.GetName()}, nil } func main() { flag.Parse() lis, err := net.Listen("tcp", fmt.Sprintf(":%d", *port)) if err != nil { log.Fatalf("failed to listen: %v", err) } s := grpc.NewServer() pb.RegisterGreeterServer(s, &server{}) log.Printf("server listening at %v", lis.Addr()) if err := s.Serve(lis); err != nil { log.Fatalf("failed to serve: %v", err) } }
官方使用 server 組合了 pb.UnimplementedGreeterServer,然後 server 實現了 SayHello 的方法。
在 main 方法裏對某個端口號進行監聽,並註冊了 SayHello 服務,因爲只有一個 rpc 方法,然後服務端進行服務。
客戶端:
package main import ( "context" "flag" "log" "time" "google.golang.org/grpc" "google.golang.org/grpc/credentials/insecure" pb "google.golang.org/grpc/examples/helloworld/helloworld" ) const ( defaultName = "world" ) var ( addr = flag.String("addr", "localhost:50051", "the address to connect to") name = flag.String("name", defaultName, "Name to greet") ) func main() { flag.Parse() // Set up a connection to the server. conn, err := grpc.Dial(*addr, grpc.WithTransportCredentials(insecure.NewCredentials())) if err != nil { log.Fatalf("did not connect: %v", err) } defer conn.Close() c := pb.NewGreeterClient(conn) // Contact the server and print out its response. ctx, cancel := context.WithTimeout(context.Background(), time.Second) defer cancel() r, err := c.SayHello(ctx, &pb.HelloRequest{Name: *name}) if err != nil { log.Fatalf("could not greet: %v", err) } log.Printf("Greeting: %s", r.GetMessage()) }
客戶端連接服務端gRPC服務地址,調用服務端的 SayHello 方法,拿到結果後返回。
如果想要詳細瞭解的話,可以看看 gRPC 官方文檔哈,如果有幫助可以幫忙點個贊哈!