asp.net core項目集成OpenTelemetry輸出到OpenObserve進行可觀測性處理

一、安裝OpenObserve和Otel-collector

使用docker-compose 安裝

記得替換下USER_MAIL [email protected]和PASSWORD xxx
這裏解釋下爲啥要用named volume來映射openobserve的data目錄，因爲目前版本(0.7.2)在windows的docker desktop裏使用會有bug，導致stream無法正確寫入文件持久化，一直在生成新的stream文件，而使用named volume則正常，詳情請見issue

openobserve佔用了5080（http）、5081（grpc）兩個端口
otel-collector佔用了5078和5079兩個端口來作爲對外提供服務的入口，後邊的配置會有解釋，其他端口有註釋這裏就不贅述了

新建一個openobserve目錄，新建docker-compose.yaml文件，內容如下

version: '3'
services:
  main:
    image: openobserve/openobserve
    volumes:
      - ob-v:/data
    ports:
      - 5080:5080
      - 5081:5081
    environment:
      - ZO_DATA_DIR=/data
      - [email protected]
      - ZO_ROOT_USER_PASSWORD=xxxx
      - TZ=Asia/Shanghai
    restart: always
  otel-collector:
    image: otel/opentelemetry-collector-contrib
    restart: always
    volumes:
      - ./otel-collector-config.yaml:/etc/otelcol-contrib/config.yaml
    ports:
      - 1888:1888 # pprof extension
      - 8888:8888 # Prometheus metrics exposed by the Collector
      - 8889:8889 # Prometheus exporter metrics
      - 13133:13133 # health_check extension
      - 5078:5078 # OTLP   gRPC receiver
      - 5079:5079 # OTLP/2 gRPC receiver
      #- 55679:55679 # zpages extension

volumes:
  ob-v: {}

otel-collector-config配置

然後在當前目錄新建otel-collector-config.yaml內容如下
指定了兩個端點，5079和5078，讓測試環境和生產環境走不同的流水線處理。otel-collector-config配置參考文檔
記得替換下邊的myapp爲自己的項目名稱

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:5079
  otlp/2:
    protocols:
      grpc:
        endpoint: 0.0.0.0:5078

processors:
  batch:

exporters:
  otlp:
    endpoint: main:5081
    headers:
      Authorization: "Basic xxxxxx=="
      organization: myapp_test
      stream-name: default
    tls:
      insecure: true
  otlp/2:
    endpoint: main:5081
    headers:
      Authorization: "Basic xxxxxx=="
      organization: myapp_production
      stream-name: default
    tls:
      insecure: true

extensions:
  health_check:

service:
  extensions: [health_check]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    metrics:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    logs:
      receivers: [otlp]
      processors: [batch]
      exporters: [otlp]
    traces/2:
      receivers: [otlp/2]
      processors: [batch]
      exporters: [otlp/2]
    metrics/2:
      receivers: [otlp/2]
      processors: [batch]
      exporters: [otlp/2]
    logs/2:
      receivers: [otlp/2]
      processors: [batch]
      exporters: [otlp/2]

然後使用命令行運行docker-compose up -d

瀏覽器訪問 服務器ip:5080, 用戶名和密碼是上邊設置的郵箱和密碼
點擊左側採集--Trace（OpenTelemetry）複製其中的Authorization: 後邊的部分，替換otel-collector-config.yaml中的Basic xxxxxx==

最後執行docker-compose restart重啓容器完成配置

二、asp.net core項目增加OpenTelemetry導出

引用OpenTelemetry Nuget包

在web api項目的csproj文件中增加以下nuget包引用

    <PackageReference Include="OpenTelemetry.Exporter.Console" Version="1.7.0" />
    <PackageReference Include="OpenTelemetry.Exporter.OpenTelemetryProtocol" Version="1.7.0" />
    <PackageReference Include="OpenTelemetry.Extensions.Hosting" Version="1.7.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.AspNetCore" Version="1.7.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.EntityFrameworkCore" Version="1.0.0-beta.8" />
    <PackageReference Include="OpenTelemetry.Instrumentation.Http" Version="1.7.0" />
    <PackageReference Include="OpenTelemetry.Instrumentation.Runtime" Version="1.5.1" />

在appsettings.json中配置啓用開關

  "OpenTelemetry": {
    "ServiceName": "UserService",
    "EndPoint": "http://192.168.1.101:5079",
    "Enable": true
  }

其中ServiceName是該web api項目名稱，EndPoint是otel-collector的一個grpc端點
比如測試環境的項目使用了5079端口，那生成環境就改爲5078端口，如果還有演示環境、預發佈環境之類的就繼續在配置裏增加端口

代碼中配置啓用Logging Metrics Traces

配置Logging導出

由於ConfigureLogging方法是IHostBuilder的擴展方法，所以只能在CreateDefaultBuilder這裏調用

Host.CreateDefaultBuilder(args)
            .ConfigureLogging((c,l) =>
            {
                if (bool.TryParse(c.Configuration["OpenTelemetry:Enable"], out var e) && e)
                {
                    l.AddOpenTelemetry(logging =>
                    logging.SetResourceBuilder(ResourceBuilder.CreateDefault().AddService(c.Configuration["OpenTelemetry:ServiceName"]))
                        .AddOtlpExporter(o =>
                        {
                            o.Endpoint = new Uri(c.Configuration["OpenTelemetry:Endpoint"]);
                        }));
                }
            })

配置 Tracing和Metrics導出

在有IServiceCollection的地方執行AddOpenTelemetry擴展方法

if (bool.TryParse(_appConfiguration["OpenTelemetry:Enable"], out var otelEnable) && otelEnable)
            {
                services
                    .AddOpenTelemetry()
                    .ConfigureResource(builder => builder
                        .AddService(serviceName: _appConfiguration["OpenTelemetry:ServiceName"], serviceInstanceId: Environment.MachineName))
                    .WithTracing(builder => builder
                        .SetSampler(new AlwaysOnSampler())
                        .AddEntityFrameworkCoreInstrumentation(options => options.SetDbStatementForText = true)
                        .AddHttpClientInstrumentation()
                        .AddAspNetCoreInstrumentation(options =>
                        {
                            options.RecordException = true;
                        })
                        .AddOtlpExporter(o =>
                        {
                            o.Endpoint = new Uri(_appConfiguration["OpenTelemetry:Endpoint"]);
                        })
                    )

                    .WithMetrics(builder => builder
                        .AddRuntimeInstrumentation()
                        .AddAspNetCoreInstrumentation()
                        .AddOtlpExporter(o =>
                        {
                            o.Endpoint = new Uri(_appConfiguration["OpenTelemetry:Endpoint"]);
                        })
                    );
            }

最後項目運行起來在OpenObserve的數據流中有對應數據就算成功了