微服務全鏈路跟蹤：jaeger集成hystrix

微服務全鏈路跟蹤：grpc集成zipkin

微服務全鏈路跟蹤：grpc集成jaeger

微服務全鏈路跟蹤：springcloud集成jaeger

微服務全鏈路跟蹤：jaeger集成istio，併兼容uber-trace-id與b3

微服務全鏈路跟蹤：jaeger集成hystrix

背景

當springcloud服務集成hystrix，並且用了hystrixCommend註解到方法上時，jaeger鏈路會斷掉

方案

在網上搜索到了大量jaeger遇到多線程時的處理方式，都是包裝線程池來做到ThreadLocal傳遞，有很多都用到了阿里開源的transmittable-thread-local。

下面說一下當集成hystrix時，jaeger鏈路丟失問題，大家都知道hystrix默認是線程池隔離，所以歸根結底還是遇到多線程線程變量沒有共享的問題，網上也羅列了幾種方案：

方案一：變更隔離方式

hystrix.command.default.execution.isolation.strategy: SEMAPHORE

當併發高時這裏設置信號量隔離是有風險的，可以根據情況優化斷路器配置來降低風險

方案二：自定義隔離策略

隔離策略官方文檔有定義：

原先我就定義了一個feign傳遞request中header信息的策略，在原有的隔離策略下面參考https://github.com/opentracing-contrib/java-concurrent項目的代碼：

將原有feign自定義隔離策略做了響應變動，代碼如下

/**
 * 自定義Feign的隔離策略:
 *   在轉發Feign的請求頭的時候, 如果開啓了Hystrix, 
 *      Hystrix的默認隔離策略是Thread(線程隔離策略), 因此轉發攔截器內是無法獲取到請求的請求頭信息的, 
 *      可以修改默認隔離策略爲信號量模式：hystrix.command.default.execution.isolation.strategy=SEMAPHORE, 
 *      這樣的話轉發線程和請求線程實際上是一個線程, 這並不是最好的解決方法, 信號量模式也不是官方最爲推薦的隔離策略;
 *   另一個解決方法就是自定義Hystrix的隔離策略:
 *      思路是將現有的併發策略作爲新併發策略的成員變量,在新併發策略中, 
 *      返回現有併發策略的線程池、Queue;將策略加到Spring容器即可;
 */
@Component
@Slf4j
public class FeignHystrixConcurrencyStrategy extends HystrixConcurrencyStrategy {
 
    private HystrixConcurrencyStrategy delegate;
    @Lazy
    @Autowired
    private Tracer tracer;
 
    public FeignHystrixConcurrencyStrategy() {
        try {
            this.delegate = HystrixPlugins.getInstance().getConcurrencyStrategy();
            if (this.delegate instanceof FeignHystrixConcurrencyStrategy) {
                // Welcome to singleton hell...
                return;
            }
 
            HystrixCommandExecutionHook commandExecutionHook =
                    HystrixPlugins.getInstance().getCommandExecutionHook();
 
            HystrixEventNotifier eventNotifier = HystrixPlugins.getInstance().getEventNotifier();
            HystrixMetricsPublisher metricsPublisher = HystrixPlugins.getInstance().getMetricsPublisher();
            HystrixPropertiesStrategy propertiesStrategy =
                    HystrixPlugins.getInstance().getPropertiesStrategy();
            this.logCurrentStateOfHystrixPlugins(eventNotifier, metricsPublisher, propertiesStrategy);
 
            HystrixPlugins.reset();
            HystrixPlugins instance = HystrixPlugins.getInstance();
            instance.registerConcurrencyStrategy(this);
            instance.registerCommandExecutionHook(commandExecutionHook);
            instance.registerEventNotifier(eventNotifier);
            instance.registerMetricsPublisher(metricsPublisher);
            instance.registerPropertiesStrategy(propertiesStrategy);
        } catch (Exception e) {
            log.error("Failed to register Sleuth Hystrix Concurrency Strategy", e);
        }
    }
 
    private void logCurrentStateOfHystrixPlugins(HystrixEventNotifier eventNotifier,
                                                 HystrixMetricsPublisher metricsPublisher,
                                                 HystrixPropertiesStrategy propertiesStrategy) {
        if (log.isDebugEnabled()) {
            log.debug("Current Hystrix plugins configuration is [" + "concurrencyStrategy ["
                    + this.delegate + "]," + "eventNotifier [" + eventNotifier + "]," + "metricPublisher ["
                    + metricsPublisher + "]," + "propertiesStrategy [" + propertiesStrategy + "]," + "]");
            log.debug("Registering Sleuth Hystrix Concurrency Strategy.");
        }
    }
 
    @Override
    public <T> Callable<T> wrapCallable(Callable<T> callable) {
        RequestAttributes requestAttributes = RequestContextHolder.getRequestAttributes();
        return new WrappedCallable<>(callable, requestAttributes,tracer,tracer.activeSpan());
    }
 
    @Override
    public ThreadPoolExecutor getThreadPool(HystrixThreadPoolKey threadPoolKey,
                                            HystrixProperty<Integer> corePoolSize,
                                            HystrixProperty<Integer> maximumPoolSize,
                                            HystrixProperty<Integer> keepAliveTime,
                                            TimeUnit unit, BlockingQueue<Runnable> workQueue) {
        return this.delegate.getThreadPool(threadPoolKey, corePoolSize, maximumPoolSize, keepAliveTime,
                unit, workQueue);
    }
 
    @Override
    public ThreadPoolExecutor getThreadPool(HystrixThreadPoolKey threadPoolKey,
                                            HystrixThreadPoolProperties threadPoolProperties) {
        return this.delegate.getThreadPool(threadPoolKey, threadPoolProperties);
    }
 
    @Override
    public BlockingQueue<Runnable> getBlockingQueue(int maxQueueSize) {
        return this.delegate.getBlockingQueue(maxQueueSize);
    }
 
    @Override
    public <T> HystrixRequestVariable<T> getRequestVariable(HystrixRequestVariableLifecycle<T> rv) {
        return this.delegate.getRequestVariable(rv);
    }

    static class WrappedCallable<T> implements Callable<T> {
        private final Callable<T> target;
        private final RequestAttributes requestAttributes;
        private final Span span;
        private final Tracer tracer;
 
        public WrappedCallable(Callable<T> target, RequestAttributes requestAttributes,Tracer tracer, Span span) {
            this.target = target;
            this.requestAttributes = requestAttributes;
            this.tracer=tracer;
            this.span=span;
        }
 
        @Override
        public T call() throws Exception {
            Scope scope = span == null ? null : tracer.scopeManager().activate(span);
            try {
                RequestContextHolder.setRequestAttributes(requestAttributes);
                return target.call();
            } finally {
                RequestContextHolder.resetRequestAttributes();
                if (scope != null) {
                    scope.close();
                }
            }
        }
    }
}

其中改動部分就是在原有的WrappedCallable中增加了span、trace的注入。

至於自定義隔離策略以及Callable是可以支持多個鏈的，這裏不做詳細描述，大家有興趣可以參考，下面的鏈接：

https://blog.csdn.net/songhaifengshuaige/article/details/80345012

https://www.jianshu.com/p/c60fe209a799

https://www.cnblogs.com/duanxz/p/10949816.html