buffer-slayer源碼說明

/**
 * @author 
 * @Description buffer-slayer 的實現類
 * @date 
 */
@Slf4j
@Service
public class BufferSlayerService {
    private static final String INSERTION = "INSERT INTO test(data, time) VALUES(?, ?);";

    static String randomString() {
        return String.valueOf(ThreadLocalRandom.current().nextLong());
    }
    @Resource
    private JdbcTemplate jdbcTemplate;

    public void doBufferSlayer(){

        AsyncReporterProperties reporterProperties = new AsyncReporterProperties()
                .setFlushThreads(5)
                .setSharedSenderThreads(10)
                .setBufferedMaxMessages(500)
                .setPendingMaxMessages(10000)
                .setMetrics("inmemory")
                .setMetricsExporter("http");

        BatchJdbcTemplate template = new BatchJdbcTemplate(jdbcTemplate, reporterProperties);

        MessageFuture<Integer> future = template.update(INSERTION, randomString(), new Date());
        future.addListener(f -> {
            log.info("返回值：" + f.get());
        });
        /*future.addListener(new GenericFutureListener<Future<? super Integer>>() {
            @Override
            public void operationComplete(Future<? super Integer> future) throws Exception {
                log.info("返回值：" + future.get());
            }
        });*/
    }
}

首先調用BatchJdbcTemplate的構造方法。
之後調用update方法

public MessageFuture<Integer> update(String sql, Object... args) throws DataAccessException {
    return reporter.report(Sql.builder()
        .sql(sql)
        .args(args)
        .build());
  }

看到了reporter，是buffer-slayer的組件之一
reporter是一個接口，實現他的是AsyncReporter

public class AsyncReporter<M extends Message, R> extends TimeDriven<MessageKey> implements Reporter<M, R>, Flushable {
....
}

AsyncReporter中report方法。

@Override
  @SuppressWarnings("unchecked")
  public MessageFuture<R> report(M message) {
    checkNotNull(message, "message");
    metrics.incrementMessages(1);

    if (REPORTER_STATE_UPDATER.get(this) == REPORTER_STATE_SHUTDOWN) { // Drop the message when closed.
      MessageDroppedException cause =
          dropped(new IllegalStateException("closed!"), singletonList(message));
      MessageFuture<R> future = (MessageFuture<R>) message.newFailedFuture(cause);
      setFailListener(future);
      return future;
    }

    // Lazy initialize flush threads
    if (REPORTER_STATE_UPDATER.compareAndSet(this, REPORTER_STATE_INIT, REPORTER_STATE_STARTED) &&
        messageTimeoutNanos > 0) {
      startFlushThreads();
    }
    //隊列衝的記錄，如果達到限制，則阻止當前線程，直到發出信號
    memoryLimiter.waitWhenMaximum();
    // If singleKey is true, ignore original message key. 默認是false
    Message.MessageKey key = singleKey ?
        Message.SINGLE_KEY : message.asMessageKey();

    // Offer message to pending queue.  向掛起隊列提供消息。
    AbstractSizeBoundedQueue pending = queueManager.getOrCreate(key);// key 即隊列的名字，如沒有隊列則新建一個隊列，有的話返回
    MessagePromise<R> promise = message.newPromise();
    pending.offer(promise);//如果元素可以添加，則通知延遲；如果由於其他原因拒絕
    setFailListener(promise);//失敗的監聽

    if (pending.size() >= bufferedMaxMessages)//大於一次提交的最大數量的時候
      synchronizer.offer(pending);
    return promise;
  }

getOrCreate方法是得到一個隊列，key就是隊列的名字。之後調用pending.offer(promise);將執行的sql語句加入到隊列中。
到這裏就要拋出問題了，sql語句加入對了是怎麼執行的呢？（大神們都說看源碼要帶着問題去看。）
請看下面：
AsyncReporter的構造方法

AsyncReporter(Builder<M, R> builder) {
    this.sender = toAsyncSender(builder);
    this.metrics = builder.metrics;
    this.memoryLimiter = MemoryLimiter.maxOf(builder.totalQueuedMessages, metrics);
    this.messageTimeoutNanos = builder.messageTimeoutNanos;
    this.bufferedMaxMessages = builder.bufferedMaxMessages;
    this.singleKey = builder.singleKey;

    this.flushThreads = builder.flushThreads;
    this.timerThreads = builder.timerThreads;

    this.queueManager =
        new QueueManager(builder.pendingMaxMessages, builder.overflowStrategy,
            builder.pendingKeepaliveNanos, this,
            metrics, initHashedWheelTimer(builder));

    this.flushThreadFactory = new FlushThreadFactory(this);
    this.bufferPool = new BufferPool(MAX_BUFFER_POOL_ENTRIES,
        builder.bufferedMaxMessages, builder.singleKey);

    if (messageTimeoutNanos > 0) {
      this.queueManager.onCreate(new CreateCallback() {
        @Override
        public void call(AbstractSizeBoundedQueue queue) {
          schedulePeriodically(queue.key, messageTimeoutNanos);
        }
      });
    }
  }

這裏注意構造方法中的最後一個方法，用了QueueManager 組件

接下來看看QueueManager 中onCreate中實現的方法。
這是一個抽象類中的方法

  void schedulePeriodically(final K timerKey, long intervalNanos) {
    cancelTimer(timerKey);
    long id = timerIdGen.getAndIncrement();
    ScheduledFuture<?> task = scheduler().scheduleWithFixedDelay(new Runnable() {
      @Override
      public void run() {
        onTimer(timerKey);
      }
    }, intervalNanos, intervalNanos, TimeUnit.NANOSECONDS);
    keyToTimer.put(timerKey, new Timer(id, task));
  }

這裏需要注意一個類ScheduledFuture
這裏使用了ScheduledExecutorService 延遲/週期執行線程池

繼續回到方法，上面的方法調用onTimer，onTimer是一個抽象方法。
查看實現

可以看到AsyncReporter實現了這個抽象類中的方法

 @Override
  protected void onTimer(MessageKey timerKey) {
    AbstractSizeBoundedQueue q = queueManager.get(timerKey);
    if (q != null && q.size() > 0) flush(q);
  }

可以看到又一大組件出現：SizeBoundedQueue 。這裏是取隊列中值，然後刷新出去執行。

@Override
  public void flush() {
    for (AbstractSizeBoundedQueue pending : queueManager.elements()) {
      Future<?> future = flush(pending);
      future.awaitUninterruptibly();
    }
  }

  Future<?> flush(AbstractSizeBoundedQueue pending) {
    CompositeFuture result;
    Buffer buffer = bufferPool.acquire();
    try {
      int drained = pending.drainTo(buffer);
      if (drained == 0) {
        return new SucceededFuture<>(null, null);
      }

      List<MessagePromise<R>> promises = buffer.drain();
      result = sender.send(promises);
    } finally {
      bufferPool.release(buffer);
    }
    // Update metrics
    metrics.updateQueuedMessages(pending.key, pending.size());
    // Signal producers
    if (!memoryLimiter.isMaximum())
      memoryLimiter.signalAll();

    logWhenFailed(result);
    return result;
  }

這裏咱們看到了又一大組件Sender -批量發送緩衝區消息。調用send方法
AsyncSenderAdaptor

@Override
  public CompositeFuture send(final List<MessagePromise<R>> promises) {
    logger.debug("Sending {} messages.", promises.size());
    final List<M> messages = extractMessages(promises);

    executor.execute(new Runnable() {
      @Override
      public void run() {
        try {
          List<R> result = delegate.send(messages);
          Promises.allSuccess(result, promises);
        } catch (Throwable t) {
          Promises.allFail(t, promises, messages);
        }
      }
    });
    return CompositeFuture.all(promises);
  }

JdbcTemplateSender

@Override
  public List<Integer> send(List<Sql> sqls) {
    Preconditions.checkArgument(sqls != null && sqls.size() > 0, "Sqls must not be empty.");
    if (!ofTheSameSql(sqls)) {
      throw new UnsupportedOperationException("Different sqls not supported");
    }

    boolean prepared = allPreparedStatement(sqls);
    int[] rowsAffected;
    if (prepared) {
      rowsAffected = underlying.batchUpdate(sqls.get(0).sql, batchPreparedStatementSetter(sqls));
    } else {
      rowsAffected = underlying.batchUpdate(extractSqls(sqls).toArray(new String[0]));
    }

    List<Integer> ret = new ArrayList<>(rowsAffected.length);
    for (int aRowsAffected : rowsAffected) {
      ret.add(aRowsAffected);
    }
    return ret;
  }

在這裏執行了sql語句。

到此sql語句執行了，返回結果。

這裏 AsyncReporter(Builder<M, R> builder) 中初始化 queueManager用到了哈希輪盤算法

this.queueManager =
        new QueueManager(builder.pendingMaxMessages, builder.overflowStrategy,
            builder.pendingKeepaliveNanos, this,
            metrics, initHashedWheelTimer(builder));

 private static HashedWheelTimer initHashedWheelTimer(Builder<?, ?> builder) {
    synchronized (AsyncReporter.class) {
      if (hashedWheelTimer == null) {
        hashedWheelTimer = new HashedWheelTimer(
            hashedWheelTimerThreadFactory,
            builder.tickDurationNanos, TimeUnit.NANOSECONDS,
            builder.ticksPerWheel);
      }
    }
    return hashedWheelTimer;
  }

有興趣的同學可以研究下
歡迎捐贈