Netflix Ribbon原理與實踐

Ribbon是什麼？

Ribbon是Netflix發佈的開源項目，主要功能是提供客戶端的軟件負載均衡算法，將Netflix的中間層服務連接在一起。Ribbon客戶端組件提供一系列完善的配置項如連接超時，重試等。簡單的說，就是在配置文件中列出Load Balancer後面所有的機器，Ribbon會自動的幫助你基於某種規則（如簡單輪詢，隨即連接等）去連接這些機器。我們也很容易使用Ribbon實現自定義的負載均衡算法。

Ribbon提供的主要負載均衡策略介紹

簡單輪詢負載均衡（RoundRobin）

以輪詢的方式依次將請求調度不同的服務器，即每次調度執行i = (i + 1) mod n，並選出第i臺服務器。

    Server server = null;
    int count = 0;
    while (server == null && count++ < 10) {
        List<Server> reachableServers = lb.getReachableServers();
        List<Server> allServers = lb.getAllServers();
        int upCount = reachableServers.size();
        int serverCount = allServers.size();

        if ((upCount == 0) || (serverCount == 0)) {
            log.warn("No up servers available from load balancer: " + lb);
            return null;
        }

        int nextServerIndex = incrementAndGetModulo(serverCount);
        server = allServers.get(nextServerIndex);

        if (server == null) {
            /* Transient. */
            Thread.yield();
            continue;
        }

        if (server.isAlive() && (server.isReadyToServe())) {
            return (server);
        }

        // Next.
        server = null;
    }

    if (count >= 10) {
        log.warn("No available alive servers after 10 tries from load balancer: "
                + lb);
    }
    return server;

///////////////////////////////////////////////////////////////////
/**
 * Inspired by the implementation of {@link AtomicInteger#incrementAndGet()}.
 *
 * @param modulo The modulo to bound the value of the counter.
 * @return The next value.
 */
private int incrementAndGetModulo(int modulo) {
    for (;;) {
        int current = nextServerCyclicCounter.get();
        int next = (current + 1) % modulo;
        if (nextServerCyclicCounter.compareAndSet(current, next))
            return next;
    }
}

 

加權響應時間負載均衡 （WeightedResponseTime）

The basic idea for weighted round robin has been obtained from JCS
  The implementation for choosing the endpoint from the list of endpoints
  is as follows:Let's assume 4 endpoints:A(wt=10), B(wt=30), C(wt=40),
  D(wt=20).

  Using the Random API, generate a random number between 1 and10+30+40+20.
  Let's assume that the above list is randomized. Based on the weights, we
  have intervals as follows:

  1-----10 (A's weight)

  11----40 (A's weight + B's weight)

  41----80 (A's weight + B's weight + C's weight)

  81----100(A's weight + B's weight + C's weight + D's weight)

  Here's the psuedo code for deciding where to send the request:

  if (random_number between 1 & 10) {send request to A;}

  else if (random_number between 11 & 40) {send request to B;}

  else if (random_number between 41 & 80) {send request to C;}

  else if (random_number between 81 & 100) {send request to D;}

 

隨機負載均衡 （Random）

隨機選擇狀態爲UP的Server

int index = rand.nextInt(serverCount);
server = upList.get(index);

 

區域感知輪詢負載均衡（ZoneAware）

區域感知負載均衡內置電路跳閘邏輯，可被配置基於區域同源關係（Zone Affinity，也就是更傾向於選擇發出調用的服務所在的託管區域內，這樣可以降低延遲，節省成本）選擇目標服務實例。它監控每個區域中運行實例的行爲，而且能夠實時的快速丟棄一整個區域。這樣在面對整個區域故障時，幫我們提升了彈性。

The key metric used to measure the zone condition is Average Active Requests,
which is aggregated per rest client per zone. It is the
total outstanding requests in a zone divided by number of available targeted instances (excluding circuit breaker tripped instances).
This metric is very effective when timeout occurs slowly on a bad zone.

The  LoadBalancer will calculate and examine zone stats of all available zones. If the Average Active Requests for any zone has reached a configured threshold, this zone will be dropped from the active server list. In case more than one zone has reached the threshold, the zone with the most active requests per server will be dropped.
Once the the worst zone is dropped, a zone will be chosen among the rest with the probability proportional to its number of instances.
A server will be returned from the chosen zone with a given Rule (A Rule is a load balancing strategy, for example {@link AvailabilityFilteringRule})
For each request, the steps above will be repeated. That is to say, each zone related load balancing decisions are made at real time with the up-to-date statistics aiding the choice.

 

具體實現：

@Override
protected void setServerListForZones(Map<String, List<Server>> zoneServersMap) {
    super.setServerListForZones(zoneServersMap);
    if (balancers == null) {
        balancers = new ConcurrentHashMap<String, BaseLoadBalancer>();
    }
    for (Map.Entry<String, List<Server>> entry: zoneServersMap.entrySet()) {
        String zone = entry.getKey().toLowerCase();
        getLoadBalancer(zone).setServersList(entry.getValue());
    }
    // check if there is any zone that no longer has a server
    // and set the list to empty so that the zone related metrics does not
    // contain stale data
    for (Map.Entry<String, BaseLoadBalancer> existingLBEntry: balancers.entrySet()) {
        if (!zoneServersMap.keySet().contains(existingLBEntry.getKey())) {
            existingLBEntry.getValue().setServersList(Collections.emptyList());
        }
    }
}

@Override
public Server chooseServer(Object key) {
    if (!ENABLED.get() || getLoadBalancerStats().getAvailableZones().size() <= 1) {
        logger.debug("Zone aware logic disabled or there is only one zone");
        return super.chooseServer(key);
    }
    Server server = null;
    try {
        LoadBalancerStats lbStats = getLoadBalancerStats();
        Map<String, ZoneSnapshot> zoneSnapshot = ZoneAvoidanceRule.createSnapshot(lbStats);
        logger.debug("Zone snapshots: {}", zoneSnapshot);
        if (triggeringLoad == null) {
            triggeringLoad = DynamicPropertyFactory.getInstance().getDoubleProperty(
                    "ZoneAwareNIWSDiscoveryLoadBalancer." + this.getName() + ".triggeringLoadPerServerThreshold", 0.2d);
        }

        if (triggeringBlackoutPercentage == null) {
            triggeringBlackoutPercentage = DynamicPropertyFactory.getInstance().getDoubleProperty(
                    "ZoneAwareNIWSDiscoveryLoadBalancer." + this.getName() + ".avoidZoneWithBlackoutPercetage", 0.99999d);
        }
        Set<String> availableZones = ZoneAvoidanceRule.getAvailableZones(zoneSnapshot, triggeringLoad.get(), triggeringBlackoutPercentage.get());
        logger.debug("Available zones: {}", availableZones);
        if (availableZones != null &&  availableZones.size() < zoneSnapshot.keySet().size()) {
            String zone = ZoneAvoidanceRule.randomChooseZone(zoneSnapshot, availableZones);
            logger.debug("Zone chosen: {}", zone);
            if (zone != null) {
                BaseLoadBalancer zoneLoadBalancer = getLoadBalancer(zone);
                server = zoneLoadBalancer.chooseServer(key);
            }
        }
    } catch (Throwable e) {
        logger.error("Unexpected exception when choosing server using zone aware logic", e);
    }
    if (server != null) {
        return server;
    } else {
        logger.debug("Zone avoidance logic is not invoked.");
        return super.chooseServer(key);
    }
}

@VisibleForTesting
BaseLoadBalancer getLoadBalancer(String zone) {
    zone = zone.toLowerCase();
    BaseLoadBalancer loadBalancer = balancers.get(zone);
    if (loadBalancer == null) {
        // We need to create rule object for load balancer for each zone
        IRule rule = cloneRule(this.getRule());
        loadBalancer = new BaseLoadBalancer(this.getName() + "_" + zone, rule, this.getLoadBalancerStats());
        BaseLoadBalancer prev = balancers.putIfAbsent(zone, loadBalancer);
        if (prev != null) {
            loadBalancer = prev;
        }
    }
    return loadBalancer;
}

private IRule cloneRule(IRule toClone) {
    IRule rule;
    if (toClone == null) {
        rule = new AvailabilityFilteringRule();
    } else {
        String ruleClass = toClone.getClass().getName();
        try {
rule = (IRule) ClientFactory.instantiateInstanceWithClientConfig(ruleClass, this.getClientConfig());
 catch (Exception e) {
throw new RuntimeException("Unexpected exception creating rule for ZoneAwareLoadBalancer", e);

    }
    return rule;
}


@Override
public void setRule(IRule rule) {
    super.setRule(rule);
    if (balancers != null) {
        for (String zone: balancers.keySet()) {
            balancers.get(zone).setRule(cloneRule(rule));
        }
    }
}

 

 

Ribbon自帶負載均衡策略比較（轉）

策略名	策略聲明	策略描述	實現說明
BestAvailableRule	public class BestAvailableRule extends ClientConfigEnabledRoundRobinRule	選擇一個最小的併發請求的server	逐個考察Server，如果Server被tripped了，則忽略，在選擇其中ActiveRequestsCount最小的server
AvailabilityFilteringRule	public class AvailabilityFilteringRule extends PredicateBasedRule	過濾掉那些因爲一直連接失敗的被標記爲circuit tripped的後端server，並過濾掉那些高併發的的後端server（active connections 超過配置的閾值）	使用一個AvailabilityPredicate來包含過濾server的邏輯，其實就就是檢查status裏記錄的各個server的運行狀態
WeightedResponseTimeRule	public class WeightedResponseTimeRule extends RoundRobinRule	根據相應時間分配一個weight，相應時間越長，weight越小，被選中的可能性越低。	一個後臺線程定期的從status裏面讀取評價響應時間，爲每個server計算一個weight。Weight的計算也比較簡單responsetime 減去每個server自己平均的responsetime是server的權重。當剛開始運行，沒有形成statas時，使用roubine策略選擇server。
RetryRule	public class RetryRule extends AbstractLoadBalancerRule	對選定的負載均衡策略機上重試機制。	在一個配置時間段內當選擇server不成功，則一直嘗試使用subRule的方式選擇一個可用的server
RoundRobinRule	public class RoundRobinRule extends AbstractLoadBalancerRule	roundRobin方式輪詢選擇server	輪詢index，選擇index對應位置的server
RandomRule	public class RandomRule extends AbstractLoadBalancerRule	隨機選擇一個server	在index上隨機，選擇index對應位置的server
ZoneAvoidanceRule	public class ZoneAvoidanceRule extends PredicateBasedRule	複合判斷server所在區域的性能和server的可用性選擇server	使用ZoneAvoidancePredicate和AvailabilityPredicate來判斷是否選擇某個server，前一個判斷判定一個zone的運行性能是否可用，剔除不可用的zone（的所有server），AvailabilityPredicate用於過濾掉連接數過多的Server。

 

Ribbon架構圖

 Ribbon使用舉例：

1. 創建Maven項目：

<dependency>
    <groupId>com.netflix.ribbon</groupId>
    <artifactId>ribbon-core</artifactId>
    <version>2.2.0</version>
</dependency>
<dependency>
    <groupId>com.netflix.ribbon</groupId>
    <artifactId>ribbon-httpclient</artifactId>
    <version>2.2.0</version>
</dependency>

 

2. 配置properties file (sample-client.properties)

# Max number of retries
sample-client.ribbon.MaxAutoRetries=1

# Max number of next servers to retry (excluding the first server)
sample-client.ribbon.MaxAutoRetriesNextServer=1

# Whether all operations can be retried for this client
sample-client.ribbon.OkToRetryOnAllOperations=true

# Interval to refresh the server list from the source
sample-client.ribbon.ServerListRefreshInterval=2000

# Connect timeout used by Apache HttpClient
sample-client.ribbon.ConnectTimeout=3000

# Read timeout used by Apache HttpClient
sample-client.ribbon.ReadTimeout=3000

# Initial list of servers, can be changed via Archaius dynamic property at runtime
sample-client.ribbon.listOfServers=www.sohu.com:80,www.163.com:80,www.sina.com.cn:80

sample-client.ribbon.EnablePrimeConnections=true

 

3. 代碼：

public static void main( String[] args ) throws Exception {
    ConfigurationManager.loadPropertiesFromResources("sample-client.properties");
    System.out.println(ConfigurationManager.getConfigInstance().getProperty("sample-client.ribbon.listOfServers"));

    RestClient client = (RestClient)ClientFactory.getNamedClient("sample-client");
    HttpRequest request = HttpRequest.newBuilder().uri(new URI("/")).build();

    for(int i = 0; i < 20; i ++) {
        HttpResponse response = client.executeWithLoadBalancer(request);
        System.out.println("Status for URI:" + response.getRequestedURI() + " is :" + response.getStatus());
    }

    ZoneAwareLoadBalancer lb = (ZoneAwareLoadBalancer) client.getLoadBalancer();
    System.out.println(lb.getLoadBalancerStats());

    ConfigurationManager.getConfigInstance().setProperty("sample-client.ribbon.listOfServers", "www.baidu.com:80,www.linkedin.com:80");

    System.out.println("changing servers ...");
    Thread.sleep(3000);

    for(int i = 0; i < 20; i ++) {
        HttpResponse response = client.executeWithLoadBalancer(request);
        System.out.println("Status for URI:" + response.getRequestedURI() + " is :" + response.getStatus());
    }
    System.out.println(lb.getLoadBalancerStats());
}