Raft一致性框架_Copycat基礎學習(一)

Copycat is a fault-tolerant state machine replication framework. Built
on the Raft consensus algorithm, it handles replication and
persistence and enforces strict ordering of inputs and outputs,
allowing developers to focus on single-threaded application logic. Its
event-driven model allows for efficient client communication with
replicated state machines, from simple key-value stores to wait-free
locks and leader elections. You supply the state machine and Copycat
takes care of the rest, making it easy to build robust, safe
distributed systems.

上面一段摘錄於Copycat官網的介紹(http://atomix.io/copycat/),那麼Copycat 是一個基於Raft一致性算法的編程框架，它能夠爲分佈式應用中的狀態提供一致性。本文主要基於Copycat官網給的示例進行學習.

1.首先在IDE裏面創建一個maven工程，並在pom文件中加入依賴:

<dependency>
    <groupId>io.atomix.copycat</groupId>
    <artifactId>copycat-server</artifactId>
    <version>1.1.4</version>
</dependency>
<dependency>
    <groupId>io.atomix.copycat</groupId>
    <artifactId>copycat-client</artifactId>
    <version>1.1.4</version>
</dependency>
<dependency>
    <groupId>io.atomix.catalyst</groupId>
    <artifactId>catalyst-netty</artifactId>
    <version>1.1.1</version>
</dependency>

2.自定義StateMachine以及Command

//自定了MapstateMachine，它繼承框架提供的StateMachine類，MapstateMachine主要處理來自客戶端的操作，如示例建的這個類，用於處理兩個操作,put和get.put用於向map中寫入鍵值，get用於獲取值
public class MapstateMachine extends StateMachine implements Snapshottable {
    //此爲copycat-server需要維護的一致性數據結構,本例使用的是MAP
    private Map<Object, Object> map = new HashMap<>();

    //定義對map的put操作
    public Object put(Commit<PutCommand> commit) {
        try {
            map.put(commit.operation().key(), commit.operation().value());
        } finally {
            commit.close();
        }
        return null;
    }
    //定義對map的get操作
    public Object get(Commit<GetQuery> commit) {
        try {
            return map.get(commit.operation().key());
        } finally {
            commit.close();
        }
    }

    //以下兩個方法來自於實現Snapshottable的接口，實現這個接口是用於copycat-server能夠對本地狀態日誌進行壓縮，並形成snapshot(快照),當copycat-server重啓後，可以從快照恢復狀態，如果有其它的server加入進來，可以將快照複製到其它server上.
    @Override
    public void snapshot(SnapshotWriter writer) {
        writer.writeObject(map);
    }

    @Override
    public void install(SnapshotReader reader) {
        map = reader.readObject();
    }
}

GetQuery類

package com.xkx.common;

import io.atomix.copycat.Query;

//定義對MapstateMachine查詢的命令
public class GetQuery implements Query<Object> {

    private final Object key;

    public GetQuery(Object key){
        this.key = key;
    }

    public Object key(){
        return key;
    }
}

PutCommand類

package com.xkx.common;


import io.atomix.copycat.Command;

public class PutCommand implements Command<Object> {

    private final Object key;
    private final Object value;

    public PutCommand(Object key,Object value){
        this.key = key;
        this.value = value;
    }

    public Object key(){
        return key;
    }

    public Object value(){
        return value;
    }

}

PutCommand和GetQuery類都實現Command接口.

3.最後定義服務器端和客戶端,copycat_server這裏我們實現3個，copyCat_server-1,copyCat_server-2,copyCat_server-3。它們共同組成一個cluster.這裏我們通過copyCat_server-2,copyCat_server-3 join到copyCat_server-1的方式形成cluseter

copyCat_server-1 實現

package com.xkx.myCopycat;

import com.xkx.common.GetQuery;
import com.xkx.common.MapstateMachine;
import com.xkx.common.PutCommand;
import io.atomix.catalyst.transport.Address;
import io.atomix.catalyst.transport.netty.NettyTransport;
import io.atomix.copycat.server.CopycatServer;
import io.atomix.copycat.server.storage.Storage;
import io.atomix.copycat.server.storage.StorageLevel;

import java.io.File;
import java.util.concurrent.CompletableFuture;

public class Main {

    public static void main(String[] args){

        //設置server_1的地址和端口
        Address address = new Address("127.0.0.1", 5000);
        //通過chain的方式創建copy_cat server
        CopycatServer server = CopycatServer.builder(address)
                .withStateMachine(MapstateMachine::new)
                .withTransport(NettyTransport.builder()
                        .withThreads(4)
                        .build())
                .withStorage(Storage.builder()
                        .withDirectory(new File("logs"))
                      .withStorageLevel(StorageLevel.DISK)
                        .build())
                .build();
        //註冊putCommand和GetQuery命令類
        server.serializer().register(PutCommand.class);
        server.serializer().register(GetQuery.class);

        //啓動服務器
        CompletableFuture<CopycatServer> future = server.bootstrap();
        future.join();

    }

}

copyCat_server-2 實現

package com.xkx.myCopycat2;

import com.xkx.common.GetQuery;
import com.xkx.common.MapstateMachine;
import com.xkx.common.PutCommand;
import io.atomix.catalyst.transport.Address;
import io.atomix.catalyst.transport.netty.NettyTransport;
import io.atomix.copycat.server.CopycatServer;
import io.atomix.copycat.server.storage.Storage;
import io.atomix.copycat.server.storage.StorageLevel;

import java.io.File;
import java.util.Collection;
import java.util.Collections;

public class Main2 {

    public static void main(String[] args){

        Address address = new Address("127.0.0.1", 5001);

        CopycatServer server = CopycatServer.builder(address)
                .withStateMachine(MapstateMachine::new)
                .withTransport(NettyTransport.builder()
                        .withThreads(4)
                        .build())
                .withStorage(Storage.builder()
                        .withDirectory(new File("logs"))
                        .withStorageLevel(StorageLevel.DISK)
                        .build())
                .build();

        server.serializer().register(PutCommand.class);
        server.serializer().register(GetQuery.class);

        //這裏通過join到copyCat-server-1實現cluster
        Collection<Address> cluster = Collections.singleton(new Address("127.0.0.1", 5000));
        server.join(cluster).join();

    }

}

這裏只給出copyCat-server-1和copyCat_server-2 的實現，copyCat-server-3跟copyCat_server-2 實現相同，只是改變了下IP地址和端口.

copycat-client實現

package com.xkx.client;

import com.xkx.common.GetQuery;
import com.xkx.common.PutCommand;
import io.atomix.catalyst.transport.Address;
import io.atomix.catalyst.transport.netty.NettyTransport;
import io.atomix.copycat.client.CopycatClient;

import java.util.Arrays;
import java.util.Collection;
import java.util.concurrent.CompletableFuture;

public class ClientMain {

    public static void main(String[] args){

        CopycatClient.Builder builder = CopycatClient.builder();

        builder.withTransport(NettyTransport.builder()
                .withThreads(2)
                .build());

        CopycatClient client = builder.build();

        //客戶端註冊命令
        client.serializer().register(PutCommand.class);
        client.serializer().register(GetQuery.class);


        //集羣的ip以及端口
        Collection<Address> cluster = Arrays.asList(
                new Address("127.0.0.1", 5000),
                new Address("127.0.0.1", 5001),
                new Address("127.0.0.1", 5002)
        );

        CompletableFuture<CopycatClient> future = client.connect(cluster);
        future.join();

        //使用PutCommand提交三個鍵值對
        CompletableFuture[] futures = new CompletableFuture[3];
        futures[0] = client.submit(new PutCommand("foo", "Hello world!"));
        futures[1] = client.submit(new PutCommand("bar", "Hello world!"));
        futures[2] = client.submit(new PutCommand("baz", "Hello world!"));

        //等待集羣完成一致性的複製後，打印完成的結果
        CompletableFuture.allOf(futures).thenRun(() -> System.out.println("Commands completed!"));

        //客戶端提交查詢
        client.submit(new GetQuery("foo")).thenAccept(result -> {
            System.out.println("foo is: " + result);
        });
    }

}

java項目工程結構:

注意copyCat-server 和 copyCat-client都應該使用相同的GetQuery,MapstateMachine,PutCommand類，所以放在了common目錄下，也就說他們都需要應用相同的類。

實驗結果:
copyCat-server-1 console內容:

copyCat-server-2 console內容:

copyCat-server-3 console內容:

copyCat-client console內容:

可以看到三臺server中copyCat-server-1被選舉爲Leader,另外兩臺爲Follower,所有請求都會到copyCat-server-1來處理，並通過Raft算法複製到另外兩臺server。