像原生C/C++，序列化Java對象

你曾經希望能夠將Java對象object快速地轉化爲字節流，就像原生語言C/C++？如果你使用標準的Java Serialization，你會對它的性能感到非常失望。Java Serialization針對不同的使用目的，而不僅僅是快速緊湊地序列化對象。

那爲什麼需要快速緊湊的序列化？我們遇到的大多數系統時分佈式的，我們需要在處理任務之間有效地通過傳遞狀態（state）進行交流，這個狀態在我們的對象中。所碰到的許多系統，相當大的一部分代價是在byte buffers序列化狀態（state）。目前已有一大批協議和機制實現了這種需求，其中的一部分協議使用簡單，但是非常低效，如Java Serialisation, XML 和JSON。另一部分採用二進制協議，性能高效，但是需要深入理解其技術。

這篇文章，將會使用簡單的二進制協議和Java的相關技術，達到與原生語言 C 或者 C++ 一樣的性能。

三種方法：

Java Serialization：實現Serializable接口；
Binary via ByteBuffer: 使用ByteBuffer API以二進制的方式寫入對象的field；這也是好的二進制編碼方法的基準。
Binary via Unsafe：引入Unsafe類及其方法，使用direct memory。

import sun.misc.Unsafe;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.lang.reflect.Field;
import java.nio.ByteBuffer;
import java.util.Arrays;

@SuppressWarnings("restriction")
public final class TestSerialisationPerf {
    public static final int REPETITIONS = 1 * 1000 * 1000;

    private static ObjectToBeSerialised ITEM = new ObjectToBeSerialised(1010L, true, 777, 99,
            new double[] { 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0 },
            new long[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 });

    public static void main(final String[] arg) throws Exception {
        for (final PerformanceTestCase testCase : testCases) {
            for (int i = 0; i < 5; i++) {
                testCase.performTest();

                System.out.format("%d %s\twrite=%,dns read=%,dns total=%,dns\n", i, testCase.getName(),
                        testCase.getWriteTimeNanos(), testCase.getReadTimeNanos(),
                        testCase.getWriteTimeNanos() + testCase.getReadTimeNanos());

                if (!ITEM.equals(testCase.getTestOutput())) {
                    throw new IllegalStateException("Objects do not match");
                }
                System.gc();
                Thread.sleep(3000);
            }
        }
    }

    private static final PerformanceTestCase[] testCases = {
            new PerformanceTestCase("Serialisation", REPETITIONS, ITEM) {
                ByteArrayOutputStream baos = new ByteArrayOutputStream();

                @Override
                public void testWrite(ObjectToBeSerialised item) throws Exception {
                    for (int i = 0; i < REPETITIONS; i++) {
                        baos.reset();
                        ObjectOutputStream oos = new ObjectOutputStream(baos);
                        oos.writeObject(item);
                        oos.close();
                    }
                }

                @Override
                public ObjectToBeSerialised testRead() throws Exception {
                    ObjectToBeSerialised object = null;
                    for (int i = 0; i < REPETITIONS; i++) {
                        ByteArrayInputStream bais = new ByteArrayInputStream(baos.toByteArray());
                        ObjectInputStream ois = new ObjectInputStream(bais);
                        object = (ObjectToBeSerialised) ois.readObject();
                    }
                    return object;
                }
            },

            new PerformanceTestCase("ByteBuffer", REPETITIONS, ITEM) {
                ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
                @Override
                public void testWrite(ObjectToBeSerialised item) throws Exception {
                    for (int i = 0; i < REPETITIONS; i++) {
                        byteBuffer.clear();
                        item.write(byteBuffer);
                    }
                }

                @Override
                public ObjectToBeSerialised testRead() throws Exception {
                    ObjectToBeSerialised object = null;
                    for (int i = 0; i < REPETITIONS; i++) {
                        byteBuffer.flip();
                        object = ObjectToBeSerialised.read(byteBuffer);
                    }
                    return object;
                }
            },

            new PerformanceTestCase("UnsafeMemory", REPETITIONS, ITEM) {
                UnsafeMemory buffer = new UnsafeMemory(new byte[1024]);
                @Override
                public void testWrite(ObjectToBeSerialised item) throws Exception {
                    for (int i = 0; i < REPETITIONS; i++) {
                        buffer.reset();
                        item.write(buffer);
                    }
                }

                @Override
                public ObjectToBeSerialised testRead() throws Exception {
                    ObjectToBeSerialised object = null;
                    for (int i = 0; i < REPETITIONS; i++) {
                        buffer.reset();
                        object = ObjectToBeSerialised.read(buffer);
                    }

                    return object;
                }
            }
        };
}

abstract class PerformanceTestCase {
    private final String name;
    private final int repetitions;
    private final ObjectToBeSerialised testInput;
    private ObjectToBeSerialised testOutput;
    private long writeTimeNanos;
    private long readTimeNanos;

    public PerformanceTestCase(final String name, final int repetitions, final ObjectToBeSerialised testInput) {
        this.name = name;
        this.repetitions = repetitions;
        this.testInput = testInput;
    }

    public String getName() {
        return name;
    }

    public ObjectToBeSerialised getTestOutput() {
        return testOutput;
    }

    public long getWriteTimeNanos() {
        return writeTimeNanos;
    }

    public long getReadTimeNanos() {
        return readTimeNanos;
    }

    public void performTest() throws Exception {
        final long startWriteNanos = System.nanoTime();
        testWrite(testInput);
        writeTimeNanos = (System.nanoTime() - startWriteNanos) / repetitions;

        final long startReadNanos = System.nanoTime();
        testOutput = testRead();
        readTimeNanos = (System.nanoTime() - startReadNanos) / repetitions;
    }

    public abstract void testWrite(ObjectToBeSerialised item) throws Exception;

    public abstract ObjectToBeSerialised testRead() throws Exception;
}

class ObjectToBeSerialised implements Serializable {
    private static final long serialVersionUID = 10275539472837495L;

    private final long sourceId;
    private final boolean special;
    private final int orderCode;
    private final int priority;
    private final double[] prices;
    private final long[] quantities;

    public ObjectToBeSerialised(final long sourceId, final boolean special, final int orderCode, final int priority,
            final double[] prices, final long[] quantities) {
        this.sourceId = sourceId;
        this.special = special;
        this.orderCode = orderCode;
        this.priority = priority;
        this.prices = prices;
        this.quantities = quantities;
    }

    public void write(final ByteBuffer byteBuffer) {
        byteBuffer.putLong(sourceId);
        byteBuffer.put((byte) (special ? 1 : 0));
        byteBuffer.putInt(orderCode);
        byteBuffer.putInt(priority);

        byteBuffer.putInt(prices.length);
        for (final double price : prices) {
            byteBuffer.putDouble(price);
        }

        byteBuffer.putInt(quantities.length);
        for (final long quantity : quantities) {
            byteBuffer.putLong(quantity);
        }
    }

    public static ObjectToBeSerialised read(final ByteBuffer byteBuffer) {
        final long sourceId = byteBuffer.getLong();
        final boolean special = 0 != byteBuffer.get();
        final int orderCode = byteBuffer.getInt();
        final int priority = byteBuffer.getInt();

        final int pricesSize = byteBuffer.getInt();
        final double[] prices = new double[pricesSize];
        for (int i = 0; i < pricesSize; i++) {
            prices[i] = byteBuffer.getDouble();
        }

        final int quantitiesSize = byteBuffer.getInt();
        final long[] quantities = new long[quantitiesSize];
        for (int i = 0; i < quantitiesSize; i++) {
            quantities[i] = byteBuffer.getLong();
        }

        return new ObjectToBeSerialised(sourceId, special, orderCode, priority, prices, quantities);
    }

    public void write(final UnsafeMemory buffer) {
        buffer.putLong(sourceId);
        buffer.putBoolean(special);
        buffer.putInt(orderCode);
        buffer.putInt(priority);
        buffer.putDoubleArray(prices);
        buffer.putLongArray(quantities);
    }

    public static ObjectToBeSerialised read(final UnsafeMemory buffer) {
        final long sourceId = buffer.getLong();
        final boolean special = buffer.getBoolean();
        final int orderCode = buffer.getInt();
        final int priority = buffer.getInt();
        final double[] prices = buffer.getDoubleArray();
        final long[] quantities = buffer.getLongArray();

        return new ObjectToBeSerialised(sourceId, special, orderCode, priority, prices, quantities);
    }

    @Override
    public boolean equals(final Object o) {
        if (this == o) {
            return true;
        }
        if (o == null || getClass() != o.getClass()) {
            return false;
        }

        final ObjectToBeSerialised that = (ObjectToBeSerialised) o;

        if (orderCode != that.orderCode) {
            return false;
        }
        if (priority != that.priority) {
            return false;
        }
        if (sourceId != that.sourceId) {
            return false;
        }
        if (special != that.special) {
            return false;
        }
        if (!Arrays.equals(prices, that.prices)) {
            return false;
        }
        if (!Arrays.equals(quantities, that.quantities)) {
            return false;
        }

        return true;
    }
}

@SuppressWarnings("restriction")
class UnsafeMemory {

    private static final Unsafe unsafe;

    static {
        try {
            Field field = Unsafe.class.getDeclaredField("theUnsafe");
            field.setAccessible(true);
            unsafe = (Unsafe) field.get(null);
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
    }

    private static final long byteArrayOffset = unsafe.arrayBaseOffset(byte[].class);
    private static final long longArrayOffset = unsafe.arrayBaseOffset(long[].class);
    private static final long doubleArrayOffset = unsafe.arrayBaseOffset(double[].class);

    private static final int SIZE_OF_BOOLEAN = 1;
    private static final int SIZE_OF_INT = 4;
    private static final int SIZE_OF_LONG = 8;

    private int pos = 0;
    private final byte[] buffer;

    public UnsafeMemory(final byte[] buffer) {
        if (null == buffer) {
            throw new NullPointerException("buffer cannot be null");
        }

        this.buffer = buffer;
    }

    public void reset() {
        this.pos = 0;
    }

    public void putBoolean(final boolean value) {
        unsafe.putBoolean(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_BOOLEAN;
    }

    public boolean getBoolean() {
        boolean value = unsafe.getBoolean(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_BOOLEAN;
        return value;
    }

    public void putInt(final int value) {
        unsafe.putInt(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_INT;
    }

    public int getInt() {
        int value = unsafe.getInt(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_INT;
        return value;
    }

    public void putLong(final long value) {
        unsafe.putLong(buffer, byteArrayOffset + pos, value);
        pos += SIZE_OF_LONG;
    }

    public long getLong() {
        long value = unsafe.getLong(buffer, byteArrayOffset + pos);
        pos += SIZE_OF_LONG;
        return value;
    }

    public void putLongArray(final long[] values) {
        putInt(values.length);
        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(values, longArrayOffset, buffer, byteArrayOffset + pos, bytesToCopy);
        pos += bytesToCopy;
    }

    public long[] getLongArray() {
        int arraySize = getInt();
        long[] values = new long[arraySize];
        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(buffer, byteArrayOffset + pos, values, longArrayOffset, bytesToCopy);
        pos += bytesToCopy;

        return values;
    }

    public void putDoubleArray(final double[] values) {
        putInt(values.length);
        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(values, doubleArrayOffset, buffer, byteArrayOffset + pos, bytesToCopy);
        pos += bytesToCopy;
    }

    public double[] getDoubleArray() {
        int arraySize = getInt();
        double[] values = new double[arraySize];

        long bytesToCopy = values.length << 3;
        unsafe.copyMemory(buffer, byteArrayOffset + pos, values, doubleArrayOffset, bytesToCopy);
        pos += bytesToCopy;

        return values;
    }
}

結果：

0 Serialisation write=3,131ns read=9,111ns total=12,242ns
1 Serialisation write=4,081ns read=12,723ns total=16,804ns
2 Serialisation write=4,337ns read=16,147ns total=20,484ns
3 Serialisation write=4,274ns read=11,520ns total=15,794ns
4 Serialisation write=4,392ns read=12,414ns total=16,806ns
0 ByteBuffer write=525ns read=564ns total=1,089ns
1 ByteBuffer write=459ns read=642ns total=1,101ns
2 ByteBuffer write=436ns read=638ns total=1,074ns
3 ByteBuffer write=435ns read=649ns total=1,084ns
4 ByteBuffer write=429ns read=637ns total=1,066ns
0 UnsafeMemory write=96ns read=220ns total=316ns
1 UnsafeMemory write=74ns read=217ns total=291ns
2 UnsafeMemory write=78ns read=216ns total=294ns
3 UnsafeMemory write=79ns read=213ns total=292ns
4 UnsafeMemory write=77ns read=214ns total=291ns

分析

使用Java Serialization寫入和讀取單個相對比較小的對象需要花費大約15,000ns，然而使用Unsafe能夠下降到300ns。有許多原因可以解釋爲什麼Java Serialisation代價高昂，如每個對象寫入類的全名和版本信息，而且ObjectOutputStream保留了一系列寫入對象，當調用close() 方法才進行合併。對每個樣本對象，Java Serialisation需要340字節，而使用二進制的方式只需要185字節。Java Serialisation的格式細節參考這裏。如果在樣本的絕大多數不使用數組，那麼序列化後的對象會遠大於Java Serialisation，因爲字段名稱。

使用Unsafe的方法獲取了性能的提升，因爲在Hotspot和其他的JVMs中，優化器對待這些操作作爲指令，替換成彙編指令調用執行內存的操作。

結論

Java序列化實現像原生C/C++一樣的性能是有可能的。上面的UnsafeMemory已提供基本框架的實現，可以進行擴充封裝這種行爲，這樣就可以使用這種鋒利的工具解決一些潛在的問題。

原文：Native C/C++ Like Performance For Java Object Serialization

像原生C/C++，序列化Java對象

三種方法：

結果：

分析

結論

圖解Java NIO Buffer

社區發現算法(一)

社區發現算法（二）

社區發現算法（三）

社區發現算法（四）

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結