C#將各種數據寫入字節數組的快速方法

    最近在優化實時數據庫(參考influxdb寫的)存儲性能時,遇到了要將bool, int, long, double, string 序列化爲字節數組(mashal),然後再Snappy壓縮的部分,通過visual studio 2017中的性能探查器,發現mashal CPU佔比2.23%,有優化潛力,於是做了如下嘗試:

     原方案: List<byte> + BitConverter.GetBytes()

     集中改進的思路:

  1. 採用ArrayPool<T>避免每次分配數組的內存;
  2. 採用BinaryWriter+MemoryStream ;
  3. 用BitConverter.TryWriteBytes + Span<T>;
  4. 自己實現一個ByteWriter。

    經過2個多小時的嘗試,發現2、3比原方案CPU佔比反而更高了,查了一下bing,貌似BitConverter.GetBytes()在底層用指針實現的,效率頗高,也許.net core2.1 裏面的BinaryWriter還沒優化吧。

    然後看到一篇讀寫二進制最快的方法的stackoverflow,核心思想是用位移來進行數據的類型轉換,受到啓發,決定自己實現一個ByteWriter,實測CPU佔爲1.12%,即下降50%,注意:ByteWriter.EndWrite返回的byte[]在使用(如Stream.Write())後,應調用Release。當然,通過unsafe指針操作性能還有提升空間。

using System.Buffers;

namespace Arim.Plat.Trend.Engine
{
    public class ByteWriter
    {
        byte[] buffer;
        int i;
        public ByteWriter(int length)
        {
            Length = length;
            buffer = ArrayPool<byte>.Shared.Rent(length);
            i = 0;
        }

        public int Length { get; private set; }

        public void Write(byte v)
        {
            buffer[i++] = v;
        }

        public void Write(bool v)
        {
            if (v) buffer[i++] = 0x01;
            else buffer[i++] = 0x00;
        }

        public void Write(ushort v)
        {
            buffer[i++] = (byte)v;
            buffer[i++] = (byte)(v >> 8);
        }

        public void Write(int v)
        {
            buffer[i++] = (byte)v;
            buffer[i++] = (byte)(v >> 8);
            buffer[i++] = (byte)(v >> 16);
            buffer[i++] = (byte)(v >> 24);
        }

        public void Write(uint v)
        {
            buffer[i++] = (byte)v;
            buffer[i++] = (byte)(v >> 8);
            buffer[i++] = (byte)(v >> 16);
            buffer[i++] = (byte)(v >> 24);
        }

        public void Write(long v)
        {
            buffer[i++] = (byte)v;
            buffer[i++] = (byte)(v >> 8);
            buffer[i++] = (byte)(v >> 16);
            buffer[i++] = (byte)(v >> 24);
            buffer[i++] = (byte)(v >> 32);
            buffer[i++] = (byte)(v >> 40);
            buffer[i++] = (byte)(v >> 48);
            buffer[i++] = (byte)(v >> 56);
        }

        public void Write(ulong v)
        {
            buffer[i++] = (byte)v;
            buffer[i++] = (byte)(v >> 8);
            buffer[i++] = (byte)(v >> 16);
            buffer[i++] = (byte)(v >> 24);
            buffer[i++] = (byte)(v >> 32);
            buffer[i++] = (byte)(v >> 40);
            buffer[i++] = (byte)(v >> 48);
            buffer[i++] = (byte)(v >> 56);
        }

        public void Write(double v)
        {
            Write(FloatEncoder.Float64bits(v));
        }

        public void Write(string v)
        {
            byte[] strBytes = System.Text.Encoding.Default.GetBytes(v);
            int len = strBytes.Length;
            Write(len);
            for(int j=0;j<len;j++)
            {
                buffer[i++] = strBytes[j];
            }
        }

        public void Write(byte[] v)
        {
            for (int j = 0; j < v.Length; j++)
            {
                buffer[i++] = v[j];
            }
        }

        public byte[] EndWrite()
        {
            return buffer;
        }

        public void Release()
        {
            ArrayPool<byte>.Shared.Return(buffer); 
        }
    }
}

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章