基於SocketAsyncEventArgs(IOCP)的高性能TCP服務器實現（一）——封裝SocketAsyncEventArgs

 

最近碰到一個需求，就是有數千臺設備，這些設備都是通過運營商的網絡，基於TCP/IP協議發送一組信息給服務器，並且這些設備只是單向發送，不需要服務器返回信息，設備的信息發送頻率在一秒鐘一次。服務器端接受到之後，解析信息，然後入庫。這是正常的操作。所以對服務器端的信息接受軟件提出了較高的要求，這麼多設備，這麼高的頻率，要保證服務器端接受軟件的健壯，不能崩潰掉。在網上查找了相關文章之後，發現SocketAsyncEventArgs這個類，可以實現上述我想要的功能。參考了其他大神的文章之後，在下不才在這裏濫竽充數一下，各位見諒。

1、定義變量和輔助類

SocketAsyncEventArgs這個是微軟提供的一個類，可以在https://docs.microsoft.com/zh-cn/dotnet/api/system.net.sockets.socketasynceventargs?redirectedfrom=MSDN&view=netframework-4.8官網查看。但是微軟的例子比較粗糙，沒有達到我想要的目的，所以我這裏進行了一些改造。首先創建一個類，我這裏叫SocketServer。

在類SocketServer裏面，首先定義一些變量，使得我的這個服務端的接收軟件，可以支持較多的客戶端，也就是設備。

        private int m_maxConnectNum;    //最大連接數  
        private int m_revBufferSize;    //最大接收字節數  
        BufferManager m_bufferManager; //處理信息的工具
        const int opsToAlloc = 2;
        Socket listenSocket;            //監聽Socket  
        SocketEventPool m_pool;
        int m_clientCount;              //連接的客戶端數量  
        Semaphore m_maxNumberAcceptedClients;
        List<AsyncUserToken> m_clients; //客戶端列表  

其中BufferManager這個類是用來管理，客戶端發來的信息的，不是特別重要，具體代碼網上已經有人實現了，這裏我就直接貼出來了：

    // This class creates a single large buffer which can be divided up 
    // and assigned to SocketAsyncEventArgs objects for use with each 
    // socket I/O operation.  
    // This enables bufffers to be easily reused and guards against 
    // fragmenting heap memory.
    // 
    // The operations exposed on the BufferManager class are not thread safe.
    class BufferManager
    {
        int m_numBytes;                 // the total number of bytes controlled by the buffer pool
        byte[] m_buffer;                // the underlying byte array maintained by the Buffer Manager
        Stack<int> m_freeIndexPool;     // 
        int m_currentIndex;
        int m_bufferSize;

        public BufferManager(int totalBytes, int bufferSize)
        {
            m_numBytes = totalBytes;
            m_currentIndex = 0;
            m_bufferSize = bufferSize;
            m_freeIndexPool = new Stack<int>();
        }

        // Allocates buffer space used by the buffer pool
        public void InitBuffer()
        {
            // create one big large buffer and divide that 
            // out to each SocketAsyncEventArg object
            m_buffer = new byte[m_numBytes];
        }

        // Assigns a buffer from the buffer pool to the 
        // specified SocketAsyncEventArgs object
        //
        // <returns>true if the buffer was successfully set, else false</returns>
        public bool SetBuffer(SocketAsyncEventArgs args)
        {

            if (m_freeIndexPool.Count > 0)
            {
                args.SetBuffer(m_buffer, m_freeIndexPool.Pop(), m_bufferSize);
            }
            else
            {
                if ((m_numBytes - m_bufferSize) < m_currentIndex)
                {
                    return false;
                }
                args.SetBuffer(m_buffer, m_currentIndex, m_bufferSize);
                m_currentIndex += m_bufferSize;
            }
            return true;
        }

        // Removes the buffer from a SocketAsyncEventArg object.  
        // This frees the buffer back to the buffer pool
        public void FreeBuffer(SocketAsyncEventArgs args)
        {
            m_freeIndexPool.Push(args.Offset);
            args.SetBuffer(null, 0, 0);
        }

    }

SocketEventPool這個類用來異步管理客戶端的，代碼如下: 

    class SocketEventPool
    {
        Stack<SocketAsyncEventArgs> m_pool;


        public SocketEventPool(int capacity)
        {
            m_pool = new Stack<SocketAsyncEventArgs>(capacity);
        }

        public void Push(SocketAsyncEventArgs item)
        {
            if (item == null) { throw new ArgumentNullException("Items added to a SocketAsyncEventArgsPool cannot be null"); }
            lock (m_pool)
            {
                m_pool.Push(item);
            }
        }

        // Removes a SocketAsyncEventArgs instance from the pool  
        // and returns the object removed from the pool  
        public SocketAsyncEventArgs Pop()
        {
            lock (m_pool)
            {
                return m_pool.Pop();
            }
        }

        // The number of SocketAsyncEventArgs instances in the pool  
        public int Count
        {
            get { return m_pool.Count; }
        }

        public void Clear()
        {
            m_pool.Clear();
        }
    }

客戶端過來的信息，在服務器接收端都是異步處理，用AsyncUserToken這個類來管理客戶端（也就是設備），代碼也有人已經實現了，下面貼出代碼：

    class AsyncUserToken
    {
        /// <summary>  
        /// 客戶端IP地址  
        /// </summary>  
        public IPAddress IPAddress { get; set; }

        /// <summary>  
        /// 遠程地址  
        /// </summary>  
        public EndPoint Remote { get; set; }

        /// <summary>  
        /// 通信SOKET  
        /// </summary>  
        public Socket Socket { get; set; }

        /// <summary>  
        /// 連接時間  
        /// </summary>  
        public DateTime ConnectTime { get; set; }

        ///// <summary>  
        ///// 所屬用戶信息  
        ///// </summary>  
        //public UserInfoModel UserInfo { get; set; }


        /// <summary>  
        /// 數據緩存區  
        /// </summary>  
        public List<byte> Buffer { get; set; }


        public AsyncUserToken()
        {
            this.Buffer = new List<byte>();
        }
    }

上面這兩段代碼都是輔助類，沒那麼重要，這裏不多說。下面重點講講，我們這個服務器接收端怎麼來實現的。首先關心的是，服務器端接收到客戶端發送來的信息，要進行處理，這裏定義了一個委託來幫助我們，也就是說服務器接收到信息，就會進入這個函數進行處理：

        /// <summary>  
        /// 接收到客戶端的數據  
        /// </summary>  
        /// <param name="token">客戶端</param>  
        /// <param name="buff">客戶端數據</param>  
        public delegate void OnReceiveData(AsyncUserToken token, byte[] buff);

        /// <summary>  
        /// 接收到客戶端的數據事件  
        /// </summary>  
        public event OnReceiveData ReceiveClientData;

這樣調用我實現的服務器端時，就可以直接註冊自定義的函數來綁定，例如下面代碼：

         _socketServer.ReceiveClientData += onReceiveData;

        private void onReceiveData(AsyncUserToken token, byte[] buff)
        {
            //Dosomething;
        }

二、創建自定義封裝SocketServer類

好了，上面我們定義了一些參數，那麼在SocketServer這個類初始化的時候，就需要把這些參數初始化，看下面代碼：

        /// <summary>  
        /// 構造函數  
        /// </summary>  
        /// <param name="numConnections">最大連接數</param>  
        /// <param name="receiveBufferSize">緩存區大小</param>  
        public SocketServer(int numConnections, int receiveBufferSize)
        {
            m_clientCount = 0;
            m_maxConnectNum = numConnections;
            m_revBufferSize = receiveBufferSize;
            // allocate buffers such that the maximum number of sockets can have one outstanding read and   
            //write posted to the socket simultaneously    
            m_bufferManager = new BufferManager(receiveBufferSize * numConnections * opsToAlloc, receiveBufferSize);

            m_pool = new SocketEventPool(numConnections);
            m_maxNumberAcceptedClients = new Semaphore(numConnections, numConnections);
        }

三、SocketServer類的初始化

上面代碼是類SocketServer的構造函數，在構造函數裏我們設定了最大連接數以及每條消息的大小，因爲我們要希望我們的服務器端軟件能夠接受更多客戶端的信息。通過構造函數初始化參數之後，我們需要給每一個客戶端或者說每一個Socket分配一定的內存，如下代碼：

        /// <summary>  
        /// 初始化  
        /// </summary>  
        public void Init()
        {
            // Allocates one large byte buffer which all I/O operations use a piece of.  This gaurds   
            // against memory fragmentation  
            m_bufferManager.InitBuffer();
            m_clients = new List<AsyncUserToken>();
            // preallocate pool of SocketAsyncEventArgs objects  
            SocketAsyncEventArgs readWriteEventArg;

            for (int i = 0; i < m_maxConnectNum; i++)
            {
                readWriteEventArg = new SocketAsyncEventArgs();
                readWriteEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(IO_Completed);
                readWriteEventArg.UserToken = new AsyncUserToken();

                // assign a byte buffer from the buffer pool to the SocketAsyncEventArg object  
                m_bufferManager.SetBuffer(readWriteEventArg);
                // add SocketAsyncEventArg to the pool  
                m_pool.Push(readWriteEventArg);
            }
        }

        void IO_Completed(object sender, SocketAsyncEventArgs e)
        {
            // determine which type of operation just completed and call the associated handler  
            switch (e.LastOperation)
            {
                case SocketAsyncOperation.Receive:
                    ProcessReceive(e);
                    break;
                case SocketAsyncOperation.Send:
                    ProcessSend(e);
                    break;
                default:
                    throw new ArgumentException("The last operation completed on the socket was not a receive or send");
            }

        }

四、服務的啓動和終止

分配好內容之後，所有的準備工作就好了，下面我們可以直接寫SocketServer這個類的啓動函數了：

        /// <summary>  
        /// 啓動服務  
        /// </summary>  
        /// <param name="localEndPoint"></param>  
        public bool Start(IPEndPoint localEndPoint)
        {
            try
            {
                m_clients.Clear();
                listenSocket = new Socket(localEndPoint.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
                listenSocket.Bind(localEndPoint);
                // start the server with a listen backlog of 100 connections  
                listenSocket.Listen(m_maxConnectNum);
                // post accepts on the listening socket  
                StartAccept(null);
                return true;
            }
            catch (Exception)
            {
                return false;
            }
        }

        // Begins an operation to accept a connection request from the client   
        //  
        // <param name="acceptEventArg">The context object to use when issuing   
        // the accept operation on the server's listening socket</param>  
        public void StartAccept(SocketAsyncEventArgs acceptEventArg)
        {
            if (acceptEventArg == null)
            {
                acceptEventArg = new SocketAsyncEventArgs();
                acceptEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(AcceptEventArg_Completed);
            }
            else
            {
                // socket must be cleared since the context object is being reused  
                acceptEventArg.AcceptSocket = null;
            }

            m_maxNumberAcceptedClients.WaitOne();
            if (!listenSocket.AcceptAsync(acceptEventArg))
            {
                ProcessAccept(acceptEventArg);
            }
        }


        private void ProcessAccept(SocketAsyncEventArgs e)
        {
            try
            {
                Interlocked.Increment(ref m_clientCount);
                // Get the socket for the accepted client connection and put it into the   
                //ReadEventArg object user token  
                SocketAsyncEventArgs readEventArgs = m_pool.Pop();
                AsyncUserToken userToken = (AsyncUserToken)readEventArgs.UserToken;
                userToken.Socket = e.AcceptSocket;
                userToken.ConnectTime = DateTime.Now;
                userToken.Remote = e.AcceptSocket.RemoteEndPoint;
                userToken.IPAddress = ((IPEndPoint)(e.AcceptSocket.RemoteEndPoint)).Address;

                lock (m_clients) { m_clients.Add(userToken); }

                if (ClientNumberChange != null)
                    ClientNumberChange(1, userToken);
                if (!e.AcceptSocket.ReceiveAsync(readEventArgs))
                {
                    ProcessReceive(readEventArgs);
                }
            }
            catch (Exception me)
            {
                LogHelper.WriteLog(me.Message + "\r\n" + me.StackTrace);
            }

            // Accept the next connection request  
            if (e.SocketError == SocketError.OperationAborted) return;
            StartAccept(e);
        }

通過上述步驟，我們就可以通過一個IP地址和一個端口來啓動SocketServer服務了。爲了使SocketServer這個服務器接收軟件更加健壯，我們再添加一個停止服務的功能：

        /// <summary>  
        /// 停止服務  
        /// </summary>  
        public void Stop()
        {
            foreach (AsyncUserToken token in m_clients)
            {
                try
                {
                    listenSocket.Shutdown(SocketShutdown.Both);
                }
                catch (Exception) { }
            }
            

            listenSocket.Close();
            int c_count = m_clients.Count;
            lock (m_clients) { m_clients.Clear(); }

            if (ClientNumberChange != null)
                ClientNumberChange(-c_count, null);
        }

上述代碼通過循環來關閉每一個Socket連接。

五、註冊服務端信息接收處理函數

在前面的代碼中，我們在初始化的時候，已經註冊了SocketAsyncEventArgs這個類中的Completed事件，這個事件的意思就是，服務器不管是接收還是發送，完成後的函數都是IO_Completed。

         readWriteEventArg.Completed += new EventHandler<SocketAsyncEventArgs>(IO_Completed);

        void IO_Completed(object sender, SocketAsyncEventArgs e)
        {
            // determine which type of operation just completed and call the associated handler  
            switch (e.LastOperation)
            {
                case SocketAsyncOperation.Receive:
                    ProcessReceive(e);
                    break;
                case SocketAsyncOperation.Send:
                    ProcessSend(e);
                    break;
                default:
                    throw new ArgumentException("The last operation completed on the socket was not a receive or send");
            }

        }

而在委託的IO_Completed函數中，我們通過接收到的信息判斷是接收消息還是發送消息，如果是接收，那麼我們使用ProcessReceive(SocketAsyncEventArgs e)這個函數來處理接收到的信息。

        // This method is invoked when an asynchronous receive operation completes.   
        // If the remote host closed the connection, then the socket is closed.    
        // If data was received then the data is echoed back to the client.  
        //  
        private void ProcessReceive(SocketAsyncEventArgs e)
        {
            try
            {
                // check if the remote host closed the connection  
                AsyncUserToken token = (AsyncUserToken)e.UserToken;
                if (e.BytesTransferred > 0 && e.SocketError == SocketError.Success)
                {
                    //讀取數據  
                    byte[] data = new byte[e.BytesTransferred];
                    Array.Copy(e.Buffer, e.Offset, data, 0, e.BytesTransferred);
                    lock (token.Buffer)
                    {
                        token.Buffer.AddRange(data);
                    }
                    if (ReceiveClientData != null)
                    {
                        ReceiveClientData(token, data);
                    }

                    //繼續接收. 爲什麼要這麼寫,請看Socket.ReceiveAsync方法的說明  
                    if (!token.Socket.ReceiveAsync(e))
                        this.ProcessReceive(e);
                }
                else
                {
                    CloseClientSocket(e);
                }
            }
            catch (Exception xe)
            {
                LogHelper.WriteLog(xe.Message + "\r\n" + xe.StackTrace);
            }
        }

上面的代碼可以看出來，我們接收到信息後，是要再把信息委託給另外的註冊委託函數進行處理：

                    if (ReceiveClientData != null)
                    {
                        ReceiveClientData(token, data);
                    }

到這裏爲止，我們就把SocketServer這個類的接收功能給實現了，如果有讀者對客戶端發送信息感興趣，就直接參考下面的代碼：

六、註冊服務端信息發送處理函數

        // This method is invoked when an asynchronous send operation completes.    
        // The method issues another receive on the socket to read any additional   
        // data sent from the client  
        //  
        // <param name="e"></param>  
        private void ProcessSend(SocketAsyncEventArgs e)
        {
            if (e.SocketError == SocketError.Success)
            {
                // done echoing data back to the client  
                AsyncUserToken token = (AsyncUserToken)e.UserToken;
                // read the next block of data send from the client  
                bool willRaiseEvent = token.Socket.ReceiveAsync(e);
                if (!willRaiseEvent)
                {
                    ProcessReceive(e);
                }
            }
            else
            {
                CloseClientSocket(e);
            }
        }

        //關閉客戶端  
        private void CloseClientSocket(SocketAsyncEventArgs e)
        {
            AsyncUserToken token = e.UserToken as AsyncUserToken;

            lock (m_clients) { m_clients.Remove(token); }
            //如果有事件,則調用事件,發送客戶端數量變化通知  
            if (ClientNumberChange != null)
                ClientNumberChange(-1, token);
            // close the socket associated with the client  
            try
            {
                token.Socket.Shutdown(SocketShutdown.Send);
            }
            catch (Exception) { }
            token.Socket.Close();
            // decrement the counter keeping track of the total number of clients connected to the server  
            Interlocked.Decrement(ref m_clientCount);
            m_maxNumberAcceptedClients.Release();
            // Free the SocketAsyncEventArg so they can be reused by another client  
            e.UserToken = new AsyncUserToken();
            m_pool.Push(e);
        }



        /// <summary>  
        /// 對數據進行打包,然後再發送  
        /// </summary>  
        /// <param name="token"></param>  
        /// <param name="message"></param>  
        /// <returns></returns>  
        public void SendMessage(AsyncUserToken token, byte[] message)
        {
            if (token == null || token.Socket == null || !token.Socket.Connected)
                return;
            try
            {
                //新建異步發送對象, 發送消息  
                SocketAsyncEventArgs sendArg = new SocketAsyncEventArgs();
                sendArg.UserToken = token;
                sendArg.SetBuffer(byte, 0, byte.Length);  //將數據放置進去.  
                token.Socket.SendAsync(sendArg);
            }
            catch (Exception e)
            {
                LogHelper.WriteLog("SendMessage - Error:" + e.Message);
            }
        }

上述的代碼，都已提供下載，下載地址：

https://download.csdn.net/download/aplsc/11817545