wcf系列學習5天速成——第四天 wcf之分佈式架構

今天是wcf系列的第四天，也該出手壓軸戲了。嗯，現在的大型架構，都是神馬的，

nginx雞羣，iis雞羣，wcf雞羣，DB雞羣，由一個人作戰變成了羣毆.......

今天我就分享下wcf雞羣，高性能架構中一種常用的手法就是在內存中維護一個叫做“索引”的內存數據庫，

在實戰中利用“索引”這個概念做出"海量數據“的秒殺。

好，先上圖：

這個圖明白人都能看得懂吧。因爲我的系列偏重於wcf，所以我重點說下”心跳檢測“的實戰手法。

第一步：上一下項目的結構，才能做到心中有數。

第二步：“LoadDBService”這個是控制檯程序，目的就是從數據庫抽出關係模型加載在內存數據庫中，因爲這些東西會涉及一些算法的知識，

在這裏就不寫算法了，就簡單的模擬一下。

LoadDBServcie

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.Runtime.Serialization;
 using System.Web.Script.Serialization;
 using System.IO;
 using System.Xml.Serialization;
 using System.Xml;
 using Common;
 
 namespace LoadDBData
 {
     class Program
     {
         static void Main(string[] args)
         {
             //模擬從數據庫加載索引到內存中,形成內存中的數據庫
 //這裏的 "Dictionary" 用來表達“一個用戶註冊過多少店鋪“，即UserID與ShopID的一對多關係
             SerializableDictionary<int, List<int>> dic = new SerializableDictionary<int, List<int>>();
             
             List<int> shopIDList = new List<int>();
 
             for (int shopID = 300000; shopID < 300050; shopID++)
                 shopIDList.Add(shopID);
             
             int UserID = 15;
 
             //假設這裏已經維護好了UserID與ShopID的關係
             dic.Add(UserID, shopIDList);
             
             XmlSerializer xml = new XmlSerializer(dic.GetType());
             
             var memoryStream = new MemoryStream();
             
             xml.Serialize(memoryStream, dic);
             
             memoryStream.Seek(0, SeekOrigin.Begin);
             
             //將Dictionary持久化，相當於模擬保存在Mencache裏面
             File.AppendAllText("F://1.txt", Encoding.UTF8.GetString(memoryStream.ToArray()));
             
             Console.WriteLine("數據加載成功！");
             
             Console.Read();
         }
     }
 }

因爲Dictionary不支持序列化，所以我從網上拷貝了一份代碼讓其執行序列化

SerializableDictionary

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.Xml.Serialization;
 using System.Xml;
 using System.Xml.Schema;
 using System.Runtime.Serialization;
 
 namespace Common
 {
     ///<summary>
 /// 標題：支持 XML 序列化的 Dictionary
 ///</summary>
 ///<typeparam name="TKey"></typeparam>
 ///<typeparam name="TValue"></typeparam>
     [XmlRoot("SerializableDictionary")]
     public class SerializableDictionary<TKey, TValue> : Dictionary<TKey, TValue>, IXmlSerializable
     {
 
         public SerializableDictionary()
             : base()
         {
         }
         public SerializableDictionary(IDictionary<TKey, TValue> dictionary)
             : base(dictionary)
         {
         }
 
         public SerializableDictionary(IEqualityComparer<TKey> comparer)
             : base(comparer)
         {
         }
 
         public SerializableDictionary(int capacity)
             : base(capacity)
         {
         }
         public SerializableDictionary(int capacity, IEqualityComparer<TKey> comparer)
             : base(capacity, comparer)
         {
         }
         protected SerializableDictionary(SerializationInfo info, StreamingContext context)
             : base(info, context)
         {
         }
 
 
         public System.Xml.Schema.XmlSchema GetSchema()
         {
             return null;
         }
         ///<summary>
 /// 從對象的 XML 表示形式生成該對象
 ///</summary>
 ///<param name="reader"></param>
         public void ReadXml(System.Xml.XmlReader reader)
         {
             XmlSerializer keySerializer = new XmlSerializer(typeof(TKey));
             XmlSerializer valueSerializer = new XmlSerializer(typeof(TValue));
             bool wasEmpty = reader.IsEmptyElement;
             reader.Read();
             if (wasEmpty)
                 return;
             while (reader.NodeType != System.Xml.XmlNodeType.EndElement)
             {
                 reader.ReadStartElement("item");
                 reader.ReadStartElement("key");
                 TKey key = (TKey)keySerializer.Deserialize(reader);
                 reader.ReadEndElement();
                 reader.ReadStartElement("value");
                 TValue value = (TValue)valueSerializer.Deserialize(reader);
                 reader.ReadEndElement();
                 this.Add(key, value);
                 reader.ReadEndElement();
                 reader.MoveToContent();
             }
             reader.ReadEndElement();
         }
 
         /**/
         ///<summary>
 /// 將對象轉換爲其 XML 表示形式
 ///</summary>
 ///<param name="writer"></param>
         public void WriteXml(System.Xml.XmlWriter writer)
         {
             XmlSerializer keySerializer = new XmlSerializer(typeof(TKey));
             XmlSerializer valueSerializer = new XmlSerializer(typeof(TValue));
             foreach (TKey key in this.Keys)
             {
                 writer.WriteStartElement("item");
                 writer.WriteStartElement("key");
                 keySerializer.Serialize(writer, key);
                 writer.WriteEndElement();
                 writer.WriteStartElement("value");
                 TValue value = this[key];
                 valueSerializer.Serialize(writer, value);
                 writer.WriteEndElement();
                 writer.WriteEndElement();
             }
         }
 
     }
 }

第三步： "HeartBeatService"也做成了一個控制檯程序，爲了圖方便，把Contract和Host都放在一個控制檯程序中,

代碼中加入了註釋，看一下就會懂的。

IAddress.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Runtime.Serialization;
 using System.ServiceModel;
 using System.Text;
 
 namespace HeartBeatService
 {
     //CallbackContract：這個就是Client實現此接口，方便服務器端通知客戶端
     [ServiceContract(CallbackContract = typeof(ILiveAddressCallback))]
     public interface IAddress
     {
         ///<summary>
 /// 此方法用於Search啓動後，將Search地址插入到此處
 ///</summary>
 ///<param name="address"></param>
         [OperationContract(IsOneWay = true)]
         void AddSearch(string address);
 
         ///<summary>
 /// 此方法用於IIS端獲取search地址
 ///</summary>
 ///<param name="address"></param>
         [OperationContract(IsOneWay = true)]
         void GetService(string address);
     }
 }

Address.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Runtime.Serialization;
 using System.ServiceModel;
 using System.Text;
 using System.Timers;
 using System.IO;
 using System.Collections.Concurrent;
 using SearhService;
 using ClientService;
 
 namespace HeartBeatService
 {
     //InstanceContextMode：主要是管理上下文的實例，此處是single，也就是單體
 //ConcurrencyMode：    主要是用來控制實例中的線程數，此處是Multiple，也就是多線程
     [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Multiple)]
     public class Address : IAddress
     {
         static List<string> search = new List<string>();
 
         static object obj = new object();
 
         ///<summary>
 /// 此靜態構造函數用來檢測存活的Search個數
 ///</summary>
         static Address()
         {
             Timer timer = new Timer();
             timer.Interval = 6000;
             timer.Elapsed += (sender, e) =>
             {
 
                 Console.WriteLine("\n***************************************************************************");
                 Console.WriteLine("當前存活的Search爲：");
 
                 lock (obj)
                 {
                     //遍歷當前存活的Search
                     foreach (var single in search)
                     {
                         ChannelFactory<IProduct> factory = null;
 
                         try
                         {
                             //當Search存在的話，心跳服務就要定時檢測Search是否死掉，也就是定時的連接Search來檢測。
                             factory = new ChannelFactory<IProduct>(new NetTcpBinding(SecurityMode.None), new EndpointAddress(single));
                             factory.CreateChannel().TestSearch();
                             factory.Close();
 
                             Console.WriteLine(single);
 
                         }
                         catch (Exception ex)
                         {
                             Console.WriteLine(ex.Message);
 
                             //如果拋出異常，則說明此search已經掛掉
                             search.Remove(single);
                             factory.Abort();
                             Console.WriteLine("\n當前時間：" + DateTime.Now + " ,存活的Search有：" + search.Count() + "個");
                         }
                     }
                 }
 
                 //最後統計下存活的search有多少個
                 Console.WriteLine("\n當前時間：" + DateTime.Now + " ,存活的Search有：" + search.Count() + "個");
             };
             timer.Start();
         }
 
         public void AddSearch(string address)
         {
 
             lock (obj)
             {
                 //是否包含相同的Search地址
                 if (!search.Contains(address))
                 {
                     search.Add(address);
 
                     //search添加成功後就要告訴來源處，此search已經被成功載入。
                     var client = OperationContext.Current.GetCallbackChannel<ILiveAddressCallback>();
                     client.LiveAddress(address);
                 }
             }
         }
 
         public void GetService(string address)
         {
             Timer timer = new Timer();
             timer.Interval = 1000;
             timer.Elapsed += (obj, sender) =>
             {
                 try
                 {
                     //這個是定時的檢測IIS是否掛掉
                     var factory = new ChannelFactory<IServiceList>(new NetTcpBinding(SecurityMode.None),
                                                                    new EndpointAddress(address));
 
                     factory.CreateChannel().AddSearchList(search);
 
                     factory.Close();
 
                     timer.Interval = 10000;
                 }
                 catch (Exception ex)
                 {
                     Console.WriteLine(ex.Message);
                 }
             };
             timer.Start();
         }
     }
 }

ILiveAddressCallback.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.ServiceModel;
 
 namespace HeartBeatService
 {
     ///<summary>
 /// 等客戶端實現後，讓客戶端約束一下，只能是這個LiveAddress方法
 ///</summary>
     public interface ILiveAddressCallback
     {
         [OperationContract(IsOneWay = true)]
         void LiveAddress(string address);
     }
 }

第四步：我們開一下心跳，預覽下效果：

是的，心跳現在正在檢測是否有活着的Search。

第五步："SearhService" 這個Console程序就是WCF的search，主要用於從MemerCache裏面讀取索引。

記得要添加一下對“心跳服務”的服務引用。

IProduct.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Runtime.Serialization;
 using System.ServiceModel;
 using System.Text;
 
 namespace SearhService
 {
     // 注意: 使用“重構”菜單上的“重命名”命令，可以同時更改代碼和配置文件中的接口名“IService1”。
     [ServiceContract]
     public interface IProduct
     {
         [OperationContract]
         List<int> GetShopListByUserID(int userID);
 
         [OperationContract]
         void TestSearch();
     }
 }

Product.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Runtime.Serialization;
 using System.ServiceModel;
 using System.Text;
 using Common;
 using System.Xml;
 using System.IO;
 using System.Xml.Serialization;
 
 namespace SearhService
 {
     public class Product : IProduct
     {
         public List<int> GetShopListByUserID(int userID)
         {
             //模擬從MemCache中讀取索引
             SerializableDictionary<int, List<int>> dic = new SerializableDictionary<int, List<int>>();
 
             byte[] bytes = Encoding.UTF8.GetBytes(File.ReadAllText("F://1.txt", Encoding.UTF8));
 
             var memoryStream = new MemoryStream();
 
             memoryStream.Write(bytes, 0, bytes.Count());
 
             memoryStream.Seek(0, SeekOrigin.Begin);
 
             XmlSerializer xml = new XmlSerializer(dic.GetType());
 
             var obj = xml.Deserialize(memoryStream) as Dictionary<int, List<int>>;
 
             return obj[userID];
         }
 
         public void TestSearch() { }
     }
 }

SearchHost.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.ServiceModel;
 using System.Configuration;
 using System.Timers;
 using SearhService.HeartBeatService;
 
 namespace SearhService
 {
     public class SearchHost : IAddressCallback
     {
         static DateTime startTime;
 
         public static void Main()
         {
             ServiceHost host = new ServiceHost(typeof(Product));
 
             host.Open();
 
             AddSearch();
 
             Console.Read();
 
         }
 
         static void AddSearch()
         {
             startTime = DateTime.Now;
 
             Console.WriteLine("Search服務發送中.....\n\n*************************************************\n");
 
             try
             {
                 var heartClient = new AddressClient(new InstanceContext(new SearchHost()));
 
                 string search = ConfigurationManager.AppSettings["search"];
 
                 heartClient.AddSearch(search);
             }
             catch (Exception ex)
             {
                 Console.WriteLine("Search服務發送失敗：" + ex.Message);
             }
         }
 
         public void LiveAddress(string address)
         {
             Console.WriteLine("恭喜你," + address + "已被心跳成功接收！\n");
             Console.WriteLine("發送時間：" + startTime + "\n接收時間：" + DateTime.Now);
         }
     }
 }

第六步：此時Search服務已經建好，我們可以測試當Search開啓獲取關閉對心跳有什麼影響：

Search開啓時：

Search關閉時：

對的，當Search關閉時，心跳檢測該Search已經死掉，然後只能從集羣中剔除。

當然，我們可以將Search拷貝N份，部署在N臺機器中，只要修改一下endpoint地址就OK了，這一點明白人都會。

第七步："ClientService" 這裏也就指的是IIS，此時我們也要添加一下對心跳的服務引用。

IServiceList.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.ServiceModel;
 
 namespace ClientService
 {
     [ServiceContract]
     public interface IServiceList
     {
         [OperationContract]
         void AddSearchList(List<string> search);
     }
 }

ServiceList.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.ServiceModel;
 using System.Configuration;
 using System.Timers;
 using System.Threading;
 
 namespace ClientService
 {
     public class ServiceList : IServiceList
     {
         public static List<string> searchList = new List<string>();
 
         static object obj = new object();
 
         public static string Search
         {
             get
             {
                 lock (obj)
                 {
                     //如果心跳沒及時返回地址，客戶端就在等候
                     if (searchList.Count == 0)
                         Thread.Sleep(1000);
                     return searchList[new Random().Next(0, searchList.Count)];
                 }
             }
             set
             {
 
             }
         }
 
         public void AddSearchList(List<string> search)
         {
             lock (obj)
             {
                 searchList = search;
 
                 Console.WriteLine("************************************");
                 Console.WriteLine("當前存活的Search爲:");
 
                 foreach (var single in searchList)
                 {
                     Console.WriteLine(single);
                 }
             }
         }
     }
 }

Program.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.ServiceModel;
 using System.Configuration;
 using System.Threading;
 using ClientService.HeartBeatService;
 using SearhService;
 using BaseClass;
 using System.Data;
 using System.Diagnostics;
 
 namespace ClientService
 {
     class Program : IAddressCallback
     {
         static void Main(string[] args)
         {
 
             ServiceHost host = new ServiceHost(typeof(ServiceList));
 
             host.Open();
 
             var client = new AddressClient(new InstanceContext(new Program()));
 
             //配置文件中獲取iis的地址
             var iis = ConfigurationManager.AppSettings["iis"];
 
             //將iis的地址告訴心跳
             client.GetService(iis);
 
             //從集羣中獲取search地址來對Search服務進行調用
             var factory = new ChannelFactory<IProduct>(new NetTcpBinding(SecurityMode.None), new EndpointAddress(ServiceList.Search));
 
             //根據userid獲取了shopID的集合
             var shopIDList = factory.CreateChannel().GetShopListByUserID(15);
 
             //.......................... 後續就是我們將shopIDList做連接數據庫查詢（做到秒殺）
 
             Console.Read();
         }
 
         public void LiveAddress(string address)
         {
 
         }
     }
 }

然後我們開啓Client,看看效果咋樣：

當然，search集羣后，client得到search的地址是隨機的，也就分擔了search的負擔，實現有福同享，有難同當的效果了。

最後：我們做下性能檢測，看下“秒殺”和“毫秒殺”的效果。

首先在數據庫的User表和Shop插入了180萬和20萬的數據用於關聯。

ClientService改造後的代碼：

Program.cs

using System;
 using System.Collections.Generic;
 using System.Linq;
 using System.Text;
 using System.ServiceModel;
 using System.Timers;
 using System.Diagnostics;
 using BaseClass;
 using ClientService;
 using ClientService.HeartBeatService;
 using System.Configuration;
 using SearhService;
 
 namespace ClientService
 {
     class Program : IAddressCallback
     {
         static void Main(string[] args)
         {
 
             ServiceHost host = new ServiceHost(typeof(ServiceList));
 
             host.Open();
 
             var client = new AddressClient(new InstanceContext(new Program()));
 
             //配置文件中獲取iis的地址
             var iis = ConfigurationManager.AppSettings["iis"];
 
             //將iis的地址告訴心跳
             client.GetService(iis);
 
             //從集羣中獲取search地址來對Search服務進行調用
             var factory = new ChannelFactory<IProduct>(new NetTcpBinding(SecurityMode.None), new EndpointAddress(ServiceList.Search));
 
             //根據userid獲取了shopID的集合
 //比如說這裏的ShopIDList是通過索引交併集獲取的分頁的一些shopID
             var shopIDList = factory.CreateChannel().GetShopListByUserID(15);
 
             var strSql = string.Join(",", shopIDList);
 
             Stopwatch watch = new Stopwatch();
 
             watch.Start();
             SqlHelper.Query("select s.ShopID,u.UserName,s.ShopName  from [User] as u ,Shop as s where s.ShopID in(" + strSql + ")");
             watch.Stop();
 
             Console.WriteLine("通過wcf索引獲取的ID >>>花費時間:" + watch.ElapsedMilliseconds);
 
             //普通的sql查詢花費的時間
             StringBuilder builder = new StringBuilder();
 
             builder.Append("select * from ");
             builder.Append("(select  ROW_NUMBER() over(order by s.ShopID) as NumberID, ");
             builder.Append(" s.ShopID, u.UserName, s.ShopName ");
             builder.Append("from Shop s left join [User] as u on u.UserID=s.UserID ");
             builder.Append("where  s.UserID=15) as array ");
             builder.Append("where NumberID>300000 and NumberID<300050");
 
             watch.Start();
             SqlHelper.Query(builder.ToString());
             watch.Stop();
 
             Console.WriteLine("普通的sql分頁 >>>花費時間:" + watch.ElapsedMilliseconds);
 
             Console.Read();
         }
 
         public void LiveAddress(string address)
         {
 
         }
     }
 }

性能圖：

對的，一個秒殺，一個是毫秒殺，所以越複雜越能展示出“內存索引”的強大之處。

源碼下載：HeartBeat.rar

wcf系列學習5天速成——第四天 wcf之分佈式架構

前端使用 Konva 實現可視化設計器（13）- 折線 - 最優路徑應用【思路篇】

騰訊QQ Web Service 接口

查詢未提交事務個數

Asp.net集成Windows域賬戶登陸

談談基於Kerberos的Windows Network Authentication[上篇]

sharepoint 2010學習資源彙總

Mac下配置sublime實現LaTeX

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結