關於數據執行效率

今天開發碰到了一個問題,從數據庫取出一個表中的數據,大約500條左右。然後通過map轉換爲另外一個Model,總共用時1分鐘,有時候會更多。所以,我開始懷疑automapper的執行效率。

於是,我決定寫個demo看看

首先我想要先創建一個entity,和一個model

我的entity是這樣的

 

public class TestTable
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public DateTime brithday { get; set; }
        public long Age { get; set; }
        public bool Gender { get; set; }
        public short IDTypeCode { get; set; }

        public int Id2 { get; set; }
        public string Name2 { get; set; }
        public DateTime brithday2 { get; set; }
        public long Age2 { get; set; }
        public bool Gender2 { get; set; }
        public short IDTypeCode2 { get; set; }

        public int Id3 { get; set; }
        public string Name3 { get; set; }
        public DateTime brithday3 { get; set; }
        public long Age3 { get; set; }
        public bool Gender3 { get; set; }
        public short IDTypeCode3 { get; set; }

        public int Id4 { get; set; }
        public string Name4 { get; set; }
        public DateTime brithday4 { get; set; }
        public long Age4 { get; set; }
        public bool Gender4 { get; set; }
        public short IDTypeCode4 { get; set; }

        public int Id5 { get; set; }
        public string Name5 { get; set; }
        public DateTime brithday5 { get; set; }
        public long Age5 { get; set; }
        public bool Gender5 { get; set; }
        public short IDTypeCode5 { get; set; }
}

 

 

然後我的Modelentity一樣

 

public class TestModel2
    {
        public int Id { get; set; }
        public string Name { get; set; }
        public DateTime brithday { get; set; }
        public long Age { get; set; }
        public bool Gender { get; set; }
        public short IDTypeCode { get; set; }

        public int Id2 { get; set; }
        public string Name2 { get; set; }
        public DateTime brithday2 { get; set; }
        public long Age2 { get; set; }
        public bool Gender2 { get; set; }
        public short IDTypeCode2 { get; set; }

        public int Id3 { get; set; }
        public string Name3 { get; set; }
        public DateTime brithday3 { get; set; }
        public long Age3 { get; set; }
        public bool Gender3 { get; set; }
        public short IDTypeCode3 { get; set; }

        public int Id4 { get; set; }
        public string Name4 { get; set; }
        public DateTime brithday4 { get; set; }
        public long Age4 { get; set; }
        public bool Gender4 { get; set; }
        public short IDTypeCode4 { get; set; }

        public int Id5 { get; set; }
        public string Name5 { get; set; }
        public DateTime brithday5 { get; set; }
        public long Age5 { get; set; }
        public bool Gender5 { get; set; }
        public short IDTypeCode5 { get; set; }
}

 

  

接着我需要一個可以給我生成數據的class,我寫了它

 

public static class BuildTestData
    {
        private static T Data<T>(int? i = null)
        {
            T data = (T)Activator.CreateInstance(typeof(T));
            if (i == null)
            {
                return data;
            }
            var propertys = data.GetType().GetProperties();
            foreach (var type in propertys)
            {
                if (!type.CanWrite)
                {
                    continue;
                }
                if (type.PropertyType == typeof(int) || type.PropertyType == typeof(int?))
                {
                    type.SetValue(data, i);
                }
                else if (type.PropertyType == typeof(long) || type.PropertyType == typeof(long?))
                {
                    type.SetValue(data, Convert.ToInt64(i));
                }
                else if (type.PropertyType == typeof(bool) || type.PropertyType == typeof(bool?))
                {
                    type.SetValue(data, true);
                }
                else if (type.PropertyType == typeof(DateTime) || type.PropertyType == typeof(DateTime?))
                {
                    type.SetValue(data, DateTime.Now);
                }
                else if (type.PropertyType == typeof(string))
                {
                    string value = "test" + type.Name + i;
                    type.SetValue(data, value);
                }
            }
            return data;
        }

        public static T CreateData<T>(this T obj)
        {
            return Data<T>(1);
        }


        public static List<T> CreateData<T>(this List<T> obj, int size, bool isAddNull = false)
        {
            obj = new List<T>();
            for (int i = 1; i <= size; i++)
            {
                var data = Data<T>(i);
                obj.Add(data);
            }
            if (isAddNull)
            {
                obj.Add(Data<T>());
            }
            return obj;
        }

}

 

然後分別使用forforeachautomapper轉換爲TestModel2

 

for

 

public List<TestModel2> ValidationFor(List<TestTable> list)
        {
            List<TestModel2> list2 = new List<TestModel2>();
            for (int i = 0; i < list.Count; i++)
            {
                list2.Add(new TestModel2()
                {
                    Id = list[i].Id,
                    Name = list[i].Name,
                    brithday = list[i].brithday,
                    Age = list[i].Age,
                    Gender = list[i].Gender,
                    IDTypeCode = list[i].IDTypeCode,
                    Id2 = list[i].Id,
                    Name2 = list[i].Name,
                    brithday2 = list[i].brithday,
                    Age2 = list[i].Age,
                    Gender2 = list[i].Gender,
                    IDTypeCode2 = list[i].IDTypeCode,
                    Id3 = list[i].Id,
                    Name3 = list[i].Name,
                    brithday3 = list[i].brithday,
                    Age3 = list[i].Age,
                    Gender3 = list[i].Gender,
                    IDTypeCode3 = list[i].IDTypeCode,
                    Id4 = list[i].Id,
                    Name4 = list[i].Name,
                    brithday4 = list[i].brithday,
                    Age4 = list[i].Age,
                    Gender4 = list[i].Gender,
                    IDTypeCode4 = list[i].IDTypeCode,
                    Id5 = list[i].Id,
                    Name5 = list[i].Name,
                    brithday5 = list[i].brithday,
                    Age5 = list[i].Age,
                    Gender5 = list[i].Gender,
                    IDTypeCode5 = list[i].IDTypeCode,
                });
            }
            return list2;
        }

 

Foreach:

public List<TestModel2> ValidationForeach(List<TestTable> list)
        {
            List<TestModel2> list2 = new List<TestModel2>();
            foreach (var item in list)
            {
                list2.Add(new TestModel2()
                {
                    Id = item.Id,
                    Name = item.Name,
                    brithday = item.brithday,
                    Age = item.Age,
                    Gender = item.Gender,
                    IDTypeCode = item.IDTypeCode,
                    Id2 = item.Id,
                    Name2 = item.Name,
                    brithday2 = item.brithday,
                    Age2 = item.Age,
                    Gender2 = item.Gender,
                    IDTypeCode2 = item.IDTypeCode,
                    Id3 = item.Id,
                    Name3 = item.Name,
                    brithday3 = item.brithday,
                    Age3 = item.Age,
                    Gender3 = item.Gender,
                    IDTypeCode3 = item.IDTypeCode,
                    Id4 = item.Id,
                    Name4 = item.Name,
                    brithday4 = item.brithday,
                    Age4 = item.Age,
                    Gender4 = item.Gender,
                    IDTypeCode4 = item.IDTypeCode,
                    Id5 = item.Id,
                    Name5 = item.Name,
                    brithday5 = item.brithday,
                    Age5 = item.Age,
                    Gender5 = item.Gender,
                    IDTypeCode5 = item.IDTypeCode,
                });
            }
            return list2;
        }

Automapper(不得不說map可以省很多事)

 

public List<TestModel2> ValidationAutoMapper(List<TestTable> list)
        {
            //this.CreateMap<List<TestModel>, List<TestModel2>>();

            Mapper.Initialize(cfg => cfg.CreateMap<List<TestTable>, List<TestModel2>>());
            //or
            var config = new MapperConfiguration(cfg => cfg.CreateMap<List<TestTable>, List<TestModel2>>());
            var result = Mapper.Map<List<TestTable>, List<TestModel2>>(list);
            return result;
        }

 

然後我用我生成數據的類,生成了十萬條數據

 

public static List<TestTable> list = new List<TestTable>();
list = list.CreateData(100000);

 

然後使用Stopwatch 測試執行時間

 

Program pa = new Program();
            Stopwatch stopTime = new Stopwatch();
            //pa.CreateEfData();
            Log log = new Log();
            list = list.CreateData(100000);
            //list2 = list2.CreateData(100000);
            stopTime.Start();
            pa.ValidationFor(list);
            stopTime.Stop();
            TimeSpan ts = TimeSpan.FromMilliseconds(stopTime.ElapsedMilliseconds);
            string str = string.Format("{0:D2}:{1:D2}:{2:D2}.{3:D3}", ts.Hours, ts.Minutes, ts.Seconds, ts.Milliseconds);
            log.log("for:" + str);
            Console.WriteLine("for:" + str);
            stopTime.Reset();

            stopTime.Start();
            pa.ValidationForeach(list);
            stopTime.Stop();
            TimeSpan ts2 = TimeSpan.FromMilliseconds(stopTime.ElapsedMilliseconds);
            string str2 = string.Format("{0:D2}:{1:D2}:{2:D2}.{3:D3}", ts2.Hours, ts2.Minutes, ts2.Seconds, ts2.Milliseconds);
            log.log("foreach:" + str2);
            Console.WriteLine("foreach:" + str2);
            stopTime.Reset();

            stopTime.Start();
            pa.ValidationAutoMapper(list);
            stopTime.Stop();
            TimeSpan ts3 = TimeSpan.FromMilliseconds(stopTime.ElapsedMilliseconds);
            string str3 = string.Format("{0:D2}:{1:D2}:{2:D2}.{3:D3}", ts3.Hours, ts3.Minutes, ts3.Seconds, ts3.Milliseconds);
            log.log("automapper:" + str3);
            Console.WriteLine("automapper:" + str3);
            stopTime.Reset();

            Console.ReadLine();

 

運行結果:

可以得出這樣一個結論

 

Foreach是最快的一個,其次是for,再接着就是automapperautomapperforforeach的整整兩倍。

 

可是,就算是這樣,我500條數據map爲什麼會用了1分鐘呢?難道是因爲數據庫?

 

我在數據庫中又接着創建了這樣一張表

 

--create table TestTable
--(
--    Id int identity(1,1) primary key,
--    Name nvarchar(100),
--    brithday datetime,
--    Age bigint,
--    Gender bit,
--    IDTypeCode smallint,

--    Id2 int ,
--    Name2 nvarchar(100),
--    brithday2 datetime,
--    Age2 bigint,
--    Gender2 bit,
--    IDTypeCode2 smallint,

--    Id3 int  ,
--    Name3 nvarchar(100),
--    brithday3 datetime,
--    Age3 bigint,
--    Gender3 bit,
--    IDTypeCode3 smallint,

--    Id4 int,
--    Name4 nvarchar(100),
--    brithday4 datetime,
--    Age4 bigint,
--    Gender4 bit,
--    IDTypeCode4 smallint,

--    Id5 int,
--    Name5 nvarchar(100),
--    brithday5 datetime,
--    Age5 bigint,
--    Gender5 bit,
--    IDTypeCode5 smallint,
--)

 

添加了6萬條數據

 

通過sql取得數據

 

public void CreateSqlData()
        {
            using (SqlConnection con = new SqlConnection("*****"))
            {
                con.Open();
                SqlCommand com = new SqlCommand();
                com.Connection = con;
                com.CommandText = "select Id, Name, brithday, Age, Gender, IDTypeCode, Id2, Name2, brithday2, Age2, Gender2, IDTypeCode2, Id3, Name3, brithday3, Age3, Gender3, IDTypeCode3, Id4, Name4, brithday4, Age4, Gender4, IDTypeCode4, Id5, Name5, brithday5, Age5, Gender5, IDTypeCode5 from TestTable";
                com.CommandType = System.Data.CommandType.Text;
                using (SqlDataReader read = com.ExecuteReader())
                {
                    while (read.Read())
                    {
                        list.Add(new TestTable()
                        {
                            Id = Convert.ToInt32(read["Id"]),
                            Name = read["Name"].ToString(),
                            brithday = Convert.ToDateTime(read["brithday"]),
                            Age = Convert.ToInt64(read["Age"]),
                            Gender = Convert.ToBoolean(read["Gender"]),
                            IDTypeCode = Convert.ToInt16(read["IDTypeCode"]),

                            Id2 = Convert.ToInt32(read["Id2"]),
                            Name2 = read["Name2"].ToString(),
                            brithday2 = Convert.ToDateTime(read["brithday2"]),
                            Age2 = Convert.ToInt64(read["Age2"]),
                            Gender2 = Convert.ToBoolean(read["Gender2"]),
                            IDTypeCode2 = Convert.ToInt16(read["IDTypeCode2"]),

                            Id3 = Convert.ToInt32(read["Id3"]),
                            Name3 = read["Name3"].ToString(),
                            brithday3 = Convert.ToDateTime(read["brithday3"]),
                            Age3 = Convert.ToInt64(read["Age3"]),
                            Gender3 = Convert.ToBoolean(read["Gender3"]),
                            IDTypeCode3 = Convert.ToInt16(read["IDTypeCode3"]),

                            Id4 = Convert.ToInt32(read["Id4"]),
                            Name4 = read["Name4"].ToString(),
                            brithday4 = Convert.ToDateTime(read["brithday4"]),
                            Age4 = Convert.ToInt64(read["Age4"]),
                            Gender4 = Convert.ToBoolean(read["Gender4"]),
                            IDTypeCode4 = Convert.ToInt16(read["IDTypeCode4"]),

                            Id5 = Convert.ToInt32(read["Id5"]),
                            Name5 = read["Name5"].ToString(),
                            brithday5 = Convert.ToDateTime(read["brithday5"]),
                            Age5 = Convert.ToInt64(read["Age5"]),
                            Gender5 = Convert.ToBoolean(read["Gender5"]),
                            IDTypeCode5 = Convert.ToInt16(read["IDTypeCode5"]),
                        });
                    }
                }
            }
        }

 

結果

 

這下最快的變成了for,其次是foreach、最後automapper,可以發現automapper已經慢的b不止一個層次的問題了。

那麼,如果我把取數據的地方換成ef呢?

首先是配置

public class MyEntities : IdentityDbContext<IdentityUser>
    {
        public MyEntities() : base("TestDb")
        {
            Database.SetInitializer<MyEntities>(null);
        }
        public DbSet<TestTable> TestTable { get; set; }
        protected override void OnModelCreating(DbModelBuilder modelBuilder)
        {
            modelBuilder.Conventions.Remove<PluralizingTableNameConvention>();
            base.OnModelCreating(modelBuilder);
        }
}

 

使用ef取得數據

 

public void CreateEfData()
        {
            MyEntities myTest = new MyEntities();
            var demo = from c in myTest.TestTable
                       select c;
            list = demo.ToList();
        }

 

結果

 

神奇的事情發生了有沒有,automapper居然又恢復到了100毫秒,難道efautomapper有幫助嗎?

可問題依然沒有得到解決,我想知道的是,爲什麼我的map只有500條數據卻用了1分鐘,讓我很糾結

於是我仔細的看的entity

忽然有兩行代碼引起了我的注意

 

[ForeignKey("AId")]
        public virtual ATable ATable { get; set; }

        [ForeignKey("BId")]
        public virtual BTable BTable { get; set; }

 

會不會是因爲表關係?

我把map刪掉後換成了foreach

 

AName = c.ATable == null ? string.Empty : c.ATable.Name,
BName = c.BTable == null ? string.Empty : c.BTable.Name,

 

運行結果

58秒(XD快了幾秒)

那如果我把這兩行代碼刪掉呢?

奇蹟發生了,15秒!!!

果然,是這兩個表關係搞得鬼,緊接着,我就把代碼做了優化

先把數據取出來,然後再進行賦值

 

var aName = string.Empty;
            var bName = string.Empty;
            if (aList != null && aList.Count > 0)
            {
                aName = aList.FirstOrDefault(s => s.Id == item.AId).Name;
            }
            if (bList != null && bList.Count > 0)
            {
                bName = bList.FirstOrDefault(s => s.Id == item.BId).Name;
            }

            AName = aName
            BName = bName

 

運行結果 18XD~

問題終於得到了解決

可是,話又說回來,ef通過virtual 取出來的關係數據效率有這麼差嗎?

還是這個地方有別的什麼可以優化的地方嗎?

這個問題還需要深究。

 

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章