思科VPP源碼分析（內存管理）

思科今年開源的vpp項目，在intel開源的dpdk之上，構建的數據包處理框架。
dpdk組件已經成功榨乾硬件IO性能，剩下的瓶頸落在業務處理部分，其中最關鍵的又在內存訪問優化上。
內存優化一句話概括：提高CPU L1，L2，L3 cache命中率。這裏將分析vpp內存管理部分源碼。

vec變長數組（vec_bootstrap.h）

len是數組元素個數，不是字節長度。每個數組元素都看作是等大小。
自定義頭部後接vec 組合使用。組合vec將引申出其他內存管理結構。

_vec_find(v)
v指向數據部分，宏返回vec頭部地址
_vec_round_size(s)
s按sizeof (uword)大小，向上對齊
vec_header_bytes (uword header_bytes)
自定義頭部+vec 組合這種模型下，header_bytes代表自定義頭部大小。函數返回自定義頭部+vec總長度，按sizeof (vec_header_t)向上對齊。
vec_header (void *v, uword header_bytes)
自定義頭部+vec 組合，按sizeof (vec_header_t)向上對齊這種模型下，v指向數據部分地址，header_bytes代表自定義頭部大小，返回自定義頭部地址。
vec_header_end (void *v, uword header_bytes)
自定義頭部+vec 組合，按sizeof (vec_header_t)向上對齊這種模型下，v指向自定義頭部地址，header_bytes代表自定義頭部大小，返回數據部分地址。
vec_aligned_header (void *v, uword header_bytes, uword align)
自定義頭部+vec 組合，按align向上對齊這種模型下，v指向數據部分地址，header_bytes代表自定義頭部大小，返回自定義頭部地址。
vec_aligned_header_end (void *v, uword header_bytes, uword align)
自定義頭部+vec 組合，按align向上對齊這種模型下，v指向自定義頭部地址，header_bytes代表自定義頭部大小，返回數據部分地址。
_vec_len(v)
v指向數據部分，返回vec頭部的vec_header_t->len
vec_len(v)
v指向數據部分，v等於null時返回0，否則返回vec頭部的vec_header_t->len
vec_reset_length(v)
v指向數據部分，vec頭部的vec_header_t->len置0
vec_bytes(v)
v指向數據部分，返回數據部分總字節大小
vec_capacity(v,b)

v指向data起始地址，b代表user_header大小，返回該mheap_elt_t字節大小，即vec的最大可佔用內存。博主覺得該函數放在這裏破壞了該頭件的獨立性。
vec_max_len(v)
類似vec_capacity(v,b)內存佈局，v指向data起始地址，返回該vec可容納最大數據塊數目。
vec_end(v)
v指向data起始地址，返回vec數據部分末尾的下一個字節。
vec_is_member(v,e)
v指向data起始地址，返回e是否在該vec地址範圍內。
vec_elt_at_index(v,i)
v指向data起始地址，返回第i個數據塊的地址
vec_elt(v,i)
v指向data起始地址，返回第i個數據塊的內容
vec_foreach(var,vec)
vec_foreach_backwards(var,vec)
vec_foreach_index(var,v)
迭代宏
mheap(mheap_bootstrap.h, mheap.c)

mheap整體視圖

注意每個elt大小不一定相等。這裏的vec_header_t把後面數據看成一個字節一個字節的元素，因此vec->len實際是整個數據部分字節長度。

mheap_elt_t視圖

基本函數操作在mheap_bootstrap.h中
該頭文件基本函數，結合上文內容很容易理解，這裏不再詳細敘述。

mheap.c中會對上圖基本結構更近一步索引管理，也是mheap的核心部分。

mheap_maybe_lock (void *v)
mheap_maybe_unlock (void *v)
自旋鎖加鎖解鎖函數，如果是持有鎖的本CPU再次進入，可以無條件獲取鎖。

mheap_get_aligned (void v, uword n_user_data_bytes, uword align, uword align_offset, uword offset_return)
mheap內存分配接口函數


void *
mheap_get_aligned (void *v,
           uword n_user_data_bytes,
           uword align, uword align_offset, uword * offset_return)
{
  mheap_t *h;
  uword offset;
  u64 cpu_times[2];

  cpu_times[0] = clib_cpu_time_now ();

  //align至少是mheap_elt_t大小，並且是2的冪
  align = clib_max (align, STRUCT_SIZE_OF (mheap_elt_t, user_data[0]));
  align = max_pow2 (align);

  /* Correct align offset to be smaller than alignment. */
  align_offset &= (align - 1);

  /* Align offset must be multiple of minimum object size. */
  if (align_offset % STRUCT_SIZE_OF (mheap_elt_t, user_data[0]) != 0)
    {
      *offset_return = MHEAP_GROUNDED;
      return v;
    }

  /* Round requested size. */
  //各種對齊，確保mheap_elt_t頭部和數據部分都是STRUCT_SIZE_OF (mheap_elt_t, user_data[0])，之後再用專門章節詳細分析數據結構對齊背後的作者設計目的
  n_user_data_bytes = clib_max (n_user_data_bytes, MHEAP_MIN_USER_DATA_BYTES);
  n_user_data_bytes =
    round_pow2 (n_user_data_bytes,
        STRUCT_SIZE_OF (mheap_elt_t, user_data[0]));

  if (!v)
    v = mheap_alloc (0, 64 << 20);//分配完整的一個mheap結構

  mheap_maybe_lock (v);

  h = mheap_header (v);

  if (h->flags & MHEAP_FLAG_VALIDATE)
    mheap_validate (v);//遍歷mheap的空閒塊鏈表，驗證是否正確鏈接。

  /* First search free lists for object. */
  //從空閒塊鏈表中獲取數據塊
  offset =
    mheap_get_search_free_list (v, &n_user_data_bytes, align, align_offset);

  h = mheap_header (v);

  /* If that fails allocate object at end of heap by extending vector. */
  if (offset == MHEAP_GROUNDED && _vec_len (v) < h->max_size)
    {
     //如果現有空閒塊沒有滿足要求的塊，則從mheap中新分配塊，注意h->max_size代表該mheap最大內存量，_vec_len (v)代表目前使用量
      v =
    mheap_get_extend_vector (v, n_user_data_bytes, align, align_offset,
                 &offset);
      //mheap_get_extend_vector()並不會導致mheap重新分配，博主認爲下面重新計算h毫無必要
      h = mheap_header (v);
      //如果擴充成功則統計值加1
      h->stats.n_vector_expands += offset != MHEAP_GROUNDED;
    }

  *offset_return = offset;
  if (offset != MHEAP_GROUNDED)
    {
      h->n_elts += 1;

      if (h->flags & MHEAP_FLAG_TRACE)
    {
      /* Recursion block for case when we are traceing main clib heap. */
      h->flags &= ~MHEAP_FLAG_TRACE;

      mheap_get_trace (v, offset, n_user_data_bytes);

      h->flags |= MHEAP_FLAG_TRACE;
    }
    }

  if (h->flags & MHEAP_FLAG_VALIDATE)
    mheap_validate (v);

  mheap_maybe_unlock (v);

  cpu_times[1] = clib_cpu_time_now ();
  h->stats.n_clocks_get += cpu_times[1] - cpu_times[0];
  h->stats.n_gets += 1;

  return v;
}


void *
mheap_alloc (void *memory, uword size)
{
  uword flags = 0;

  if (memory != 0)
    flags |= MHEAP_FLAG_DISABLE_VM;

//這裏指的是利用cpu的SIMD向量指令加快緩存塊的cache查找，之後會詳細介紹
#ifdef CLIB_HAVE_VEC128
  flags |= MHEAP_FLAG_SMALL_OBJECT_CACHE;
#endif

  return mheap_alloc_with_flags (memory, size, flags);
}


void *
mheap_alloc_with_flags (void *memory, uword memory_size, uword flags)
{
  mheap_t *h;
  void *v;
  uword size;

  if (!mheap_page_size)
    mheap_page_size = clib_mem_get_page_size ();

  if (!memory)
    {
      /* No memory given, try to VM allocate some. */
      //通常用的是UNIX模式，該分配函數將mmap()匿名內存使用
      memory = clib_mem_vm_alloc (memory_size);
      if (!memory)
    return 0;

      /* No memory region implies we have virtual memory. */
      flags &= ~MHEAP_FLAG_DISABLE_VM;
    }

  /* Make sure that given memory is page aligned. */
  {
    uword am, av, ah;

    //把memory向上對mheap_page_size 大小對齊
    am = pointer_to_uword (memory);
    av = mheap_page_round (am);
    //計算對齊後的mheap頭部地址
    v = uword_to_pointer (av, void *);
    h = mheap_header (v);
    ah = pointer_to_uword (h);
    //mheap頭部地址小於memory起始地址，則增加一個mheap_page_size。這樣mheap頭部地址既在mmap()分配的內存範圍中，也滿足對齊要求。
    while (ah < am)
      ah += mheap_page_size;

    h = uword_to_pointer (ah, void *);
    v = mheap_vector (h);

    if (PREDICT_FALSE (memory + memory_size < v))
      {
    /*
     * This will happen when the requested memory_size is too
     * small to cope with the heap header and/or memory alignment.
     */
    clib_mem_vm_free (memory, memory_size);
    return 0;
      }
    //size代表mheap數據部分大小
    size = memory + memory_size - v;
  }

  /* VM map header so we can use memory. */
  if (!(flags & MHEAP_FLAG_DISABLE_VM))
    clib_mem_vm_map (h, sizeof (h[0]));

  /* Zero vector header: both heap header and vector length. */
  memset (h, 0, sizeof (h[0]));
  _vec_len (v) = 0;

  h->vm_alloc_offset_from_header = (void *) h - memory;
  h->vm_alloc_size = memory_size;

  h->max_size = size;
  h->owner_cpu = ~0;

  /* Set flags based on those given less builtin-flags. */
  h->flags |= (flags & ~MHEAP_FLAG_TRACE);

  /* Unmap remainder of heap until we will be ready to use it. */
  //雖然不解除映射也不會影響性能。但是這樣好處是一旦訪問了未分配的內存會導致段錯誤，提示開發人員編碼錯誤
  if (!(h->flags & MHEAP_FLAG_DISABLE_VM))
    mheap_vm (v, MHEAP_VM_UNMAP | MHEAP_VM_ROUND_UP,
          (clib_address_t) v, h->max_size);

  /* Initialize free list heads to empty. */
  //MHEAP_GROUNDED代表該bin下沒有空閒數據塊，值爲~0。所以這裏用0xFF初始化。
  memset (h->first_free_elt_uoffset_by_bin, 0xFF,
      sizeof (h->first_free_elt_uoffset_by_bin));

  return v;
}


/* Search free lists for object with given size and alignment. */
static uword
mheap_get_search_free_list (void *v,
                uword * n_user_bytes_arg,
                uword align, uword align_offset)
{
  mheap_t *h = mheap_header (v);
  uword bin, n_user_bytes, i, bi;

  n_user_bytes = *n_user_bytes_arg;
  //計算bin值，用來索引對應大小空閒塊鏈表
  bin = user_data_size_to_bin_index (n_user_bytes);

  //如果請求的是滿足條件的塊，則從一個額外的cache中獲取，避免了搜索空閒塊鏈表開銷，還可以利用CPU的SIMD向量指令來加速匹配。
  if (MHEAP_HAVE_SMALL_OBJECT_CACHE
      && (h->flags & MHEAP_FLAG_SMALL_OBJECT_CACHE)
      && bin < 255
      && align == STRUCT_SIZE_OF (mheap_elt_t, user_data[0])
      && align_offset == 0)
    {
      //cache中尋找數據塊，之後會詳細分析
      uword r = mheap_get_small_object (h, bin);
      h->stats.n_small_object_cache_attempts += 1;
      if (r != MHEAP_GROUNDED)
    {
      h->stats.n_small_object_cache_hits += 1;
      return r;
    }
    }

  //h->non_empty_free_elt_heads是一個位圖，記錄了對應bin鏈表是否爲空
  for (i = bin / BITS (uword); i < ARRAY_LEN (h->non_empty_free_elt_heads);
       i++)
    {
      uword non_empty_bin_mask = h->non_empty_free_elt_heads[i];

      /* No need to search smaller bins. */
      //把比bin小的位置的位圖略掉
      if (i == bin / BITS (uword))
    non_empty_bin_mask &= ~pow2_mask (bin % BITS (uword));

      /* Search each occupied free bin which is large enough. */
      //foreach_set_bit()從右往左遍歷每個bit，爲1的bit調用mheap_get_search_free_bin()
      //foreach_set_bit()中first_set (uword x)用來計算從右算第一個爲1的bit代表的數
      foreach_set_bit (bi, non_empty_bin_mask, (
                         {
                         uword r =
                         mheap_get_search_free_bin (v,
                                        bi
                                        +
                                        i
                                        *
                                        BITS
                                        (uword),
                                        n_user_bytes_arg,
                                        align,
                                        align_offset);
                         if (r !=
                             MHEAP_GROUNDED) return
                         r;}
               ));
    }

  return MHEAP_GROUNDED;
}

/*如果每次都是計算bin值再從鏈表中找空閒塊就太麻煩了，對滿足特定條件塊，也應該是vpp使用頻率最多的那種類型的塊做了個cache，只需要計算bin值就能直接得到可用空閒塊。該函數就是幹這事。*/
always_inline uword
mheap_get_small_object (mheap_t * h, uword bin)
{
/*cache中c->bins.as_u8數組每個元素記錄了(bin+1),爲0代表該位置沒有空閒塊。c->offsets記錄對應bin的空閒塊。
  mheap_small_object_cache_t *c = &h->small_object_cache;
  //查找c->bins.as_u8數組中，所有值爲(bin+1)的索引組成的位圖
  uword mask = mheap_small_object_cache_mask (c, bin + 1);
  uword offset = MHEAP_GROUNDED;

  //如果位圖不爲0，說明有空閒塊，查找位圖最左邊那個爲1bit的索引，返 回對應的c->offsets值，即可用空閒塊。
  if (mask)
    {
      uword i = min_log2 (mask);
      uword o = c->offsets[i];
      ASSERT (o != MHEAP_GROUNDED);
      c->bins.as_u8[i] = 0;
      offset = o;
    }

  return offset;
}


always_inline uword
mheap_small_object_cache_mask (mheap_small_object_cache_t * c, uword bin)
{
  uword mask;

/* $$$$ ELIOT FIXME: add Altivec version of this routine */
//本函數依賴於CPU支持向量指令。
#if !defined (CLIB_HAVE_VEC128) || defined (__ALTIVEC__)
  mask = 0;
#else
  //返回一個由16字節組成的向量，每個字節的值都是bin
  u8x16 b = u8x16_splat (bin);

  ASSERT (bin < 256);

/*u8x16_is_equal()比較兩個128bit位向量，每個字節進行對比，如果相等，返回字節爲1，一共返回16個字節，組成一個返回值128bit向量。u8x16_compare_byte_mask()把返回的128bit向量轉化成位圖，共16個bit位圖*/
#define _(i) ((uword) u8x16_compare_byte_mask (u8x16_is_equal (b, c->bins.as_u8x16[i])) << (uword) ((i)*16))
  mask = _(0) | _(1);
  if (BITS (uword) > 32)
    mask |= _(2) | _(3);
#undef _

#endif
//32位系統返回的是32bit位圖，64位系統返回64bit位圖
  return mask;
}

/*該函數搜索bin對應的空閒塊鏈表。找到一個大小滿足要求的空閒塊是很容易的，本函數額外大量代碼花在了滿足空閒塊的對齊要求上。align:首地址相對於heap偏移對齊，align_offset:首地址對齊後，再向左做個偏移。
分配內存時從對應的slot中查找，值得注意的是查找算法爲了滿足參數中對齊要求，很有可能從空閒塊中間部分分配內存，這樣首尾部分空閒塊重新鏈入新的slot。一切的一切都是爲了數據對齊。如果分配的內存是一個帶頭部的數據結構，例如vec+data類型。那麼分配內存時需要把頭部大小作爲align_offset參數，使數據部分滿足起始地址align對齊。*/

static uword
mheap_get_search_free_bin (void *v,
               uword bin,
               uword * n_user_data_bytes_arg,
               uword align, uword align_offset)
{
  mheap_t *h = mheap_header (v);
  mheap_elt_t *e;

  /* Free object is at offset f0 ... f1;
     Allocatted object is at offset o0 ... o1. */
  //o0,f0是用於首部，f0，f1用於尾部，lo_free_usize切分後首部空間，hi_free_usize切分後尾部空間
  word o0, o1, f0, f1, search_n_user_data_bytes;
  word lo_free_usize, hi_free_usize;

  ASSERT (h->first_free_elt_uoffset_by_bin[bin] != MHEAP_GROUNDED);
  e = mheap_elt_at_uoffset (v, h->first_free_elt_uoffset_by_bin[bin]);

  search_n_user_data_bytes = *n_user_data_bytes_arg;

  /* Silence compiler warning. */
  o0 = o1 = f0 = f1 = 0;

  h->stats.free_list.n_search_attempts += 1;

  /* Find an object that is large enough with correct alignment at given alignment offset. */
  while (1)
    {
      uword this_object_n_user_data_bytes = mheap_elt_data_bytes (e);

      ASSERT (e->is_free);
      if (bin < MHEAP_N_SMALL_OBJECT_BINS)
    ASSERT (this_object_n_user_data_bytes >= search_n_user_data_bytes);

      h->stats.free_list.n_objects_searched += 1;

      if (this_object_n_user_data_bytes < search_n_user_data_bytes)
    goto next;

      /* Bounds of free object: from f0 to f1. */
      //f0指向原始塊首部地址偏移，f1指向原始塊尾部地址偏移
      f0 = ((void *) e->user_data - v);
      f1 = f0 + this_object_n_user_data_bytes;

      /* Place candidate object at end of free block and align as requested. */
      //從尾部往左分配目標塊，o0指向首部偏移並且滿足對齊要求
      o0 = ((f1 - search_n_user_data_bytes) & ~(align - 1)) - align_offset;
      //o0比f0還小了，右偏移一個align
      while (o0 < f0)
    o0 += align;

      /* Make sure that first free fragment is either empty or
         large enough to be valid. */
      //確保切分後首部空閒塊至少有MHEAP_ELT_OVERHEAD_BYTES大小，這是要給切去的塊做頭部的
      while (1)
    {
      lo_free_usize = o0 != f0 ? o0 - f0 - MHEAP_ELT_OVERHEAD_BYTES : 0;
      if (o0 <= f0 || lo_free_usize >= (word) MHEAP_MIN_USER_DATA_BYTES)
        break;
      //越偏移lo_free_usize越小，什麼鬼，博主強烈認爲這是bug，應該是+=
      o0 -= align;
    }

      //o0~o1之間就是我們要找的內存了，大小，對齊性都滿足
      o1 = o0 + search_n_user_data_bytes;

      /* Does it fit? */
      if (o0 >= f0 && o1 <= f1)
    goto found;

    next:
      /* Reached end of free list without finding large enough object. */
      if (e->free_elt.next_uoffset == MHEAP_GROUNDED)
    return MHEAP_GROUNDED;

      /* Otherwise keep searching for large enough object. */
      e = mheap_elt_at_uoffset (v, e->free_elt.next_uoffset);
    }

found:
  /* Free fragment at end. */
  hi_free_usize = f1 != o1 ? f1 - o1 - MHEAP_ELT_OVERHEAD_BYTES : 0;

  /* If fragment at end is too small to be a new object,
     give user's object a bit more space than requested. */
     //從中間切開得到我們要的內存，尾部如果太小，就不單獨做成空閒塊了，直接給我們用。
  if (hi_free_usize < (word) MHEAP_MIN_USER_DATA_BYTES)
    {
      search_n_user_data_bytes += f1 - o1;
      o1 = f1;
      hi_free_usize = 0;
    }

  /* Need to make sure that relevant memory areas are mapped. */
  //切去的內存mmap下
  if (!(h->flags & MHEAP_FLAG_DISABLE_VM))
    {
      mheap_elt_t *f0_elt = mheap_elt_at_uoffset (v, f0);
      mheap_elt_t *f1_elt = mheap_elt_at_uoffset (v, f1);
      mheap_elt_t *o0_elt = mheap_elt_at_uoffset (v, o0);
      mheap_elt_t *o1_elt = mheap_elt_at_uoffset (v, o1);

      uword f0_page_start, f0_page_end;
      uword o0_page_start, o0_page_end;

      /* Free elt is mapped.  Addresses after that may not be mapped. */
      f0_page_start = mheap_page_round (pointer_to_uword (f0_elt->user_data));
      f0_page_end = mheap_page_truncate (pointer_to_uword (f1_elt));

      o0_page_start = mheap_page_truncate (pointer_to_uword (o0_elt));
      o0_page_end = mheap_page_round (pointer_to_uword (o1_elt->user_data));

      if (o0_page_start < f0_page_start)
    o0_page_start = f0_page_start;
      if (o0_page_end > f0_page_end)
    o0_page_end = f0_page_end;

      if (o0_page_end > o0_page_start)
    clib_mem_vm_map (uword_to_pointer (o0_page_start, void *),
             o0_page_end - o0_page_start);
    }

  /* Remove free object from free list. */
  remove_free_elt (v, e, bin);

  /* Free fragment at begining. */
  //切分後首部塊重新鏈入bin表
  if (lo_free_usize > 0)
    {
      ASSERT (lo_free_usize >= (word) MHEAP_MIN_USER_DATA_BYTES);
      mheap_elt_set_size (v, f0, lo_free_usize, /* is_free */ 1);
      new_free_elt (v, f0, lo_free_usize);
    }

  mheap_elt_set_size (v, o0, search_n_user_data_bytes, /* is_free */ 0);

  //切分後尾部塊重新鏈入bin表
  if (hi_free_usize > 0)
    {
      uword uo = o1 + MHEAP_ELT_OVERHEAD_BYTES;
      mheap_elt_set_size (v, uo, hi_free_usize, /* is_free */ 1);
      new_free_elt (v, uo, hi_free_usize);
    }

  /* Return actual size of block. */
  //分配的elt的數據部分大小
  *n_user_data_bytes_arg = search_n_user_data_bytes;

  h->stats.free_list.n_objects_found += 1;

  //返回分配的elt的數據部分偏移值
  return o0;
}

user_data_size_to_bin_index (uword n_user_data_bytes)
空閒mheap_elt_t按照數據部分大小分類鏈接到不同slot中。該函數計算bin值。

/* Find bin for objects with size at least n_user_data_bytes. */
always_inline uword
user_data_size_to_bin_index (uword n_user_data_bytes)
{
  uword n_user_data_words;
  word small_bin, large_bin;

  /* User size must be at least big enough to hold free elt. */
  //n_user_data_bytes至少要能包含mheap_elt_t頭部
  n_user_data_bytes = clib_max (n_user_data_bytes, MHEAP_MIN_USER_DATA_BYTES);

  /* Round to words. */
  n_user_data_words =
    (round_pow2 (n_user_data_bytes, MHEAP_USER_DATA_WORD_BYTES) /
     MHEAP_USER_DATA_WORD_BYTES);

  ASSERT (n_user_data_words > 0);
  //除去頭部的純數據部分長度，換算成佔用word個數，即爲bin值
  small_bin =
    n_user_data_words -
    (MHEAP_MIN_USER_DATA_BYTES / MHEAP_USER_DATA_WORD_BYTES);
  ASSERT (small_bin >= 0);

  large_bin =
    MHEAP_N_SMALL_OBJECT_BINS + max_log2 (n_user_data_bytes) -
    MHEAP_LOG2_N_SMALL_OBJECT_BINS;
  //1 ~ MHEAP_N_SMALL_OBJECT_BINS每個值對應單獨bin
  //>=  MHEAP_N_SMALL_OBJECT_BINS多個值映射到一個bin中。
  //可見，分配內存越小，越能精準快速定位到可選空閒塊。
  return small_bin < MHEAP_N_SMALL_OBJECT_BINS ? small_bin : large_bin;
}

vector
實際使用的緩存一般是如下結構：

user_header指具體數據結構頭部，比如：mhash_t，pool_header_t等等

思科VPP源碼分析（內存管理）

TDengine docker安裝方法

vue項目獲取富文本編輯器wangEditor內容導出爲word（html轉word格式並下載）

dotnet C# 創建 X11 應用時設置窗口背景顏色

vue3組件通信與props

sapui5

Alpine Linux apk add DNS lookup error

部分JDK版本的發佈時間

工作中用到的腳本合集

合併代碼時Beyond Compare設置

Navicat安裝與激活教程

將man出來的信息保存到文本

OpenSSL 常用函數——Base64編碼及解碼和證書操作

Open SSL 常用函數——簽名與驗證

一個fwrite的錯誤

Server開發(deamon)進程排他性(文件鎖),腳本排他性(fuser)

https://yachay.unat.edu.pe/blog/index.php?comment_area=format_blog&comment_component=blog&comment_co

linux以太網驅動總結