618 技術特輯（二）幾百萬人同時下單的秒殺，爲什麼越來越容易搶到了

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"​​摘要：幾百萬人同時下單，你搶到的概率爲何越來越大？電商企業又是如何掌握大促期間上億的銷售數據？","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文分享自華爲雲社區","attrs":{}},{"type":"link","attrs":{"href":"https://bbs.huaweicloud.com/blogs/279700?utm_source=infoq&utm_medium=bbs-ex&utm_campaign=other&utm_content=content","title":"","type":null},"content":[{"type":"text","text":"《618 技術特輯（二）幾百萬人同時下單的秒殺，爲什麼越來越容易搶到了？》","attrs":{}}]},{"type":"text","text":"，原文作者：技術火炬手。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"把心儀的商品加入購物車，點擊結算，付款，商品頁面的銷量+1，庫存量-1，如果只有幾十個人下單，系統輕鬆應付，但是當這個數量變爲百萬、千萬，就是另一番情形了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"電商企業在大促期間面臨流量訪問壓力：網站的訪問用戶激增讓單服務器超負荷運行，從而導致網站訪問卡頓或失敗，嚴重影響用戶體驗，","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"彈性負載均衡服務可以輕鬆解決這個難題。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"彈性負載均衡：哪裏壓力大往哪裏搬","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"彈性負載均衡（ElasticLoad Balance，簡稱ELB）是將訪問流量根據轉發策略分發到後端多臺服務器的流量分發控制服務，華爲雲的ELB就可以通過流量分發擴展應用系統對外的服務能力，並通過消除單點故障提升應用系統的可用性。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"面對不同的電商業務需求，ELB可以靈活處理。舉個例子，對於業務量訪問較大的業務，可以通過ELB設置相應的轉發規則，將訪問量均勻的分到多個後端雲服務器處理；對於存在潮汐效應的業務，可以隨時在ELB上添加和移除後端雲服務器，比如在大促第一波高潮前增加服務器，到了中期再移除，最後收尾的時候重新加上。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這樣既能保證大促期間，商品頁面不會因爲過多訪問而阻塞，也能控制成本的開支。但運營人員如何知道什麼時候添加/減少服務器，什麼時候爲ELB設置轉發規則，就得對整個促銷的節奏以及自家產品數據有基本的預判。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"所以，大促期間，除了訪問流量壓力，更讓人頭疼的則是數據的實時分析管理能力。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"數據倉庫：有貨沒貨，賣了多少單，系統門清","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通常情況下，電商數據主要分兩塊：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第1類是面向交易的訂單、商品、活動、發貨等數據；","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第2類是平臺運行的實時日誌及用戶在平臺上活動產生的行爲數據。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果企業無法應對大規模在線交易及“實時分析”的業務要求，這“618”這樣的大促中就無法掌握主動權。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"舉個例子，某運動服飾類電商企業原有系統採用傳統解決方案，交易和BI相互獨立。交易平臺採用分佈式中間件+單機版數據庫搭建。由於該方案不具備數據的強一致性能力，在同一時刻系統中數據可能是不完整、不準確的，比如一分前已經銷售了100單，但庫存沒有變化，最後爲銷售對單帶來極大困難。爲保證數據的最終一致性，交易系統數據需要通過ETL工具時隔數小時後同步到BI系統，無法做到實時分析，銷售及運營主管無法實時掌握經營情況。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解決這個問題，他們後來採用了華爲雲混合負載數據倉庫DWS。DWS採用“一庫兩用”的設計理念，一套數據倉庫集羣既可以支持超高併發、低時延的業務交易請求，同時可支撐複雜的海量數據分析和BI應用，減少開發和運維成本。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"相比於原系統，BI系統時效性大大提高，且數據分析性能提升3倍。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ac/ac8e577c56f1bb5d10422b0035870bd7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"做到數據實時一致的同時，DWS也確保了對單數據的準確，以及運營報表的“零”時延。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"DWS小規模可支持萬級TPS的寫入能力，橫向擴展後可達百萬至千萬級TPS，支撐該電商企業平穩度過“雙十一”、“雙十二”。而且由於它原生具備分佈式事務ACID特性，具有數據的強一致性保證，在任何時刻均可保證交易系統數據的準確性、完整性，確保對單數據無誤。再加上DWS的高可用架構設計，提供自動化數據備份功能，可靠性達到小數點後的11個9，保障業務數據不丟失。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"數據可視化：銷售有沒有破紀錄，就看它了","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當我們把不同系統模塊數據整合到數倉中，讓它進一步清洗、整合、規則處理，實現實時、完整以及準確的數據後，還需要通過數據可視化平臺推送到產品及運營人員進行可視分析。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"華爲商城之前就將電商大數據應用由TIDB+Spark集羣搬遷到基於華爲雲以DWS數據庫爲核心的數倉平臺中。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在數據的實時集成方面，華爲雲提供了基於Flink、Spark兩種流計算引擎的CS流計算服務。隨着Flink社區的活躍及FlinkSQL對應用開發者極大的門檻降低，通過SQL的形式即可實現流計算。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"華爲雲將通用的消息中間件封裝成對應數據輸入的source算子，邏輯層用SQL作爲表示，各種數據存儲平臺作爲sink算子封裝。","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"如果用原生Flink API的方式，開發及發佈一個簡短的輸入輸出任務，起碼2小時纔可以完工，但現在一站式的流計算平臺10分鐘即可完成。","attrs":{}},{"type":"text","text":" 而且進行語法校驗後，絕大多數問題都可以在這個環節規避，發佈之後，就能通過數據可視界面，查看其輸入輸出的數據信息進行校驗監控。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"​","attrs":{}},{"type":"text","text":"BI數據可視模塊選擇的是華爲雲DLV服務，以大屏場景爲主。華爲商城某次大促活動需要臨時加一個指揮大屏看數據，2個開發人員加了2小時班，一個做前臺，一個處理數據邏輯，很快完成了0.1版本併發布，之後再持續迭代優化效果。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/76/76ea35e1d364b3bd0bc91c28806d0f51.jpeg","alt":"圖：某企業經營數據看板","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖：某企業經營數據看板","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"​","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"核心數倉裝備之外，DAYU數據開發套件平臺可極大提升發佈效率，DAYU以數據調度平臺爲核心作爲擴展，融合了數據監控、元數據庫管理、數據服務發佈，這些服務華爲商城大數據平臺都有逐步使用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"簡單介紹下其中開發人員最常用的DLF（調度平臺），以一個數據API發佈來說，開發人員先在DLF上開啓一個Job，拖入三個任務：CDM（將TIDB任務集成到DWS）、2個DWS SQL（一個做DWR層規則處理，一個做數據DWD結果呈現），然後","attrs":{}},{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"將結果表數據通過DAYU的DLG服務進行API發佈，其他領域即可進行調用。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"電商企業上雲後，解決了基礎設施的難題，得以把更多的精力聚焦在業務邏輯開發上，不用再考慮服務負載均衡，容災等等，也降低了運維的負擔和人力成本。對於消費者來說，大促期間的搶購體驗更好，流量高峯期下單也能搶到心儀的商品。","attrs":{}}]},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"最後","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"電商在爭奪消費者的戰爭中，技術的進步至關重要。從消費需求的產生，到購買行爲的深入，從供應鏈到平臺交易數據的分析管理，從直播帶貨的低時延到業務數據的安全，雲計算、AI、大數據等技術的迭代，一步步改變在線購物的模式，也定義了新的商業模式。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"曾經，AR/VR技術在內容呈現和消費者互動上提供了新的可能性，隨着5G技術的成熟，以及經歷前期高投入和試錯後，模擬真實場景的VR購物，或許會讓足不出戶邊逛邊買成爲一種常態化的購物模式。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在消費者的另一端——智慧物流側，IoT、邊緣計算、機器視覺、無人駕駛，這些技術已經在改變傳統的物流倉儲和運送體系，從自動化立體倉儲、自動輸送、自動分揀到機器人作業，規整統一的自動化操作提高了運作效率。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"618是消費者購物狂歡的節日，它對電商企業也是一次大考，創新高的業績背後考驗他們的技術實力。換個角度看，618在拉動內需的同時，也拉緊了技術創新、產業升級背後的那根繩子。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://bbs.huaweicloud.com/blogs/278907?utm_source=infoq&utm_medium=bbs-ex&utm_campaign=other&utm_content=content","title":"","type":null},"content":[{"type":"text","text":"618技術特輯（一）不知不覺超預算3倍，你爲何買買買停不下來？","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果你想了解爲什麼每當大促的時候，我們的錢包總是被掏空？這背後到底是大家自制力不夠，還是電商平臺太會讀懂人心，我們不妨從技術維度，抽絲剝繭一探究竟。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https://bbs.huaweicloud.com/blogs?utm_source=infoq&utm_medium=bbs-ex&utm_campaign=other&utm_content=content","title":"","type":null},"content":[{"type":"text","text":"點擊關注，第一時間瞭解華爲雲新鮮技術~","attrs":{}}]}]}]}