FreeWheel實時數據系統彈性伸縮實踐

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"1.引言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着大數據技術的不斷髮展,數據實時性的需求變得越來越迫切,這對實時數據處理的基礎架構提出了更高的要求。如何應對實時數據的流量變化,特別是突發流量,成爲實時數據處理架構不得不面對的挑戰性問題。對於FreeWheel這樣一家服務全美90%的主流電視媒體和運營商的視頻廣告投放和管理平臺,對於突發流量的應對能力尤爲重要。數據基礎架構團隊作爲FreeWheel數據處理的載體,提供了公司級的實時和離線數據處理及查詢服務。在本文中,我們將和大家分享我們在實時數據處理彈性伸縮方面的實踐。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"2.FreeWheel實時數據平臺架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"FreeWheel"},{"type":"text","text":"數據基礎架構基於大數據開源軟件和部分自研軟件構建,所有服務均部署在"},{"type":"text","text":"AWS EC2"},{"type":"text","text":"上。目前,整個數據平臺爲"},{"type":"text","text":"Lambda"},{"type":"text","text":"架構,並逐漸向流批一體的融合平臺發展。其中,實時數據平臺整體上主要分爲四個部分:基於"},{"type":"text","text":"Kafka"},{"type":"text","text":"構建的消息隊列,自研的消息處理中間件"},{"type":"text","text":"Matcher"},{"type":"text","text":",基於"},{"type":"text","text":"SQL"},{"type":"text","text":"開發方式的實時計算平臺"},{"type":"text","text":"SQL on Streaming(SOS)"},{"type":"text","text":"以及以"},{"type":"text","text":"Druid"},{"type":"text","text":"爲主體構建的實時數據查詢服務。"},{"type":"text","text":"  "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/dc\/64\/dcfaec4bb9yy5e11b43867816d647664.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章