使用 Amazon Kinesis 快速構建流式數據分析架構

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"簡介"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon Kinesis 可讓您輕鬆收集、處理和分析實時流數據,以便您及時獲得見解並對新信息快速做出響應。Amazon Kinesis 提供多種核心功能,可以經濟高效地處理任意規模的流數據,同時具有很高的靈活性,讓您可以選擇最符合應用程序需求的工具。藉助 Amazon Kinesis,您可以獲取視頻、音頻、應用程序日誌和網站點擊流等實時數據,也可以獲取用於機器學習、分析和其他應用程序的 IoT 遙測數據。藉助 Amazon Kinesis,您可以即刻對收到的數據進行處理和分析並做出響應,無需等到收集完全部數據後纔開始進行處理。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實際上,Amazon Kinesis包含以下四個服務:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon Kinesis Video Streams – 利用 Amazon Kinesis Video Streams,您可以輕鬆而安全地將視頻從互聯設備流式傳輸到 AWS,用於分析、機器學習 (ML) 和其他處理"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon Kinesis Data Streams – Amazon Kinesis Data Streams 是一種可擴展且持久的實時數據流服務,可以從成千上萬個來源中以每秒數 GB 的速度持續捕獲數據。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon Kinesis Data Firehose – Amazon Kinesis Data Firehose 是將流數據可靠地加載到數據湖、數據存儲和分析服務中的最簡單方式。該服務可以捕獲和轉換流數據並將其傳輸給 Amazon S3、Amazon Redshift、Amazon Elasticsearch Service、通用 HTTP 終端節點和服務提供商(如 Datadog、New Relic、MongoDB 和 Splunk)"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon Kinesis Data Analytics – Amazon Kinesis Data Analytics 是通過 SQL 或 Apache Flink 實時處理數據流的最簡單方法,您無需瞭解新的編程語言或處理框架。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本篇文章將完整展示如何使用Kinesis構建流式數據分析架構,如下圖所示:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/c8\/bf\/c84fde1cfaf45db4ac6cd2a73bb96ebf.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"提前準備"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"進入Demo之前,先提前創建好名字爲kinesis-demo-us2的S3存儲桶,創建過程可以參看:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"https:\/\/docs.aws.amazon.com\/quickstarts\/latest\/s3backup\/step-1-create-bucket.html"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然後,再創建Redshift集羣,具體步驟可以參看:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章