接口需求HiveMysqlRedis性能要求請求QPS>1s<1√√√<1s<10
| √√<500ms>100
|
| √<100ms>100
|
| √"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這就造成了機票大數據部門的redis集羣內存需求暴漲，目前我們統計redis使用的數據：掛在機票大數據部門的redis集羣數量有幾十個，內存達到了十幾個T。當然接口的性能也達到了前所未有的快速和高效，基本都是10ms左右。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.2 如何查詢"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis的查詢方式比較單一：通過唯一key去查詢value。這種查詢方式在簡單的唯一值查詢中比較有效，但是當遇到，同一個數據源多關鍵字查詢的時候，就得維護多份數據源。舉例：在價格趨勢的接口中，我們提供了多種價格趨勢組合：國內、國際、單程、往返、航線、航班。如果使用redis存儲，需要維護同一份數據多種key的存儲方式，極大地浪費了存儲空間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Redis還有一個問題是時間範圍的篩選，還是在上面的價格趨勢接口中，需要按照查詢時間返回歷史同期在一定起飛時間範圍的價格數據，所以我們需要存儲多個時間日期的數據（當然也可以用set等結構，但是會面臨如何刪除過期數據的問題），同時在查詢的時候需要循環取一定時間範圍的價格。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.3 如何維護"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"1）接口維護"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大數據基礎平臺團隊一共維護了幾百個接口，其中1\/3的接口是提供數據給調用方的，這當中又有一些接口只是提供簡單的查詢操作，但就是這些簡單的查詢，需要我們提供海量的數據存儲、快速精準的查詢。每個接口的上線需要經過項目資源申請（包括機器資源、人員資源）、數據同步、開發、測試流程，最後才能上線。一整套流程走下來，耗費2-3天\/人，而且基本上都是是重複性的工作。如何解放這些人力和機器資源，就變得很迫切了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"2）數據同步"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"提供給外部使用的數據大部分都是存儲在hive中，在不使用presto api的方式訪問時，我們需要將hive數據導入到redis或者mysql中，供接口訪問。在zeus平臺上，我們建立了各種導數據的流程，如何將這些簡單、重複度高的流程自動化呢？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整個接口的架構圖如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/24\/24c5923a25368dcc09a63bae23ec92a8.jpeg","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖1 redis\/mysql作爲主要存儲的架構圖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"三、機票大數據接口的大道之旅"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"認真研究了接口調用方本身的性能，我們發現調用方在調用第三方提供的接口時，基本都是異步進行的。如果把調用方調用的所有第三方接口當成一個木桶，機票大數據基礎架構團隊的接口就是其中的一塊木板，只要不是最短的木板，就可以在保證性能的情況下降低整個接口的響應時間（當然這不是技術上的退步，而是選擇合適的方案）。通過上面的存儲選型對比之後，發現在100ms-500ms這個性能段裏面沒有一個合適的存儲方案能夠提供。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們調研了幾種NOSQL數據庫方案，綜合存儲、查詢等指標發現CrateDB比較符合現實需求。將幾種存儲做了一個對比，如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"embedcomp","attrs":{"type":"table","data":{"content":"對比RedisMongoCrateDB查詢速度<10ms100ms~500ms100ms~500msSQL不支持不支持支持數據結構化不支持支持支持存儲機制hashSharding+partitionSharding+partition資源利用內存資源硬盤+內存硬盤+內存數據可重複使用不支持，單一固定key支持支持"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 CrateDB介紹"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CrateDB是構建在NoSQL（ElasticSearch）基礎之上的分佈式SQL數據庫，它結合了SQL的通曉程度和NoSQL的可擴展性和數據靈活性："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"a、使用SQL處理結構化或非結構化的任何類型的數據"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"b、以實時速度執行SQL查詢，甚至JOIN和聚合"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"c、簡單縮放"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2 CrateDB與接口存儲"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CrateDB很好地解決了100ms-500ms性能段的短板，並且使用磁盤+內存的方式存儲數據，減少了內存的使用。目前在我們生產時間中，通過12臺8核24G虛擬機30%的磁盤空間覆蓋了10億數據（如果是redis至少需要300G的內存，如果做slave，容量double）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3 CrateDB與接口查詢"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CrateDB提供瞭如MYSQL的表、字段等概念（底層使用ES存儲引擎），我們可以將同一份數據源進行多維度的操作，比如上述講到的價格趨勢裏面基於航線和航班的價格趨勢，這兩個接口可以使用同一套數據源，因爲航線的價格可以基於航班數據進行聚合操作，這樣就大大減少了冗餘的數據。同時類MYSQL表的特性使得時間範圍的查詢變的so easy了。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.4 CrateDB與接口維護"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"1）與接口結合使用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因爲CrateDB支持標準的SQL，我們開發了機票大數據基礎平臺的通用性api系統，通過將取數邏輯SQL化的方式，同時利用qconfig api將新增的數據需求進行模板化、配置化，統一了接口代碼開發的流程。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"配置頁面如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/66\/669c26a6518c3b07e0814bd670cc86e1.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖3 接口配置頁面"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"2）數據同步"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過zeus api將同步數據流程模板化，配置頁面如下圖。並且在zeus平臺上，使用spark shell方式將hive數據導入到CrateDB中，拋棄了以前jar包的方式。這種方案可以在幾分鐘內導入千萬級的數據（取決於CrateDB表的數據結構，減少索引、doc_values以及刷新間隔會都有利於導入的速度）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/66\/66186d3ad351eb6408915abb90f65a5e.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖4 zeus流程配置頁面"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"3）容器化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何更加有效地管理、維護CrateDB集羣？爲此我們上了k8s，將CrateDB容器化。爲了更好地管理這些k8s集羣，引入了rancher，rancher是開源的企業級容器管理平臺，通過rancher，我們再也不必自己去從頭搭建容器服務平臺。同時rancher提供了在生產環境中使用的管理docker和kubernetes的全棧化容器部署與管理平臺。將網絡、磁盤虛擬化之後，資源的利用率大大提高，減少了虛擬機的使用。自動水平擴展，以及pod的監控等特性，都極大地提高了維護CrateDB的能力，我們管理的CrateDB集羣如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/79\/79e0c98d003358d52fe3aa87b36279f3.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖5 rancher管理CrateDB集羣圖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.5 與接口結合的其他優勢"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1）存儲機制多樣化，底層的存儲機制支持多樣化的數據類型，同時支持partition、sharding；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2）數據結構化，CrateDB提供結構化的展示，有利於數據的可視化以及降低非技術人員的理解難度，解決了redis可讀性差的問題；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3）存儲可靠性，數據持久化存儲在磁盤上，支持replica，相比於redis的內存存儲更加可靠（當然redis也可以落盤，但這就會限制redis的速度）；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4）成熟的優化機制，針對es的優化我們有豐富經驗的技術人員支持。舉個例子：我們有9000萬+的用戶行程數據，因爲數據比較詳細，字段的內容比較龐大。通過去掉部分字段的索引，去掉doc_values等操作將數據存儲大小從90G降到了30G，同時也提升了搜索速度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前在生產上我們部署了2個CrateDB集羣，其中一個集羣由12臺8核24G內存虛擬機組成。在集羣中建立了12個數據表，存儲了20+億條數據，經受了生產的實際考驗，接口性能指標如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"embedcomp","attrs":{"type":"table","data":{"content":"數據量99line95lineavg查詢特點描述10億+200ms80ms10ms多關鍵字、時間範圍查詢整個集羣請求量1500qps500w+150ms50ms10ms多關鍵字查詢、排序單個表請求量400qps9000w+200ms100ms60ms多關鍵字查詢| 單個表請求量10qps"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"性能滿足了大部分調用方的使用需求，同時系統數據上線的流程由以前的申請資源、開發代碼、測試、上線，到現在的系統配置、測試、上線，釋放了部分的開發資源，並且保證了數據的質量。接口上線時間由以前平均2-3天，縮短爲2-3小時。新的接口架構圖如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/1e\/1ebc38f2b23fd3bd3bbe3ce09dac9a3e.jpeg","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖6 CrateDB作爲主要存儲的架構圖"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"四、安裝部署"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"CrateDB有官方版以及社區版，爲了更好地進行自維護，我們選擇了社區版（通過源碼編譯）。CrateDB的部署與ES的部署基本一致。需要注意的是，在分配內存的時候儘量多留一些內存給系統，這將有利於數據查詢速度。部署後的webui如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/49\/49b1f4bcecfce06f1321e9ec52260f9d.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖7 CrateDB webUI"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"五、數倉中的實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前在數倉中的應用主要體現在各種指標dashboard、metrics的展示，比如fltinsight。與以往通過presto接口獲取數據的方式相比，更加直接、高效。而且CrateDB支持各種字段的聚合、統計，是各種指標存儲、展示的不二之選。當然後續數倉組也會在數據展示這一塊全面推廣CrateDB的使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"六、小結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"沒有完美的存儲方案，只有最適合的存儲方案。"},{"type":"text","text":"通過上述機票大數據平臺在數據存儲這一塊的實踐經驗，相信每個團隊在面對選擇存儲方案的時候，結合自身需求去選擇適合自己的存儲技術方案，達到“大道”。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"嘉賓介紹："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Loredp，攜程數據分析經理，關注大數據存儲、大數據處理以及linux等領域。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：攜程技術中心（ID：ctriptech）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/8mLllQZq6E0eePutkr_L1A","title":"xxx","type":null},"content":[{"type":"text","text":"乾貨 | CrateDb在攜程機票BI的實踐"}]}]}]} | | | | | | | | | | | | | | | | | | | | | | | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | |
| | | | | | | | | | | | | | | | | | | | |
{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着整個互聯網流量紅利進入末期，各大廠在着力吸引新客的同時，在既有客戶羣體的運營上也是煞費苦心，各種提高客戶體驗、個性化服務的場景層出不窮。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"攜程機票大數據部門在實踐過程中需要同步數據、選型引擎來存儲處理數據，利用接口將模型結果開放給生產環境調用，因此我們的數據存儲修煉之旅會涉及到"},{"type":"text","marks":[{"type":"strong"}],"text":"接口現狀、接口大道之旅、安裝部署、同步數據、生產應用以及未來的趨勢-如何實現容器化"},{"type":"text","text":"。這當中，我們遇到了很多問題，也解決了很多問題，本文將分享機票大數據平臺在數據存儲這一塊的實踐經驗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、機票大數據接口現狀"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"攜程機票大數據平臺接口組碰到的問題："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何存儲"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何查詢"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何維護"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.1 如何存儲"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"機票大數據基礎架構團隊接口組在2018年之前，數據的存儲方案基本是：hive、mysql、redis。以下是我們現有的存儲選型："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"embedcomp","attrs":{"type":"table","data":{"content":"