資源數據治理的應用實踐

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"導讀：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"資源數據是運營商核心資產和重要的生產要素，運營商業務的售前、售中、售後都對資源的數據高度依賴。資源數據的質量好壞，直接影響業務的拓展和保障。無論是電信、聯通還是移動，每年在資源數據治理專項工作上都有很大的投入，但是效果都不夠顯著。如何保障和提升資源數據質量，已經成爲資源系統最關鍵的任務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文將基於我們對資源數據治理基本方法的理解，分享我們在多個資源數據治理項目上進行的不同實踐。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"資源數據治理基本方法","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何進行資源數據治理？每個運營商省份資源系統建設階段、管理專業範圍和數據質量情況不同，所以在數據治理方法和手段上不盡相同。各運營商不斷加快網管系統和資源系統的融合節奏，在一些重要專業如PON、傳輸、無線、數據專業的融合尤爲迫切，所以網資融合、比對拼接是針對有源資源的一項有效的數據治理方法。針對管道、杆路、光纜、啞設備等無源資源的數據治理，更多的需要藉助數據清查、人工覈查等來進行長期的治理，需要引入一些創新的IT應用嵌入到治理場景中來實現數據智能修復。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"無論是無源和有源的數據治理，藉助IT工具手段進行數據治理，能獲得數據質量提升的局部成果或者階段性成果，長期的數據治理，或許還需要運營商從企業管理角度上進行管控或者變革，要從數據的源頭把控好，實現數據的閉環管理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"實踐一：基於傳輸鏈路告警的光路數據治理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"移動市場A省份近年來大力發展政企客戶業務，越來越多的政企客戶需要提供高質量的專線產品服務，傳輸專業維護室壓力極大，傳輸專業資源中設備光口至ODF端口、光纜、跳纖、光纜成端等數據都是靠人工收集、手工錄入，且內外線操作人員一般都是兩撥人，數以萬計光纜資源數據錄入的準確率與完整性很低，原始的人工覈查效率低、準確性差。爲此A省份建設傳輸運維智慧平臺系統，打造基於傳輸鏈路告警的光路普查應用，實現光路資源數據治理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ed/edb34448d0acca35117153a45c589c09.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"光路普查工作中主要解決光交等設施的大量AZ端子跳接錯誤數據問題，核心解決思路是通過現場拔插光纖，打斷光路通道，造成電路告警，現場普查人員對光交尾纖一拔一插，手機APP會在30秒內獲取到告警和去告警信息，兩個信息相互驗證，獲得準確的AZ網元端口信息，形成光交A端子、AZ網元端口的關係。通過多次的拔插，系統智能匹配AZ端網元端口、光交A端子和AZ端網元端口、光交Z端子數據，自動修復光交A端子和光交Z端子的跳接關係。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"現場拔纖：","attrs":{}},{"type":"text","text":"通過APP掃描光交二維碼快速調出設備面板圖，自動帶出AZ端網元，選擇具體光交端子進行拔纖，對接告警平臺，分析相應網元是否有電路斷開的告警；插上成端的光纖，確認告警是否恢復正常。自動關聯告警電路與所拔光纖、光交端子的關係。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/86/86341c3ee6394440fe7e91e82578718c.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"智能分析：","attrs":{}},{"type":"text","text":"隨着大量的普查工作開展，累加大量的撥測數據。針對相同的告警設備端口信息，進行自動計算出路由正確跳接關係，逐步拼接一條完整的光/電路鏈路路由。","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/af/af78383ac16e060d2f10333593bdc899.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"路告警的光路普查運行3個月，移動運營商A省的光路完整率由原來低於10%提升至50%，光路和電路關聯準確性由原來不到30%提升到60%，各地市光路普查率最低的也提升到50%以上，最高的提升到80%。花費在普查上的人力成本也大大降低，試點地市的光路普查原來需要3個月完成的工作，現在1個月就完成了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"實踐二：基於網資融合的資源數據治理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"電信運營商B省份資源系統建設多年，系統能力不足，資源數據不準，後端維護服務手段落後，導致承接集團重點工作明顯喫力，不足以支撐集團關於智慧運營的轉型要求，2019年借公司IT系統架構升級建設統一網元庫系統。通過統一網庫系統的建設，融合傳輸、無線、數據等核心專業網管數據和資源系統數據，有源數據進行比對、拼接，無源數據進行稽覈，實現各專業以及跨專業端到端拓撲數據呈現，實現資源數據全面治理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/94/94c283d0da660c5eb8d90abc39ab35ab.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"統一網元庫融合網管和資源的數據，通過數據拼接實現網絡路徑還原，通過比對實現數據的清洗，從數據採集、轉換、拼接、比對、同步全流程，實現網資數據全面融合。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"數據採集：","attrs":{}},{"type":"text","text":"全量和增量數據採集，支持多種格式採集","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"數據轉換：","attrs":{}},{"type":"text","text":"源數據轉換成目標數據，關聯關係、唯一標識處理，數據拼接、結構化展示，網管數據和資源數據拼接網絡鏈路。數據轉換支持配置化處理，對於通用的轉換對象及其屬性通過屬性拖拉可視化配置，對於需進行關聯計算或複雜邏輯處理的屬性，支持java腳本、python腳本、大數據等插件配置","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/24/2434d7adcf601e1b4fbf1ab1909f89ce.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"數據比對：","attrs":{}},{"type":"text","text":"比對規則管理、比對方式管理（正向比對和反向比對）、比對報告輸出。對各專業對象及屬性進行靈活的業務包配置，比對成功後生成關鍵指標的可視化報告","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"稽覈：","attrs":{}},{"type":"text","text":"定義稽查規則，自動稽查異常數據，輸出可視化稽覈報告","attrs":{}}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"拓撲呈現：","attrs":{}},{"type":"text","text":"融合各專業物理、邏輯、業務數據，生成物理拓撲、邏輯拓撲、業務端到端拓撲","attrs":{}}]}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/86/864bbff1e28f44b298437b97c1e09e58.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"統運行1年來，傳輸、數據、無線專業的網管和資源側的設備、機框、插槽、板卡、端口的一致率經過1年多由70%左右提升到94%，資源系統數據質量得到顯著提升，通過統一網元庫治理後的數據共享，實現集中監控等上層應用快速建設。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/de/de67bce1e60ea3981472d45d0df62391.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"設備框板槽匹配率","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"實踐三：基於責任體系+端到端建鏈的資源數據治理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"電信市場C省份多年來致力於提升資源系統數據質量，無論是優化生產流程，新增系統功能，用盡了各種招數，數據前清後亂，數據質量一直沒有明顯變化。客戶意識到資源數據治理，並非通過新建一個IT系統就能解決，資源數據治理還需要關注企業內部的業務管理模式。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"今年C省借政企兩線業務發展契機，進行企業內部管理改革，建立以雲網資源爲核心的一到五級縱向責任體系，劃分現場資源維護、遠程資源維護、資源運營分析、資源運營決策四類角色，將企業管理制度與IT技術手段相結合，打造資源責任到“人”，落實到“田”，責任制到“位”的資源管理運營體系。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲什麼要構建責任體系？爲了解決專業維護部門資源責任意識薄弱，導致資源存量數據不準、浪費運營成本，後端部門重故障搶修，輕資源運營的現狀。建立以資源責任體系爲基礎的雲網資源運營體系，各專業把資源運營作爲核心職責，企業員工要對資源準確率、資源運營效能負責。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如何構建責任體系？首先梳理責任人角色，根據資源管理的角色範圍差異，梳理出現場資源維護、遠程資源維護、資源運營分析、資源運營決策4大類，每類細化多種責任人角色。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然後建立責任人、責任田、責任制規則，劃分每種責任人負責的資源範圍，形成責任田。制定各類責任人考覈的評估指標，形成責任制。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f2/f23e7e80834afba22b256aaf5deee494.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後進行資源責任認領，無源設備通過GIS網格自動認領，有源設備通過人工認領，建立完整的資源責任體系數據，把原有的資源存量數據與企業的員工形成關聯，並賦予企業的業務管理職責。有了這層管理數據，就可以通過企業組織管理制度來驅動資源數據質量提升。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/af/af5221594408096ec56945dfd4705951.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"構建了責任體系後，如何實現從發現數據問題到解決數據問題全流程閉環管理？先來看一下網絡資源的層級關係：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6b/6b00c2145c463a426bb6bfb7f3f1c839.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"資源系統數據是複雜的分層分網絡的結構，任何一層都會形成完整的鏈路關係。接下來通過進行端到端建鏈，來自動判斷全程鏈路的數據是否完整。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"端到端建鏈就是分析資源網絡鏈路數據連接是否完整、資源屬性數據是否正確的全過程，根據配置的幾百條業務稽覈規則，系統自動進行業務數據稽覈，建鏈不成功的數據或者錯誤的資源數據，通過責任體系關係查找到對應的資源責任人，系統自動派單到人，責任人收到錯誤的數據清單明細，進行數據修復處理，處理完成後進行回單，回單需要進行錯誤數據修復實時稽覈校驗，如果數據錯誤修復完成，則回單成功，否則無法回單，責任人需要進一步修復錯誤數據。通過從資源數據問題發現到數據問題修復，實現數據修復閉環管理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9c/9c9120002e1c64c3b6eb0c2733995544.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於責任體系+端到端建鏈模式的資源數據治理處於建設初期，還沒有足夠的數據來體現業務收益，企業管理的變革帶來的成效還需要時間去檢驗，不過可以拭目以待，相信在不久的將來客戶會取得想要的收益。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":"br"}},{"type":"heading","attrs":{"align":null,"level":1},"content":[{"type":"text","text":"後續期待","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"資源數據治理是企業數據治理範疇的一種，是體系化工程，運營商針對資源數據治理越來越重視，未來將會從組織、工具、管理、制度等方面體系化開展數據治理工作。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"運營商對支撐廠家的要求也會從傳統提供數據治理工具，逐步演進到一起實施資源數據治理大工程，一起探討如何實現運營商內部業務管理模式改革，帶來資源數據質量的長效提升。我們也可以藉助在不同的資源項目上和客戶一起實踐和探索新型資源數據治理方法，爲運營商的發展做出更大的貢獻。","attrs":{}}]}]}