滴滴AR實景導航背後的技術

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1c/1c2e872a4da78c671ea4e9d5b26121b6.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/f5/f556549a01ae41b093f7f8ade671525d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"桔妹導讀："},{"type":"text","text":"機場、商場、火車站等大型室內場所內GPS信號不穩定、室內面積大、路線複雜、用戶判斷方向難等問題，給在大型場所內發單的乘客找上車點帶來了很大的挑戰，用戶急需一種操作簡單、交互友好的引導功能。本文講述了使用三維重建技術、傳感器計算技術和增強現實(AR)技術所開發的滴滴AR實景導航產品，並對開發過程中遇到的難點、挑戰和解決思路展開介紹。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"相信很多人都有過這樣的經歷：來到一個自己不熟悉的場景，特別是在一些GPS信號不準確的室內場所，很難找到建築物內部的一些特定地點。本文將以幫助用戶在大型機場等場所中快速找到上車點爲出發點，介紹滴滴AR實景導航產品研發過程中的挑戰和關鍵技術。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"1. "},{"type":"text","text":"應用背景"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7a26fb6a07b2693d87252e61271bf8ae.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們在用戶調研中發現，在一些大型的機場、商場、火車站內部，滴滴乘客在下單成功之後，往往需要更多的時間才能找到上車點，其主要原因是在這些大型的室內場所中，GPS信號不準確，而這些建築往往面積很大、內部路線複雜，當乘客對場景不熟悉時，找到上車點存在很大的困難。爲了解決這個問題，地圖團隊提出了“圖文引導”的方式來幫助用戶，通過特定場景的圖片和文字的方式相結合來引導用戶找到上車點。與此同時，我們也在持續探索是否有更加直觀、易理解的方式來幫助用戶，受到增強現實(AR)技術在遊戲中應用的啓發，我們提出了使用AR的方式來幫助用戶找到上車點，最終開發出了滴滴AR實景導航產品。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當乘客在支持AR導航的場站，使用滴滴出行App選擇推薦上車點發單成功後，可以通過產品界面中的AR按鈕進入導航界面，並按指引操作，體驗在AR元素指引下到達上車點。如下圖所示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3a/3a652a3e3e4afa1228404a66062a12ba.gif","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":10}}],"text":"鄭州機場"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"2. "},{"type":"text","text":"問題分析 "}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7a26fb6a07b2693d87252e61271bf8ae.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"想要給用戶提供一個良好的導航產品，需要解決幾個關鍵問題：第一，場景所在的地圖是怎麼樣的；第二，如何確定用戶的位置；第三，如何使用更加直觀的方式引導用戶走到目的地。這些問題，在通常的室外場景來看，可能都不是\"問題\"，因爲可以直接使用現有的地圖、GPS定位、規劃路線和GPS實時位置引導到達目的地。但是這些能力在室內場景下，卻變得很難。具體原因包括：在室內場景中，GPS信號受到建築物遮擋往往定位不準確，而現有的Wi-Fi、基站等定位技術也因場景中基礎設施情況的不同而表現出精度差異很大；同時，與室外相比，室內場景結構複雜度高、判斷方向難，給用戶引導也帶來很大的挑戰。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解決上述問題，給用戶提供更加友好的室內導航服務，我們推出了滴滴AR實景導航產品，其主要方案是採用低成本的視覺定位技術來提升用戶的定位精度，並結合增強現實技術來將引導信息顯示到用戶手機上，給用戶提供所見即所得的交互體驗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"3. "},{"type":"text","text":"技術挑戰 "}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7a26fb6a07b2693d87252e61271bf8ae.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"要想實現一個理想的AR導航系統，我們調研了很多技術方案，最終選擇了基於視覺的三維重建技術來解決地圖構建和路徑計算的問題、視覺定位技術來提供更高精度的定位能力以及傳感器位置推算與渲染技術來實現更加精確的AR交互顯示。雖然上述技術在學術研究領域已經有了很多年的研究，併產出了一些相對成熟的方案，但是在實施過程中，也遇到了很多挑戰："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"1、在三維重建方面"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於視覺的三維重建技術依賴相機拍攝到的場景圖像進行三維結構恢復，一般應用於辦公室、公寓等幾百平米的場景內。在機場、火車站、商場等超過幾萬平米的大型場景下，會存在人羣密集、重複紋理多、光照變化大、場景空曠、狹長通道等不利於視覺重建的因素，這種超大現實室內場景三維重建是業界難題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"2、在視覺定位方面"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"室內環境複雜多樣，室內空間佈局、拓撲易受人爲的影響，導致聲、光、電等環境容易發生變化，對於以特徵匹配爲基本原理的定位方法，定位結果將受到較大影響。機場、火車站、商場內大量重複出現的指示牌、廣告牌都極易產生誤匹配，影響定位的精度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"3、在傳感器位置推算方面"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於傳感器噪聲的存在，使得基於慣性傳感器的位置推算存在累積誤差。當長時間使用時，導致導航路線偏離正確路徑，嚴重影響着用戶體驗。此外，用戶行走行爲和手機硬件的多樣性，使得單一模型的慣性傳感器位置推算很難解決所有場景遇到的問題。因此，亟需提出一種模型自適應機制來提升導航系統的準確度和魯棒性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"4. "},{"type":"text","text":"關鍵解決方案 "}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7a26fb6a07b2693d87252e61271bf8ae.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解決上述問題，結合具體的應用場景以及技術積累，我們分別針對每個關鍵模塊進行了系列優化，現簡要分享一下。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"▍"},{"type":"text","text":"基於視覺的三維重建技術"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"三維重建解決的是室內地圖構建的問題。三維重建一般需要藉助運動恢復結構（Structure from Motion，SfM），其學術定義是「在未知的環境中，讓一個機器人能進行環境的構建，並且估計自身的運動」。用通俗的語言講，是利用相機拍攝到的圖像或視頻，恢復整個場景的三維結構。系統的輸入是多張圖像或視頻流，輸出是場景的三維結構和每張圖像拍攝的位姿。相機位姿是6自由度，3個自由度表示位置，3個自由度表示姿態（相機朝向）。位置可以理解成三維空間當中的一個點，姿態就是這個相機的朝向，如果在二維上就是360度的朝向，擴展到三維就是一個球體的朝向。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先我們瞭解下SfM的技術框架，一共四個主要步驟：①數據採集；②特徵提取；③數據關聯；④結構恢復。在恢復場景的三維結構時，並不是使用圖像中的所有像素點，而僅從圖像中提取穩定、顯著的點，也就是特徵點；數據關聯是確定有共視區域的兩張圖像中哪些特徵點是對應的；後續會藉助幾何的方法以及最優化技術（Bundle Adjustment）將這些圖像中的點恢復爲空間的三維點。整個問題最終被轉化爲大型非線性最優化問題求解，優化目標是重建的三維點重投影到圖像中的位置與圖像中的觀測點位置差最小，也就是最小化重投影誤差。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/60/6022c5b647c8ed4836cbda5206f0c10b.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對大型場站室內場景存在的人羣密集、重複紋理多、場景有變化等挑戰。我們設計了一種基於視覺的大型室內場景三維重建方案：針對大型機場、火車站存在的規模大、場景複雜（重複紋理、相似紋理、狹長通道、動態物體等），提出了一種基於視頻的分塊三維重建方案，首先構建圖像間的關聯圖，問題可以建模成Graph Cut問題，可以自動進行數據分塊；之後利用Pose Graph Optimization實現了塊間合併與優化，並通過引入關鍵幀、點雲選擇等策略，實現了自動化建圖，效率提升70%，構建的超過6萬平米的室內場站三維模型是業界已知的最大單體模型之一。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"▍"},{"type":"text","text":"視覺定位技術"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在AR導航的使用場景中，定位的目的就是確定用戶的位置。手機的定位源主要包含3大類：①導航衛星接收機：包括中國的北斗，美國的GPS，歐洲的Galileo、俄羅斯的GLONASS等；②內置傳感器：包括加速度計、陀螺儀、磁力計、氣壓計、光線傳感器、相機等；③射頻信號：包括Wi-Fi、藍牙、蜂窩無線通信信號等。除了衛星導航接收機外，這些傳感器和射頻信號都不是爲定位而設置的，儘管如此，這些傳感器還是爲我們提供了很多的室內定位源。主流的全球衛星導航系統（Global Navigation Satellite System，GNSS）目前雖然已經被大規模商業應用，在室外開闊環境下定位精度已能解決大部分定位需求，但該類信號無法覆蓋室內，難以形成定位。室內環境複雜，無線電波通常會受到障礙物的遮擋，發生反射、折射或散射，改變傳播路徑到達接收機，形成非視距（non line-of-sight，NLOS）傳播。NLOS傳播會使定位結果產生較大的偏差，嚴重影響定位精度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"經過充分的調研與對比，考慮到Wi-Fi指紋匹配方式定位精度2~5米，而藍牙iBeacon作用距離短且佈設成本較高等問題，最終我們採用了視覺定位的方式。該方法基於相機交互的方法，定位精度可達亞米級，魯棒性較好，且只需利用手機攝像頭、成本較低，無需佈設額外設備，而且隨着近年視覺定位技術的不斷優化迭代，其精度與魯棒性已完全能滿足室內定位的需求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"（1）基於傳感器融合的圖像檢索重排序技術"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通常情況下，用戶手機獲取一張查詢圖後，首先對圖像進行特徵提取，然後採用特徵描述子對特徵點進行描述，根據圖中提取的大量2D特徵點，會在當前場景三維模型中檢索相似的top N張圖像，根據最相似的圖像進一步尋找2D點與模型中3D點的匹配，最後利用RANSAC+PnP求解，計算查詢圖的位置與姿態[R|t]。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c0/c08d83aefb19ca031a8151b4799c780c.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是由於室內場景存在大量人造物體，例如機場、火車站中大量出現的指示牌，商場中大量出現的廣告牌，都很容易因爲局部特徵相似，導致錯誤的匹配。考慮到手機本身集成了磁力計和GNSS傳感器，雖然這些低成本的傳感器精度有限，但作爲定位初值還是可以加以利用，我們根據磁力計獲取的大致方位，以及GNSS獲取的大致位置作爲先驗知識，根據傳感器的精度指標，對候選圖像進行聚類，並將參數加權帶入圖像重排序（rerank）的計算公式，可以剔除具有明顯方向差異或者位置差異顯著的候選圖像。在降低誤匹配概率的同時，也顯著提高了計算的效率，保證了定位結果的精度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"（2）基於上下文信息的位姿校正技術"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於室內人造結構的特殊性，很多場景都是對稱修建或者呈規則排列的幾何佈局，導致在視覺定位的時候，不僅是局部特徵幾乎一致，甚至大範圍場景內的全局特徵也是非常相似的，這就導致初始定位存在一定概率的誤匹配情況。當用戶剛好位於此類區域的時候，很有可能會得到錯誤的引導信息。不過在實際情況中，這些容易混淆的區域不會完全一模一樣，當用戶移動一小段距離或者視角發生變化的時候，往往會出現具有顯著區分度的地物特徵，依賴這些信息可以對用戶當前的位姿進行校正，重新生成一條正確的引導路線。但是同樣是特徵匹配，當前後兩幀信息都滿足定位條件，但定位結果存在較大差異的時候，如何分辨究竟哪個是正確的結果呢。這時就需要利用到上下文的信息，即我們在定位的時候並不完全依賴於一次的定位結果，而是一片區域或者一段軌跡中的多個定位結果，當這些結果呈現出較好的一致性的時候，通常有更大的概率是正確的定位，相反，那些局部特徵相似的區域，由於匹配到了錯誤的參考圖像，其結果往往呈現出孤立的跳變，與前後幀並不能平滑過渡，行走軌跡也是不光滑的。利用這種定位結果的上下文信息，可以有效檢測出錯誤的定位結果，對其進行過濾，或者及時發現當前已經處於錯誤的引導路線上，根據正確的結果進行校正。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/51/51fc86b7ad573445fe3400d38c6f70d9.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"▍"},{"type":"text","text":"傳感器位置推算技術"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在完成視覺定位後，需要根據手機的位置進行實時的路線指引和渲染，這裏我們使用了基於傳感器的位置推算技術，通過讀取手機上的傳感器信息並結合人體運動模型等方法，推算出對應的位置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"慣性傳感器包括加速度計和陀螺儀，通常還會與磁力計組合使用。每個傳感器均具有三個自由度。加速度計測量施加給移動設備的加速度讀數，陀螺儀測量移動設備的旋轉運動，磁力計通過感知磁場的變化，推理出用戶當前的朝向。這些傳感器可以在沒有外部數據的情況下確定移動設備的位置，而不需要基礎設施的輔助。基於慣性傳感器的位置推算由於以下優點逐漸成爲研究的熱點：①低能耗全天24小時運行；②不受外界環境干擾；③內置於每一部移動智能終端。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現有的基於積分的慣性導航算法主要利用線性加速度計讀數（加速度計讀數減去重力數據）進行二次積分得到位移和陀螺儀讀數積分得到姿態，繼而解算當前位置。然而，慣性傳感器具有噪聲，導致慣性積分算法估計的位置很快飄移。因此，當前慣性積分算法主要應用於高精度慣性器件中，例如航天級別的慣性傳感器。爲了解決在低端手機中慣性傳感器的迅速飄移問題，研究者們提出了PDR (Pedestrian Dead Reckoning) 算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PDR (Pedestrian Dead Reckoning) 算法是一種基於相對位置的位置推算方法，主要思想是給定初始位置和朝向，根據行走的位移推算出下一步的位置。主要包括計步、步長估計和朝向估計模塊，如下圖所示。計步主要是通過檢測加速度計讀數的峯值點實現步子識別；步長估計利用一步之內的加速度計讀數，根據人的身高、體重、步頻等特徵擬合當前一步的步長；朝向估計利用加速度計和陀螺儀讀數，通過濾波或者優化技術實現設備的姿態估計，繼而獲取當前一步的行走朝向。由於上述每一個模塊都會有誤差，且對不同的人羣和設備的多樣性比較敏感，隨着時間的推移，誤差累積增大，直至導航系統不可用，爲AR引導帶來了技術挑戰。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/12/125e221ffb6ae57b41d8f622ab95f6bd.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解決上述技術挑戰，我們提出了魯棒的位置推算算法，該方法發表在機器人領域頂級會議International Conference on Intelligent Robots and Systems (IROS) 2020上。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"所提出的方法主要包括基於步態強度的計步算法，基於步頻和統計特徵的步長估計算法，基於深度模型的朝向迴歸算法和基於機器學習的運動分類算法。下面介紹一下相關算法。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/1b/1bd419f4eb8f7b12d9bdcfbd32548ea9.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":5},"content":[{"type":"text","text":"（1）基於步態強度的計步算法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"給定三軸加速度計讀數，首先對每一幀加速度計讀數求取幅值。利用當前幀加速度計的幅值和其鄰近數據，檢測加速度計讀數波形的峯值，每一個峯值對應着候選步子，如下公式所述："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ec/ec21cd223c7c8d7f3585ee434431f4d5.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中，S表示當前加速度計讀數的狀態，即是否爲候選步子，如果爲1，則表示當前時刻檢測到候選步子。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其次，根據長短窗內的加速度計讀數的均值來判斷當前時刻是否爲候選步子，其下公式所述："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d1/d145aa2dd1532c357ecd8cc7eee24427.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a4/a4a13be108331672795f1478f39ba59a.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"表示短窗口內的加速度計讀數的均值，"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a3/a3e2ba7f9b60e671d612936b8ae424f8.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"表示長窗口內的加速度計讀數的均值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再次，根據加速度計數據計算步態強度，它描述了一步之內加速度的變化情況。當步態強度超過一定闕值後，我們認爲當前時刻是候選步子："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2e/2eb86b6be14625447e2d3edaf981b888.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中，"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a9/a93143dd993ec40012ddaf13bdf82dac.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下圖展示了真實行走狀態下步態強度的變化情況，其中藍色點表示步態強度，紅色點表示步點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/35/353696dcceb8791bd6ead1e6544d1d61.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，當上述狀態均取得1時，認爲當前時刻爲檢測到一步，計算公式如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/9e/9ea75f283f5f34755e9044b88320f3bd.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"計步算法示意圖如下圖所示，其中藍色曲線表示加速度計讀數，紅色三角形表示識別到一步。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d6/d63eb9148d18ea056c8dd44d43c91152.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"（2） 步長估計算法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們提出了一種基於統計特徵的步長估計算法，其中統計特徵包括步頻、一步內的加速度計讀數的方差、極差的n次開方，計算公式如下所述："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/bf/bf4848b418b1b154c7b33612c46191bd.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中，a, b, c和d是超參數，根據實際應用進行訓練。f表示行走步頻，"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/68/68d8b989e7542f432c511a19d7645a58.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"表示加速度計讀數極差的m次開方，"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/60/6049762b6ca21d6393a717e995eb9817.webp","alt":null,"title":"","style":[{"key":"width","value":"25%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"表示加速度計讀數的方差。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"（3）朝向估計算法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現有的朝向估計算法利用互補濾波和卡爾曼濾波實現設備朝向的估計，而設備朝向和人員行走朝向之間存在變化的偏差角會導致PDR的位姿估計累積飄移。爲了解決該問題，我們利用深度學習算法來回歸用戶行走的朝向。具體而言，我們提出了一種heading-confidence模型，採用LSTM和ResNet作爲深度網絡框架。LSTM主要負責行走朝向的迴歸，ResNet主要負責行走速度的迴歸，最後，我們設計了置信度函數來進行朝向的融合。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"（4） 運動分類"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了提升PDR系統對運動類別的魯棒性，我們設計了一種基於梯度提升決策樹算法的運動分類模型。主要包括行走、隨意行走、靜止和手持設備搖擺四種姿態。通過識別不同的行走姿態，我們自適應的加載對應的模型參數，繼而提高了PDR算法的精度。實驗結果如下圖所示。其中，黑色曲線表示參考真值，藍色曲線表示沒有運動分類的PDR算法軌跡，紅色曲線表示引入了運動分類的PDR軌跡。可以看到，紅色軌跡更加接近於參考真值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/64/6499363dd2bc71c62332faf8e2b14f39.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","marks":[{"type":"italic"},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}}],"text":"5. "},{"type":"text","text":"總結"}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/7a/7a26fb6a07b2693d87252e61271bf8ae.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"大型機場、商場、火車站內部的上車點引導能力對用戶體驗影響很大，爲了解決這個問題，地圖團隊做出了很多努力，推出了“圖文引導”方案來使用圖像和文字的方式幫助用戶。同時，爲了給用戶提供更好的體驗，我們也推出了滴滴AR實景導航產品，使用人工智能算法等科技手段，爲用戶提供準確的、易用的導航產品，來幫助用戶更快的找到上車點。目前在鄭州、深圳以及日本東京等24個機場、商場或者火車站上線了AR導航服務，數據顯示，它能幫助用戶節省近四分之一的時間，讓他們更便捷地到達上車點。與此同時，在南京火車站等大型場站，我們正在進一步探索基於AR導航的技術來幫助乘客快速抵達公交站點。後續我們將持續優化和打磨我們的算法，爲乘客和司機創造更多的價值。歡迎大家體驗我們的產品，並對我們提出寶貴的意見，希望通過我們的努力可以爲大家提供更好的出行體驗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d8/d8d64a35446c0e9db19232ccb78b415e.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":"center","level":4},"content":[{"type":"text","text":"團隊介紹"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":22}},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}},{"type":"strong"}],"text":"▬"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"滴滴地圖視覺計算團隊以使用計算視覺技術解決地圖和出行場景中的痛點問題爲目標，團隊在目標檢測、圖像分割、場景文字檢測識別、三維重建、SLAM、傳感器計算等方向具有豐富的技術積累，並將技術與業務場景緊密結合，成功落地地圖更新、引擎賦能、場站AR導航等多個系列項目，發揮了人工智能技術在降本增效、用戶體驗提升等方面的獨特優勢。團隊積極關注前沿技術發展，取得實例分割、圖像匹配等多項國際比賽冠軍，並在NeurIPS、CVPR、AAAI、TIP等頂級會議和期刊上發表多篇文章。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":"center","level":4},"content":[{"type":"text","text":"作者介紹"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":18}},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}},{"type":"strong"}],"text":"▬"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/c6/c6cb83aa9c6fbb727d5b238eed3d9dce.webp","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"滴滴地圖視覺計算團隊負責人、計算機視覺方向博士，十餘年計算機視覺領域研究和開發經歷，在視覺分析和理解領域有較豐富的算法研究和項目落地經驗。曾就職於微軟，從事微軟小冰視覺理解系統的研究和開發工作。累計發表視覺領域會議和期刊論文50餘篇，並擁有多項國內和國際專利。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":"center","level":4},"content":[{"type":"text","text":"延伸閱讀"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":18}},{"type":"color","attrs":{"color":"#FF7021","name":"orange"}},{"type":"strong"}],"text":"▬"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/cb/cb97a53fd6fdfc498cbf3ad05360aa10.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"https://mp.weixin.qq.com/s?__biz=MzU1ODEzNjI2NA==&mid=2247502881&idx=1&sn=fb1388bc3f67e3b1b8c8befbf4f350cf&chksm=fc29a286cb5e2b90a2426407707d14b64155c54218fedc844748d0ababebbe7a6bc44e9f7968&scene=21#wechat_redirect","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/07/078161b68258ab4f12b55313afbc4c6c.png","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"https://mp.weixin.qq.com/s?__biz=MzU1ODEzNjI2NA==&mid=2247501570&idx=1&sn=8ee5f8d6d56e2bca01354c3d2d78cb9e&chksm=fc29a9a5cb5e20b3e85ea6d1e3904ca2cbf708e4ca8a50d993693f7db14f1e060de9546a93cb&scene=21#wechat_redirect","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/a6/a6a44151a59e6650c257c943acc1b441.webp","alt":null,"title":"","style":[{"key":"width","value":"100%"},{"key":"bordertype","value":"none"}],"href":"https://mp.weixin.qq.com/s?__biz=MzU1ODEzNjI2NA==&mid=2247501156&idx=1&sn=003048d31f778b0d9d9bef8cf4c43aa8&chksm=fc29abc3cb5e22d5b696ac022ead1f1b6b2bed465fd7e8a6015a495e6cbf1d7420a7377e3721&scene=21#wechat_redirect","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"內容編輯 | Charlotte&Teeo"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"聯繫我們 | [email protected]"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/80/80440ffc3a1046abad9360dcaa0e8a0f.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}