基於 TensorFlow.js 和 MoveNet 的下一代姿態檢測

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"近日，我們非常高興地發佈了最新的姿態檢測模型 MoveNet，並在 TensorFlow.js 中推出了新的姿態檢測 API。MoveNet 是一個超快速和準確的模型，能夠檢測出身體的 17 個關鍵點。這個模型在 TF Hub 上提供了兩種變體，即 Lightning 和 Thunder。Lightning 用於延遲關鍵型應用，而 Thunder 用於需要高精度的應用。在大多數現代臺式機、筆記本電腦和手機上，這兩種模型都比實時要快（30+FPS），這對於實時性的健身、運動和健康應用都是至關重要的。通過在瀏覽器中使用 TensorFlow.js 在客戶端完全運行模型，無需調用服務器，也無需在初始頁面加載之後安裝任何依賴關係，就可以實現這一點。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/9c\/9c1bc1e678048465414f4c747f09c6a5.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"MoveNet 可以通過快速運動和非典型姿勢來追蹤關鍵點"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在過去的五年裏，人體姿勢追蹤技術有了很大進展，但在許多場景中仍未得到廣泛應用。那是因爲人們更加關注與讓姿勢模型變得更大、更準確，而不是爲了讓它們能夠迅速部署到任何地方。對於 MoveNet 來說，我們的任務是設計並優化一個模型，在保持儘可能少的推理時間的同時，利用先進架構的最佳方面。這樣的模型可以在不同的姿勢、環境和硬件設置中提供準確的關鍵點。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了解 MoveNet 是否有助於解鎖患者的遠程醫療，我們與數字健康和性能公司 IncludeHealth 進行了合作。IncludeHealth 開發了一個交互式網絡應用程序，指導患者在自己舒適的家中（使用手機、平板電腦或筆記本電腦）完成各種日常操作。這些程序是由物理治療師以數字方式設定的，用來檢驗平衡、力量和活動範圍。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這項服務需要一個基於網絡和本地運行的隱私姿勢模型，它能提供高幀率下的精確關鍵點，然後用來量化和定量分析人體姿勢和運動。一種典型的現成檢測器，對於肩部外展或全身下蹲等簡單動作來說是足夠的，而更爲複雜的姿勢，比如坐姿膝蓋伸展或仰臥姿勢（躺下），甚至對於針對錯誤數據進行訓練的最先進的檢測器來說，也會帶來麻煩。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/ad\/adb2cb47257d840227283b57c03a7631.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"傳統探測器（上圖) 和 MoveNet（下圖) 在高難度姿勢下的對比"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們爲 IncludeHealth 提供了早期版本的 MoveNet，可以通過新的 pose-detection API 進行訪問。這個模型是通過健身、舞蹈和瑜伽姿勢訓練得到的（詳細信息見下面的訓練數據集）。IncludeHealth 在其應用程序中集成了這個模型，並用 MoveNet 和其他可用的姿勢檢測器做了基準測試："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“MoveNet 模式注入了強大的組合，提供治療所需的速度和準確性。在其他模式相互交換時，這種獨特的平衡使得下一代護理服務得以展開。在這個過程中，谷歌團隊是一個傑出的合作伙伴。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"—— Ryan Eder，IncludeHealth 公司的創始人兼首席執行官。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作爲下一步，IncludeHealth 將與醫院系統、保險機構等合作，將傳統的護理和培訓延伸到實體醫院之外。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/wechat\/images\/b1\/b1a5a5bfa0160db75ad565ac36efe920.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"在瀏覽器中運行的 IncludeHealth 演示應用程序，使用由 MoveNet 和 TensorFlow.js 支持的關鍵點估計對平衡和運動進行量化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"安裝"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有兩種方法來使用 MoveNet 和新的姿勢檢測 api："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過 NPM："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"null"},"content":[{"type":"text","text":"import * as poseDetection from '@tensorflow-models\/pose-detection';\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通過腳本標記："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"null"},"content":[{"type":"text","text":"