點雲識別-Learning to Sample

Learning to Sample

2019 CVPR

摘要

原文	譯文
Processing large point clouds is a challenging task. Therefore, the data is often sampled to a size that can be processed more easily.	處理大規模點雲是一項具有挑戰性的任務，因此，一般把點雲下采樣到數量較少的size，方便處理
The question is how to sample the data? A popular sampling technique is Farthest Point Sampling (FPS). However, FPS is agnostic to a downstream ap-plication (classification, retrieval, etc.).	問題是如何對點雲進行採樣？目前最常用的方法是最遠點採樣(FPS)，但是FPS對於應用的任務是不可知的(FPS是通用的一個採樣技術，沒有爭對點雲處理的不同任務做不同的適應)
The underlying assumption seems to be that minimizing the farthest point distance, as done by FPS, is a good proxy to other objective functions.	FPS潛在的最遠點假設似乎對於點雲處理的目標函數指一種不錯的proxy。
We show that it is better to learn how to sample. To do that, we propose a deep network to simplify 3D point clouds.	本文提出學習如何採樣，提出用網絡來採樣\簡化三維點雲
The network, termed S-NET, takes a point cloud and produces a smaller point cloud that is optimized for a particular task.	我們稱他S-Net,S-Net的輸入時一個點雲，輸出一個更小的對特定任務優化後的點雲
he simplified point cloud is not guaranteed to be a subset of the original point cloud. Therefore, we match it to a subset of the original points in a post-processing step.	輸出的點雲可能不是輸入點雲的子集，所以我們提出一種匹配方法。
We contrast our approach with FPS by experimenting on two standard data sets and show significantly better results for a variety of applications.	我們在2個數據集上，將本文的採樣方法和FPS進行對比實驗。

創新點

1. A task-specific data-driven sampling approach for point clouds
2. A Progressive sampling method that orders points according to their relevance for the task

S-Net

按照文中的描述，先把task network預訓練好，用的是PointNet。然後用S-Net輸入原始點雲，輸入指定點數的點雲，把這個點雲經過匹配後送入Task Network裏，然後訓練S-Net。
訓練S-Net用到的loss包括：
（1）$L_f(G,P)=\frac{1}{|G|}\sum _{g\in G}min||g-p||_{2}^{2}$
對所有生成的點$g$要和原始點雲$P$相似。
（2）$L_m(G,P)=max_{g \in G}min_{p \in P}||g-p||_{2}^{2}$
對於最不相似的點要儘量相似。
（3）$L_b(G,P)=\frac{1}{|P|}\sum _{p\in P}min_{g \in G}||g-p||_{2}^{2}$
希望生成的點不衝突，儘量分散在輸入點雲中。

這張圖最能反應這篇文章的核心內容，S-Net的優勢體現在當點數少的時候，S-Net可以將那些關鍵點取出，而FPS不行。

Matching

由於不能保證生成的點集$G$ 是原始點集$P$的子集，所以需要加入一個匹配的步驟，作者用鄰近點搜索方法。
這裏要提一下原文中一句話

The matching process is only applied at inference time, as the final step of inference. During training, the generated points are processed by the task network as-is, since the matching is not differentiable and cannot propagate the task loss back to S-NET .

因爲取點這一操作是不可逆不可導的，所以task loss是不能回傳到S-Net的。之前考慮過從採樣這一環節入手改進PointNet++，看到文章題目的時候以爲作者也是做了這樣的工作，其實不然。最大的問題和困難就在不能BP，如何設計可以學習的採樣策略還是沒有解決，sad。

ProgressiveNet

這部分是對S-Net的一個擴展，可以產生任意size的點雲，作者頁做了相關工作。

文獻

從這篇文章裏我發現了幾篇不錯有關點雲的文章，待讀

1. [2018 ACCV] Flex-Convolution
2. PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows
3. Spherical Kernel for Efficient Graph Convolution on 3D Point Clouds
4. [CVPR 2019] Modeling Point Clouds with Self-Attention and Gumbel Subset Sampling