圖像技術在上億規模實拍圖片中的應用（算法好文）

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"閒魚作爲一個自由的商品和內容分發市場，每天有上百萬的用戶圖片上傳，其中不乏圖片重複，圖片描述內容不明確、圖文不符等低質量的商品；也有各種搞笑、抖機靈等倒流的內容；還有色情、黑產等違規或違法灰色地帶。如果讓這些低質的商品和內容都正常流入每天日常的商品展示中，不但會影響用戶的交易效率，還會降低產品在市場中的口碑和價值，甚至增加被監管治理和叫停的風險，其中較爲典型的問題包括："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖片內容重複：在閒魚日常的發佈內容中，有些賣家爲了增加自己商品的曝光機會用不同的描述和圖片創建多個相同商品進行投放，這時文本的表述可能完全不同，但商品圖片看上去基本一樣如圖1所示:"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/b4\/b4094e354e1a8b215350a20258b716db.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖1.相同商品圖片"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖文內容不一致：部分圖片內容與所售商品描述不一致，在排序時這些商品與其他一致的商品混排在一起時會影響整體搜索的體感和效果，如圖2所示；"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/49\/490db40173870ecc5f94168a71aa2c14.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖2.商品圖片和文本描述差異"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖片內容質量：用戶上傳的圖片不是所有圖片都適合做商品圖片展示的，例如商品包裝，商品局部，非商品圖片，發票單據，商品描述圖等，如圖3；"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6a\/6a1290a02692dcbc3ab229977cb394d1.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖3.不適合的商品圖片"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"違規：一些賣家爲了吸引買家的注意，會把一些如美女、性感和搞笑等違規圖片作爲商品主圖導流，嚴重影響閒魚的品牌價值和公平優良的交易環境，如圖4。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f5\/f5bce70bcfcc01c68ab9b8fb6f749ffc.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖4.美女首圖商品"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上述問題都可以歸結爲和視覺技術所涉及的範圍，實際上互聯網公司已經大規模的應用相關技術解決實際問題，例如阿里、百度、騰訊等頭部公司都有自己的視覺算法團隊，每年各大頂會也都是常客。大家不但探索相關的前沿技術，同時將其轉化應用到實際的產品中，如圖像特徵在拍立淘、百度識圖中的應用，圖像檢測直接運用在自動駕駛和工業質檢中，圖像識別相關技術分別在審覈、短視頻、廣告等業務得到廣泛的應用。本文針對閒魚產品中涉及的部分問題介紹如何利用視覺技術解決，例如圖片內容本身非商品或是色情等違規的物料都是可以利用圖像分類，圖像特徵等方法去解決，以下主要從以下幾個部分介紹："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"構建大規模圖片分類模型，學習閒魚產品中的圖片分佈特徵；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"基於分類模型學習圖像比對特徵；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"組合圖像分類和圖像特徵解決實際問題；"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"構建大規模圖片分類模型"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圖像分類模型是視覺模型的基礎，檢測，分割等視覺問題都依賴基礎的圖像的分類模型。在閒魚場景構建圖像分類模型存在以下幾個難點："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"圖片大部分是用戶上傳，圖像質量偏低，增加識別難度；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"圖片內容不限於商品類目本身，涵蓋很多和商品不相關其他類別，整體類別定義困難；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"閒魚商品的title是由用戶自己填寫的，結構化信息參差不齊，同時還包含很多口語化的噪聲；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"同類的商品包含的噪聲圖片很大，不能直接訓練"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":"數據標註成本高，短時間內不能有效覆蓋大部分數據；"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們這裏要求不直接識別出衆多的具體商品類目名稱，只要可以區分彼此即可，對於需要重點識別的類別，我們採用訓練好的特徵進行樣本挖掘，整體流程如圖5所示，包括基礎圖像特徵學習，聚類樣本構建，和分類模型的訓練："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/fc\/fcc5d2be6d3b0b60846f9893e4fd8e38.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖5.半自動圖像分類識別"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"基礎圖像特徵學習"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基礎圖像模型主要是爲了學習數據的整體分佈，在儘可能提高樣本覆蓋的情況下發掘較簡單的樣本使得模型可以冷啓動。首先根據線上的query請求的展示結果，收集高頻query下的點擊商品作爲候選集合，因爲有用戶誤點擊和高點擊導流樣本存在，所以需要過濾掉點擊率偏低和偏高的商品樣本，同時需要語義相近的query進行去重，經過上述幾步就構成了圖像分類的基礎數據。模型訓練採用resnet101模型，對比了softmax和arcface[5]兩種損失的效果，結果softmax好於arcface，原因可能是因爲樣本不純arcface很難收斂到較好的目標。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"聚類樣本構建"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有了基礎模型後，接下來需要收集閒魚類目樣本。首先收集閒魚每個類目下商品樣本，這步可以把語義概念控制在一定的範圍內，接下來根據title中的中心詞的命中率把該類目下下樣本劃分爲不同的子類目，每個子類目下的樣本都有明確的語義含義，但是圖片存在較大變化如上圖8-1所示“詹姆斯”這個子類目下的樣本存在多個品類的情況，這類樣本無法直接訓練。這時我們可以利用上面得到的基礎圖像特徵對子類目下的樣本提純，即在每個子類目類做聚類，把互爲圖像特徵餘弦距離最近的樣本聚合成新的類別，過濾掉樣本較少的類目，最終就得到了我們訓練閒魚場景下的圖像分類樣本，針對不同的距離閾值，我們分別選取了4.6K,7.4K,12K三組類目細分類，人工評估效果7.4K較爲理想，其他兩個對樣本的劃分要麼太粗要麼太細。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"分類模型的訓練"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於之前的分類模型參數，訓練新的類目標籤，採用batchsize=256, centercorp=224, 加入隨機crop,鏡像，cutout預處理，學習率採用cos學習率同時在一定輪數加入熱重啓，使模型在後期還能進一步提升準確率，最終驗證集合top@1準確率74%，基本達到了應用要求。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了能識別票據，文字圖，建築物，人物等特定類別，我們需要對這些類別做定製的樣本挖掘：通過已經訓練好的模型抽取特徵搭建商品庫的檢索系統，收集需要特殊處理的樣本進行最緊鄰查找，對查找後的結果卡閾值並再次進行檢索繼續擴充需要的樣本集合如圖6，最終合併這些類目與原有類目一起訓練。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/8c\/8cf321b6d628ed3462a92410afade3d1.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖6.特定類目樣本挖掘"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"基於分類模型學習圖像比對特徵"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"比對特徵主要是用來判斷商品是否同款，過濾掉或打散圖片重複的商品。由於系統裏每個商品已經單獨定義，所以這裏採用deepid[1][2]][3]方案，訓練同款特徵，但直接用每個商品的圖片訓練存在兩個問題：1.每個id類別存在變化較大的多張圖片並不能直接使用所有圖片；2.每個id類目的樣本稀少，直接訓練很難收斂。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對上述問題1我們有個先驗的假設即用戶上傳的多張圖片中與商品主要意圖相關的圖片佔大多數，所以我們可以借鑑之前聚類的思路，在同一個商品的多張圖片中進行聚類，選取樣本最多的類別爲我們的候選集合，如果類別間的樣本相差不大則認爲該商品不適合做訓練應丟棄。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了每個類別都可以儘可能挖掘到同款樣本，我們選取在線上同一個query下有交易行爲的商品和點擊率較高的商品作爲候選集合同時對每個商品內部進行聚類保證樣本純度，在實驗時我們發現品牌類query或品類query下的樣本存在較大的多樣性,如“安踏”、“華爲手機”、“電動車”，即使點擊行爲較爲稠密，但可能屬於不同sku的商品，所以我們需要對query進行限制，儘量保證query到單獨sku粒度消除歧義性，如“華爲p40 pro”、“極米投影儀h3”、“小牛電動車G2”。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有了上述的樣本我們就可以訓練deepid模型，這裏採用人臉常用的arcface loss[5], backbone選擇之前的分類模型和初始化參數，開始階段先用較小margin參數，然後遍歷scale參數，經過選取合適的margin和scale後訓練模型到飽和，接下來增大margin參數同樣選取合適的scale參數訓練，經過3次這樣的迭代使模型在不過擬合的情況下訓練，最終相款識別準確率95%， 商品sku內召回79%。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基於同款特徵也可以用於在相同識別的場景中，即利用同款特徵做召回再利用sift特徵做最後的幾何校驗，可以實現對旋轉、部分裁剪和遮擋的相同圖識別。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"組合圖像分類和圖像特徵解決相關性和多樣性問題"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"閒魚搜索相關性問題"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對用戶提交商品圖片多樣性的特點，爲了提升搜索體感，採用首圖相關性聚類的方式重排檢索結果如下圖7-1示例，以“銳鯊”搜索query爲例：圖7.1 舉例對照組檢索結果，可以看出整體隊列都是和便攜工具相關的商品，但也存在包裝（坑位1，6）和歧義（坑位3）的體驗不好的case，通過上述的商品圖像分類模型對商品多張圖片進行預測並聚類，我們可以得到每個商品的主要的top3類別，隨後統計整個隊列的top類別做爲該隊列置信類別，最後根據置信度分檔重排，結果參考圖7.2看到top商品都是移動工具相關。另一個case如圖8.1所示query爲“詹姆斯”經過圖像特徵重排球鞋這個主要類別被重排到前排如圖8.2，其他一些不太相關的類別被降權。線上的case中，也會有badcase，例如有些query如“華爲”，“蘋果”存在多類別是否需要重排需要根據用戶反饋進行優化，重排的實驗結果如下，點擊率還是有明顯的提升。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/cd\/cd74334ea426b0b30280a2e593aef29d.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/55\/550234821518a66d647de1f06a58b806.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖7.1.query=\"銳鯊\" 對照組top6結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/ac\/acd41ceee1ecd042e77ea706a585e53e.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖7.2.query=\"銳鯊\"圖像特徵重拍實驗組top6結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/98\/984d8a952c62f4a2330b3eb3fb1b17f3.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖8.1.query=\"詹姆斯\" 對照組top6結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/3a\/3acd23ca8225f2c8cee5506b37813935.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖8.2.query=\"詹姆斯\"圖像特徵重拍實驗組top6結果"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"閒魚Feed多樣性問題"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多樣性是推薦效果的一個重要的指標，相關結果太集中會影響用戶的體驗。閒魚用戶定義的商品特殊性，在多樣性問題上很難用一種方法解決，需要利用類目，文本描述，商品圖片等多維度共同解決，其中商品圖片多樣性問題解法與上面搜索一致性問題有些相似。如下圖9.1所示，同爲“華爲mate Xs”一款商品，用戶定義類目可能屬於不同類別，所以直接用戶定義類目無法做多樣性處理，但從圖片維度我們可以發現這些商品有相同的元素即商品包裝圖片， 所以可以利用圖像分類模型分別對每個商品圖片進行預測類目實現打散。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/8c\/8c998a76e8c405aaac0e7bc5250f5760.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖9.1 視覺同類但提交類目不同的case舉例"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先需要過濾掉人物和文字類目這些圖像表示一致但語義變化很大的商品不做處理，之後對商品的多張商品圖進行類目預測並對預測後的置信度top1的類目進行聚合，具有相同類目的商品即爲將要去重的候選集合，實際中發現有些相同商品的top1類目並不相同，可能是在top3中，但直接使用top3去重影響面會很大，會帶來不少badcase，所以這裏採用迭代兩次方式即對top1聚合後的結果top3類目投票，選取具有非共現但投票過半的類目進行第二次去重如9.2所示，經過兩次去重後可對文本和類目不好去重的case有效的補充。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/cc\/cccf21e920bc47180cb18cea0d681855.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"圖9.2 圖像多樣性去重邏輯"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上線後各項指標都有提升"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6e\/6e0ad708779ad66a3ee4ac9ca3e3caa0.png","alt":"圖片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"相同商品圖片去重"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"閒魚商品中有的賣家爲了增加自己的曝光會創建多個商品並採用相似的描述和視覺上變化很小的商品圖片，另外還有一些賣家會使用相同原始的商品圖片，這類商品如果出現在同一個搜索結果頁會帶來不好的用戶體驗，也會降低商品的交易效率。這裏利用之前圖像對比特徵搭建圖像搜索引擎，索引方式採用通用的乘積量化的方法構建1.2億商品庫，整體流程如下圖10.1。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每天實時的商品相對全庫比例較小，我們不需要做到實時精準去重，所以我們採用離線的去重方案：首先每天新增商品會以增量的方式入庫，在構建索引的同時會同步當前的商品在架、是否有效等狀態過濾掉無效的商品，然後利用乘積量化的方法構建離線去重檢索引擎，每天新增的商品會經過去重檢索引擎找到與庫裏相同的商品並更新到線上的KV存儲中，由於原有庫中的商品和新商品產生了新鏈接關係，所以還需要根據新的鏈接匹配關係去更新KV存儲中原有庫中該商品的相同商品列表。線上的請求在完成召回後，會根據召回的商品id去KV存儲中實時查詢對應的相同商品，最後打散邏輯會對當前頁中命中的相同商品進行分頁打散完成去重。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e9\/e93bb682d914008aec548d6e14767c84.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"10.1 商品離在線去重方案"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"不合規商品過濾"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"閒魚中的不合規商品主要包括美女首圖，性感首圖，搞笑圖片，人體局部等，但實際出售的商品和這些商品圖片沒有任何關係，賣家使用這類圖片主要是爲了吸引用戶的注意騙取流量。採用普遍使用的審覈模型、OCR識別可以過濾掉大多數色情、涉黃、涉政和暴恐等非法內容。然後還有些違規類的商品如圖11.2和搞笑、段子類的內容如圖11.3影響正常的交易市場，這類問題無法直接用審覈等模型過濾掉。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/c7\/c71867068545de36a35177f18f16e494.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"11.1 違規商品識別流程"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對與上述問題，我們設計了違規商品識別的方案如圖11.1所示，因爲涉及到線上業務策略這裏不做詳細描述，新增商品會經過兩個邏輯判斷是否違規："}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"首先識別和人相關的主圖，利用通用分類給所有商品圖片打標籤，然後過濾掉語義不一致的商品，但這是會有一定的badcase如圖11.4，這裏的用戶曬單是正常的商品，所以我在過濾後還需判斷是否有同款的商品，如果識別同款，則會通過該商品；"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"搞笑、段子類的圖片一般都爲熱門圖片，雖然會有可能再編輯，但主題內容是沒變化的，這類圖片通過建立違規內容庫利用相同識別可以得到解決，如圖11.1中的相同圖識別分支；"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/93\/93043e1844368722c5e40b8f4aa4e51d.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"11.2 違規商品示例"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e8\/e84cd7f2b7bc15b7fc7c9b510e6a7002.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"11.3 非商品搞笑、段子等示例"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/ad\/ad454a9495713ba8b0b1dba74019f6b2.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"11.4 人物首圖正常商品示例"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文主要介紹了利用視覺技術中的分類，特徵學習解決實際用戶分發商品中的應用，然而閒魚用戶自定義的商品和內容的多樣性使得在審覈、治理和結構方面給我們帶來了很多挑戰，單純利用一種模態和內容很難把所有問題都解決，所以在實際應用中會綜合多種方案和技術組合求解，例如在識別類目問題上，不但需要利用文本、圖像內容提升識別精度，還在產品側引導用戶幫助完成商品結構化；對於樣本提純不但可以標註，還可以充分合理利用用戶的反饋行爲幫助我們；另外檢索系統、數據加工流程和同步管理等工程化是最終模型體現價值的關鍵，光只有單純幾個模型並不能直接解決線上問題；在違規治理問題上正所謂“魔高一尺，道高一丈”，有些不合規用戶總鑽平臺的漏洞爲自己牟利，與他們的鬥智鬥勇是個長期的過程，也需要技術不斷迭代和超越。感謝合作團隊:閒魚結構化團隊，閒魚架構團隊，達摩院、審覈部分等兄弟團隊給與的人力和技術支持。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"引用"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[1]Yi Sun,Xiaogang Wang,Xiaoao Tang. Deep Learning Face Representation from Predicting 10,000 Classes .CVPR 2014"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[2]Yi Sun"},{"type":"sup","content":[{"type":"text","text":"[1]"}]},{"type":"text","text":", Xiaogang Wang"},{"type":"sup","content":[{"type":"text","text":"[2]"}]},{"type":"text","text":", Xiaoou Tang"},{"type":"sup","content":[{"type":"text","text":"[3]"}]},{"type":"text","text":". DeepID2: deep learning face representation by joint identification-verification. CVPR 2014"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[3]Yi Sun"},{"type":"sup","content":[{"type":"text","text":"[4]"}]},{"type":"text","text":", Xiaogang Wang"},{"type":"sup","content":[{"type":"text","text":"[5]"}]},{"type":"text","text":", Xiaoou Tang"},{"type":"sup","content":[{"type":"text","text":"[6]"}]},{"type":"text","text":". Deeply learned face representations are sparse, selective, and robust. CVPR 2014"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[4]Yi Sun"},{"type":"sup","content":[{"type":"text","text":"[7]"}]},{"type":"text","text":", Ding Liang"},{"type":"sup","content":[{"type":"text","text":"[8]"}]},{"type":"text","text":", Xiaogang Wang"},{"type":"sup","content":[{"type":"text","text":"[9]"}]},{"type":"text","text":", Xiaoou Tang"},{"type":"sup","content":[{"type":"text","text":"[10]"}]},{"type":"text","text":". Face Recognition with Very Deep Neural Networks. CVPR 2015"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[5]Jiankang Deng"},{"type":"sup","content":[{"type":"text","text":"[11]"}]},{"type":"text","text":", Jia Guo"},{"type":"sup","content":[{"type":"text","text":"[12]"}]},{"type":"text","text":", Niannan Xue"},{"type":"sup","content":[{"type":"text","text":"[13]"}]},{"type":"text","text":", Stefanos Zafeiriou"},{"type":"sup","content":[{"type":"text","text":"[14]"}]},{"type":"text","text":". ArcFace: Additive Angular Margin Loss for Deep Face Recognition. CVPR 2018"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"References"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[1]"}]},{"type":"text","text":"Yi Sun: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Sun%2C+Y"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[2]"}]},{"type":"text","text":"Xiaogang Wang: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Wang%2C+X"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[3]"}]},{"type":"text","text":"Xiaoou Tang: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Tang%2C+X"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[4]"}]},{"type":"text","text":" Yi Sun: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Sun%2C+Y"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[5]"}]},{"type":"text","text":"Xiaogang Wang: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Wang%2C+X"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[6]"}]},{"type":"text","text":"Xiaoou Tang: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Tang%2C+X"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[7]"}]},{"type":"text","text":"Yi Sun: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Sun%2C+Y"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[8]"}]},{"type":"text","text":"Ding Liang: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Liang%2C+D"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[9]"}]},{"type":"text","text":"Xiaogang Wang: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Wang%2C+X"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[10]"}]},{"type":"text","text":"Xiaoou Tang: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Tang%2C+X"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[11]"}]},{"type":"text","text":"Jiankang Deng: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Deng%2C+J"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[12]"}]},{"type":"text","text":"Jia Guo: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Guo%2C+J"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[13]"}]},{"type":"text","text":"Niannan Xue: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Xue%2C+N"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"[14]"}]},{"type":"text","text":"Stefanos Zafeiriou: "},{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/arxiv.org\/search\/cs?searchtype=author&query=Zafeiriou%2C+S"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：閒魚技術（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/AQqaxvok9N6PWd83GZ_wpA","title":"xxx","type":null},"content":[{"type":"text","text":"圖像技術在上億規模實拍圖片中的應用（算法好文）"}]}]}]}