Airbnb基於深度學習的搜索算法優化實踐

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Airbnb搜索業務中，根據其業務特殊性，工程師們經過多次迭代嘗試，深入挖掘DNN在搜索排序中的潛力，預定量和收入都得到了很大的提升。在論文"},{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/pdf\/2002.05515.pdf","title":null,"type":null},"content":[{"type":"text","text":"《Improving Deep Learning For Airbnb Search》"}]},{"type":"text","text":"中，研究團隊從DNN網絡結構的搭建到解決冷啓動問題、位置偏差問題等方面分享了非常寶貴的算法經驗。我們將在本文對這項研究工作進行詳細解讀。"}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#000000","name":"user"}}],"text":"概覽"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Airbnb是一個雙邊市場，它將擁有出租房子的房東和來自全球各地的潛在客人聚集在一起。Airbnb的搜索排名問題是根據客人的查詢對住宿地點進行排名，這些查詢通常包括位置、客人數量和入住\/退房日期。演進到深度學習是Airbnb搜索排名發展的一個重要里程碑。我們在[6]中對這段旅程的描述使我們與許多行業實踐者進行了交談，允許我們能夠交流見解和評論。在這樣的交流之後經常出現的一個問題是：下一步怎麼辦？我們試圖在本文中回答這個問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"深度學習排名的推出引起了很多人的慶祝，不僅因爲它帶來了預訂量的增長，還因爲它給我們未來的路線圖帶來了變化。最初的看法是，通過Airbnb對深度學習的排名，我們可以接觸到這個龐大的機器學習思想寶庫，它似乎每天都在增長。我們可以簡單地從論文研究中挑選出最好的點子，一個接一個地推出，從此過上幸福的生活。但事實證明，這是樂觀情緒的頂峯。很快，我們熟悉的進入絕望谷的模式就出現了，在其他地方取得顯著成功的技術在我們自己的應用中被證明是相當中性的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這將導致我們在第一次發佈之後如何迭代深度學習的策略進行全面修訂。在本文中，我們捕獲了在[6]中描述的DNN發佈之後的主要增強。除了深入研究核心機器學習技術本身，我們還關注導致突破的過程和推理。從現在的大局來看，我們更看重在dnn上迭代的經驗教訓，而不是任何單獨的技術。我們希望那些專注於在行業環境中應用深度學習的人會發現我們的經驗很有價值。我們先來看看我們改進DNN體系結構的努力。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"優化架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"什麼是深度學習？好吧，添加更多的層。至少這是我們在回顧了開創當前深度學習時代的一系列進步之後的天真的解釋。但當我們試圖複製[14]中總結的縮放數據和添加層的好處時，我們只得到了中性的測試結果。試圖解釋爲什麼增加的層沒有顯示出任何好處，我們從文獻中借用了更多的思想，比如應用殘差學習[7]和批處理規範化[8]。不過，NDCG拒絕在離線測試中讓步。我們從實驗中得到的結論是，增加層數是卷積神經網絡的一種有效技術，但不一定適用於所有的DNN。對於像我們這樣完全連接的網絡，兩個隱藏層就足夠了，模型容量不是我們的問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們假設，如果更深層次的網絡不是適合我們的架構，那麼更專業化的架構可能會更適合我們。因此，我們嘗試了能夠更明確地處理查詢和列表之間交互的架構，比如deep&wide，其中查詢房屋特徵交叉被添加到寬部分。隨後是[16]中基於注意力的網絡變體。其目的是使從查詢特徵派生的隱藏層將注意力集中在從listing特徵派生的隱藏層的某些部分上。對這些努力的簡短總結是，他們也未能採取行動。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在嘗試將成功的深度學習體系結構導入到產品應用程序中時，在翻譯過程中經常會迷失的是，一個體繫結構的成功與它的應用程序上下文密切相關。所報告的架構性能增益來自於解決與之比較的基線的某些缺點。由於深度學習普遍缺乏可解釋性，因此很難準確推斷新體系結構正在解決什麼缺陷以及如何解決。因此，判斷這些確切的缺點是否也困擾着這款產品，就成了一個猜測。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了提高成功的機會，我們放棄了{download paper→implement→A\/B test}這樣的循環流程。相反，我們決定基於一個非常簡單的原則來驅動這個過程：用戶主導，模型跟隨。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.1 用戶主導，模式跟隨"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這裏的想法是首先量化用戶問題。模型調整是在隨後進行的，並且是針對用戶的問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"沿着這些思路，我們首先觀察到[6]中描述的一系列成功的排名模型發佈不僅與預訂量的增加有關，而且還與搜索結果的平均掛牌價格下降有關。這表明模型迭代越來越接近客人的價格偏好，這低於之前模型的估計。我們懷疑，即使在連續降價之後，這款車型的價格選擇和客人的喜好之間也可能存在差距。爲了量化這個差距，我們研究了顧客看到的搜索結果的中間價與顧客預訂的物品價格之間的差異分佈。由於價格服從對數正態分佈，所以在計算價格對數後計算差額。圖1描繪了差異是如何分佈的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/48\/4815deeeab34a380e725597a266959ed.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們的預期是，預訂價格將對稱分佈在搜索結果的中間價附近，並且類似於以零爲中心的正態分佈。相反，它的負面影響很大，表明客人傾向於降低價格。這給了我們一個需要調查的具體用戶問題：價格較低、更接近客人偏好價格的房源是否需要排名靠前。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"考慮到兩個其他東西都相同的普通房源，我們的直觀理解是，客人更喜歡更經濟的房源。我們的排名模型真的理解這個“便宜就是好”的原則嗎？我們不能完全確定。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.2 強制使越便宜越好"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"模型是DNN，我們不清楚它是如何學習房屋價格的。熟悉的工具，如檢查logistic迴歸模型中的相應權重或繪製GBDT模型的部分依賴圖，在DNN模型上不再有效。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了使價格更具可解釋性，我們採用了以下更改："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"1）刪除了價格作爲DNN的輸入特徵。我們將這個修正的DNN表示爲DNN θ（u，q，l (no- price）)。這裏θ是DNN參數、u是用戶特徵、q是查詢特徵和l(no_price)房屋特徵列表（價格除外）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"2）將模型的最終輸出表示爲:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/e9\/e9e7d1ef53b8137b336c7c9bbb331f1a.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"其中，使用反向傳播來學習附加參數w和b，以及"}]},{"type":"image","attrs":{"src":"https:\/\/uploader.shimo.im\/f\/E9yy97wxhiLvH3VZ.png!original","alt":null,"title":null,"style":[{"key":"width","value":"50%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這裏price是原始價格特徵，price（median）是根據記錄的房屋價格的中值計算出來的常數。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"術語−tanh（）允許我們通過單調地降低相對於價格上漲的產出分數來實施“越便宜越好”。易於解釋的w和b參數使我們能夠描繪出價格的精確影響。對於參數的學習值，w=0.33和b=−0.9，圖2顯示了排名過程中遇到的典型P範圍。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/5a\/5a749a317278ae42d8ffde24a89968c2.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"當在線測試時，作爲針對[6]中兩個隱藏層DNN的A\/B實驗，搜索結果的平均價格下降了−5.7%，與離線分析相一致。但由於預訂量下降了1.5%，價格的可解釋性付出了沉重的代價。我們的假設是，價格與其他特徵之間有很大的相互作用。將價格與模型隔離導致擬合不足。這一假設得到了訓練和測試NDCG都下降的事實的支持。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.3 廣義單調性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了在模型中保留“便宜即是好”的直覺，但允許價格特徵與其他特性交互，我們開始研究DNN體系結構，這些體系結構在某些輸入方面是單調的。在[19]中描述的晶格網絡（Lattice network）爲這個問題提供了一個優雅的解決方案。但將整個系統轉向晶格網絡是一個巨大的挑戰，我們尋求一種破壞性更小的機制。因此，我們構建瞭如圖3所示的體系結構，除了Tensorflow中固有的那些節點之外，它不依賴任何專門的計算節點。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f3\/f36c4ba43a7fe186bcc926dfe5f44cf4.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們討論逐步構建體系結構的流程，確保從輸入價格節點到最終輸出的所有路徑相對於價格都是單調的："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（1）我們將 −P 作爲輸入，DNN是對價格是單調遞減的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（2）在輸入層，我們不是用−P乘以權重，而是乘以權重的平方。由於−w2*P+b對於w和b的任何實值都是單調遞減的，所以到第一隱層的輸入對於價格總是單調地降低。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（3）對於隱藏層，我們使用tanh激活來保持單調性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（4）給定f0（x）和f1（x），x的兩個單調遞減函數w0^2*f0（x）+w1^2*f1（x）+b也是 x的單調遞減函數，其中w0和w1可以是任意實數權重。我們在第二個隱藏層和輸出層使用這個屬性，其中所有的權重都是平方的。圖3中的第二個隱藏層和輸出層用粗體實線表示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（5）添加一個子網，該子網既沒有價格作爲輸入，也沒有任何單調性約束，以允許其餘特徵之間的無約束交互。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"儘管比第2.2節中描述的架構更靈活，但在線測試的結果非常相似，導致預訂量下降-1.6%。與它的前身一樣，該體系結構強制要求模型輸出在所有情況下對於價格都是單調地降低。這種架構的失敗表明，價格方面的單調性是一個過於嚴格的約束。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.4 軟單調性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"雖然第2.3節描述的體系結構揭示了DNN在支持模型約束方面的多功能性，但它也教會了我們DNN的另一個特點：它們的行爲就像團隊中的另一位明星工程師。對於一個問題，任由他們自己解決，他們通常會想出一個合理的解決方案。但如果強迫他們往某個方向發展，災難就會很快接踵而至。所以在下一次迭代中，我們決定通過設置上下文而不是控制來管理DNN。我們沒有強制要求模型的輸出與價格單調相關，而是添加了一個柔和的暗示，即越便宜越好。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"通常，每個訓練示例都包含一對房屋，一個已預訂，另一個未預訂。將DNN應用於這兩個房屋的特徵將生成相應的logit，損失的定義如表1所示。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6e\/6e30bb37c204c2f31c06e4c2aa0690f8.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了增加價格提示，我們爲每個訓練樣本引入了第二個標籤，指出兩個示例中哪個房屋的價格更低，哪個價格較高。如表2，我們對loss進行了修改。alpha參數提供了一種方式來控制我們的結果是以相關性排序還是價格排序。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/89\/89a0ef8ef4cdd7c661af64541e493fc3.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了驗證這一想法，我們將alpha超參數調整到最小值，以便在離線測試中得到與基線模型相同的NDCG。這使我們能夠在不損害相關性的前提下，儘可能地推動“便宜即是更好”的直覺，至少在離線測量時是這樣。在在線A\/B測試中，我們觀察到搜索結果的平均價格下降了-3.3%。但預訂量也下降了0.67%。離線分析的侷限性在於，它只評估對日誌中可用的最高結果進行重新排序。在在線測試中，將新訓練的模型應用於整個庫存，揭示了將價格損失作爲訓練目標的一部分的真實成本。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.5 添加一些ICE"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"降價實驗帶來的災難讓我們陷入了一種自相矛盾的狀態：搜索結果中的房屋價格似乎高於客人的喜好，但壓低價格卻讓客人不高興。爲了瞭解新模型的不足之處，有必要比較基準模型是如何利用價格特徵的，但這被完全連接的DNN缺乏可解釋性所掩蓋。如前所述，部分依賴圖這樣的概念是沒有用的，因爲它們依賴於一個給定特徵對模型的影響獨立於其他特徵的假設。在DNNs的情況下，這根本不是真的。試圖畫出價格的部分依賴關係會產生平緩傾斜的直線，這表明DNN對價格有一些輕微的線性依賴，這與我們所知道的一切都是矛盾的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了取得進展，我們縮小了DNN可解釋性的問題。我們沒有試圖對價格如何影響DNN做一般性的陳述，而是專注於一次解釋一個搜索結果。借鑑文獻[5]中單個條件期望（ICE）圖的思想，我們從單個搜索結果中提取房屋列表，在保持所有其他特徵不變的情況下，掃描價格範圍，並構建模型分數圖。圖4顯示了一個示例圖。這些圖表明[6]中的完全連接的兩層DNN學習了便宜是好的這一結果。對從日誌中隨機選擇的一組搜索結果重複ICE分析進一步強化了這一結論。通過試圖進一步壓低價格，失敗的架構在房屋質量上作出了妥協。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/fc\/fc558004446df2835f4877923865f420.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.6 雙塔結構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"回到圖1，通過這個圖，我們瞭解到了客人給我們傳遞的信息。但是，那些試圖用相關性換取價格的體系結構錯誤地解釋了這一信息。重新解釋圖1是合理的。重新解釋必須符合價格，以及相關性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"圖1的另一種解釋出現在我們計算客人搜索結果的中間價和他們預訂的價格之間的差額，並計算出按城市分組的平均值。正如預期的那樣，各城市之間存在差異。但與主要城市相比，尾城的差異要大得多。尾城通常也位於發展中市場。圖5顯示了一些選定城市搜索結果的中間價與預訂價之間的平均差。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這就產生了這樣一種假設，即圖1背後的DNN正遭受着大多數人的暴政，關注的是價格-質量的權衡，這些權衡是針對最受歡迎的預訂地點而調整的。將這些折衷推廣到尾部查詢並沒有起到很好的效果，而且模型無法適應當地的條件。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這一假說與另一項有關DNN的特徵的觀察結果吻合得很好。考慮到DNN是使用成對損失進行訓練的，兩個房屋列表中不同的特徵似乎是影響最大的因素。查詢特徵，這個交叉對很常見，似乎沒有什麼影響，刪除它們對NDCG的影響很小。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"新的想法是，該模型對“便宜即是好”有充分的理解，它所缺少的是旅行的正確價格的概念。理解這一概念需要更密切地關注諸如位置之類的查詢特徵，而不是純粹基於房屋特徵來區分。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這啓發了下一次的架構修改，它由雙塔組成，類似於[10]。第一個塔由查詢和用戶特徵提供信息，生成一個100維向量，從概念上表示查詢用戶組合的理想列表。第二座塔根據所列特徵構建了一個100維向量。兩個向量之間的歐幾里德距離被用來衡量給定房屋與查詢用戶的理想房屋之間的距離。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"訓練樣本包括成對的房屋列表：一個已預訂，另一個未預訂。損失函數的定義是，與已預定的房屋相比，未預定的房屋與理想情況的差距有多大。因此，對這兩塔的訓練使這兩塔的預訂房源更接近理想，同時也將未預訂的房源推到了另一邊。這與[12]中引入的triplet loss類似。這裏的主要區別在於，我們沒有訓練三元組，而是隻有成對的房源列表，三元組中丟失的錨定房源由查詢用戶塔自動學習。查詢和房源塔的成對訓練如圖6所示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/ec\/ecd06f12456c2ae0478e6ea9cf3c8f92.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"表3顯示了該體系結構的抽象Tensorflow代碼。實際實現略有不同，以優化訓練速度。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/ee\/ee42e2af44add872cdddd5f97704c2d0.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.7 測試結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"當在A\/B實驗中針對[6]中的完全連接的兩層DNN進行在線測試時，雙塔結構記錄了+0.6%的預訂收益。由於在線計算的NDCG提高了+0.7%，搜索的容易程度提高了。雖然雙塔架構並不是爲了降低價格，但我們發現搜索結果的平均價格下降了2.3%，這是相關性增強的副作用。預訂量的增加抵消了價格下降對收入的影響，導致整體增長+0.75%。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"除了提高結果的質量，雙塔結構還允許我們優化在線評分DNN的延遲時間。對於完全連接的體系結構，評估第一個隱藏層對評分延遲的貢獻最大。計算第一隱藏層的計算複雜度可以表示爲O（H∗（Q+L）），其中Q是獨立於房源的查詢和用戶特徵的數量，L是與房源相關的特徵的數量，H是第一層的隱藏單元的數量。對於評估有N個房源的搜索結果集，總複雜度可以表示爲O（N*H*（Q+L））。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在新架構中的兩個塔中，查詢塔獨立於房源列表。這樣就可以對整個搜索結果集精確地對該塔進行一次評分，並且只爲每個房源評估與房源相關的塔。第一個隱藏層的計算複雜度降低到O（N*Hl*L+Hq*Q），其中Hl和Hq是房源和查詢塔中隱藏單元的數量。當在線測試這一結果時，這導致第99百分位評分延遲降低了-33%。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.8 架構回顧"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"就在我們慶祝成功的同時，隨着DNN迭代的啓動，質疑也悄然出現。體系結構是否按預期工作，或者DNN無意中發現了其他東西？DNNs的不可穿透性使得在過去很難回答這樣的疑問。但是考慮到雙塔結構的直覺是針對用戶問題開發的，我們現在可以利用這些直覺更好地瞭解DNN是如何工作的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"重新審視價格的ICE的繪圖，我們看到了明顯的變化。我們看到的不是那些總是隨着價格向下傾斜的繪圖，而是強調價格越便宜越好的解釋，而是看到分數在某些價格附近達到峯值，如圖7所示。這更接近旅行解釋的正確價格。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/9c\/9c7825bd25beb53c9742ce9dd836b7ec.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在這種情況下，人們經常提出的一個問題是，低質量的房源是否可以通過設定一個價格來獲得新模式的排名。對ICE曲線的仔細研究發現，某些價格附近的分數峯值只出現在高質量的房源身上，它們通常一開始就排名靠前。對於大多數普通房源，對價格的繪圖仍保持單調下降曲線。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"合適的價格和理想房源的概念圍繞着查詢塔生成的向量，所以自然的後續工作就是研究這些向量到底是什麼樣子。爲了進行分析，我們在隨機抽樣的搜索中運行了雙塔DNN，並收集了查詢塔的輸出向量。由於100維向量無法被人類理解，我們應用t-SNE將其縮減爲二維向量，如圖8所示。與圖5中的一些城市相對應的查詢會標記在圖上。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/d8\/d87dae706327babe37e7cff991f47845.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"令人欣慰的是，在類似的參數值（如客流量和行程）周圍形成大型簇類。在大型簇中，直覺上感覺相似的城市彼此之間的距離相對較近。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"值得強調的是，聚類並不是簡單的價格聚類。與查詢相對應的預訂房源的價格由點的顏色表示，我們可以看到聚類具有所有範圍的顏色。雖然莫斯科通常比巴黎便宜，但莫斯科的預訂價格很容易超過巴黎的預訂價格，這取決於遊客數量、停留時間、距離旅遊景點的距離、週末與工作日以及其他一系列因素。價格與所有其他維度都有着千絲萬縷的聯繫，掌握一次旅行的合適價格意味着要同時掌握所有其他因素。我們所做的分析都不能作爲確鑿的證據，證明雙塔模型確實學習到了。但是，對價格的ICE圖、查詢塔輸出的t-SNE可視化以及對各城市價格變動的額外分析的結合，給了我們足夠的信心，使我們相信該機制正在按預期工作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"暫且不談一系列的架構操作，接下來我們將着手解決一個排名挑戰，這個挑戰不僅影響到客人，也影響了Airbnb社區的另一半，即主人。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"改善冷啓動"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在旅行的機器學習應用程序中，任何時候都有很大一部分用戶是新用戶或在長時間間隔後使用產品。對於所有實際用途，用戶都處於[2]所述的連續冷啓動狀態。處理用戶級冷啓動是核心排名公式本身的一部分。因此，當提到冷啓動問題時，我們將注意力集中在項目級別的冷啓動（即如何處理排名中的新房源）。正如在第2.1節中對DNN體系結構的改進一樣，我們探索的起點不是文獻調查，而是對用戶問題的觀察。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"使用NDCG來量化已預訂房源在搜索結果中的位置一直是我們最可靠的模型性能指標。因此，調查用戶問題的一個自然的地方就是尋找NDCG與整體NDCG相比較低的部分。將新平臺中新登記的登記房源的NDCG劃分出來，並將其與現有房源進行比較，我們發現差距爲-6%。就上下文而言，我們觀察到，與NDCG模型相比，在線預訂的差異僅爲0.7%，在統計學上存在顯著差異。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這表明，這種模式使客人更加努力地去發現值得預訂的新房源。爲了更好地理解這一點，我們從DNN中刪除了所有基於與客人的歷史交互生成的輸入特徵，例如一個房源的過去預訂數量。取消這些預定特徵導致NDCG下降了−4.5%。顯然，DNN在很大程度上依賴於預定特徵。由於新房源與客人之間缺乏這些細粒度的特徵，DNN被迫根據剩餘的特徵作出大致判斷，以接近新房源的平均表現。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.1 冷啓動和Explore-Exploit方法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"冷啓動問題的一個可能框架是將其視爲探索和利用之間的折衷。排名策略可以通過利用對當前庫存的瞭解來優化短期內的預訂，並且只對那些有着可靠業績記錄的房源下注。但爲了市場的長期成功，它需要付出一些成本來探索新的庫存。這種折衷可以作爲一個顯式的排名提升來實現，它爲新房源分配比DNN所確定的更高的排名。這使得新的房源可以收集客人的反饋，而預訂費用很低。通用方法在電子商務排名應用中已經很流行，例如在[15]中。這種提升可以進一步細化，通過展現計數來封頂，或者引入時間衰減。我們的第一次迭代是爲了測試這種提升。通過在線A\/B測試，我們將新的房源排名提升調整爲預訂中性，而不是沒有提升，同時爲新房源分配+8.5%的額外首頁展現機會。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"但是，在explore-exploit範式下運行帶來了嚴峻的挑戰："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（1）新的房源排名提升在不同的方向上受到兩種對立力量的影響：1）由於搜索結果相關性降低，用戶體驗在短期內下降（我們可以準確衡量這種影響）；2）長期來看，由於庫存增加，用戶體驗有所改善（我們發現這種影響更爲明顯，難以量化）。由於缺乏對最佳提升量的明確和客觀定義，導致了激烈的內部辯論，沒有一個解決方案能讓每一個感興趣的團隊都滿意。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（2）甚至在對勘探成本進行任意組合後，預算的合理使用取決於特定地點的供求關係。當需求量大時，對勘探的容忍度很高，但當一個地方的需求量很少時，就沒有那麼大了。而且，在良好供應受到限制的地區，勘探和擴大庫存的需求很高。當大量高質量的房源處於空置狀態時，很少有動機去承擔勘探成本。供應和需求反過來又取決於地理位置、季節性和客流量等參數。因此，爲了最佳地利用全局勘探預算，需要數千個定位參數，這是一項人工無法完成的任務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.2 估計未來用戶參與度"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了讓這個系統變得更容易管理，我們又採取了什麼措施？當然，答案是缺乏用戶生成的參與特徵，如預訂數量、點擊量、評論等。其他屬性，如價格、位置、便利設施等，與其他房源一樣是衆所周知的。從理論上講，如果我們有一個神諭來預測一個新的房源的參與特徵，它可以以最佳的方式解決冷啓動問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"因此，我們沒有將冷啓動視爲一種探索-利用權衡，而是將其重新定義爲一個估計新房源的參與度值的問題。重新定義問題發現了一些重要的東西：它允許我們爲問題定義一個客觀的理想，並積極地朝着它努力。爲了解決冷啓動問題，我們引入了一個新的組件，它可以在訓練和評分時預測新房源的用戶參與特徵。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了測量估計器的準確性，我們採用了以下步驟："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（1）從日誌中獲取O（100M）的搜索結果樣本。對於每個搜索結果，從前100個位置隨機抽取一個房源。這些房源樣本充分受到客戶的關注，因此他們的參與度特徵可以充分覆蓋。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（2）讓R real表示從日誌中獲得的抽樣房源的排名。我們將排名表示爲實數，以表示房源的參與特徵是真實客戶交互的結果。根據排名，我們計算實際折扣排名爲DR real=log（2.0）\/log（2.0+R real）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（3）接下來，對於每個抽樣房源，我們移除所有的參與特徵，並用測試集預估器預測的參與特徵來替換它們。我們用預測的參與度特徵對房源進行評分，在相應的日誌搜索結果中找到它的新排名，然後從中計算折扣排名。我們用DR predicted表示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（4）對於每個抽樣房源，我們計算參與估計誤差爲（DR real−DR predicted）^2。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（5）爲了得到總體誤差，我們對所有抽樣房源中的參與度估計誤差取平均值。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"理想的參與度估計器將產生0誤差。爲了在兩個估計量之間作出決定，可以選擇誤差較小的估計量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了驗證，我們比較了兩種估計方法。基線是生產中使用的系統，它爲缺少的特徵分配默認值，包括新房源的參與度特性。默認值是通過手動分析相應特徵而構建的常量。這項比較是針對一個預估器，該預估器通過平均新房源地點附近房源的參與特徵來預測參與特徵。爲了提高準確性，它只考慮與新房源的客流量相匹配的相鄰房源，並計算滑動時間窗口內的平均值，以考慮季節性。例如，爲了估計一個有兩個客人容量的新房源的預訂數量，它取的是能容納兩人新房源半徑範圍內所有房源的平均預訂量。這在概念上類似於[11]中的樸素貝葉斯推薦，它使用生成方法來估計缺失的信息。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.3 測試結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在離線分析中，與使用默認值相比，上述參與度估算器減少了−42%的參與度估計誤差。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在在線A\/B實驗中，我們觀察到新創建的的預訂量提高了+14%，同時首頁結果的瀏覽量增加了+14%。除了對新房源的影響外，整體預訂量增加了0.38%，這表明用戶體驗總體上有所改善。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"從研究向DNN提供數據的挑戰，我們過渡到圍繞DNN如何解釋呈現數據的問題，以及位置偏差的問題。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"消除位置偏差"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們調查位置偏差的出發點是完全不相關的。與新房源的NDCG較低的觀察結果類似，另一個表現低於預期的細分市場是精品酒店和傳統的牀和早餐，作爲庫存的一部分，這一部分增長迅速。從觀察中得出的一個假設是，由於位置偏差，在訓練數據中歷史上被低估的庫存沒有得到最佳排序。但與新房源業績與冷啓動之間的聯繫不同，沒有強有力的理由相信位置偏差是這起案件的唯一罪魁禍首；還有其他多種假設。雖然我們發現關注用戶問題比簡單地從文獻調查中導入想法要好得多，但這本身並不是萬能藥。在用戶問題和模型中的缺陷之間建立因果關係遠非易事。在當前的場景中，我們是在黑暗中拍攝。但在這一過程中，我們決定尋找能夠解釋觀測結果的模型中最大的差距。文獻調查對於確定我們模型中潛在的主要差距是至關重要的。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.1 相關工作"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"給定發出查詢q的用戶u，用戶從搜索結果預訂房源l的概率可以分解爲兩個因素："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（1）房源與用戶。這個概率可以表示爲P（relative=1 | l，u，q），表示房源、用戶和查詢的顯式依賴關係。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（2）用戶在搜索結果的k位置檢查房源的概率。這可能取決於用戶（例如，移動用戶可能對最高結果有更高的偏好）或查詢（例如，短期的用戶可能更不關注底部結果）。我們將此概率表示爲P（examined=1 | k，u，q），獨立於房源l。房源對預訂事件的影響完全由P（relevant=1 | l，u，q）來解釋。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"利用文獻[4]中描述的基於位置的模型的簡化假設，我們將用戶預訂房源的概率表示爲兩個分解概率的乘積。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/60\/609fe4a6645bccd0af2c42a010c68f5f.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"通過直接訓練一個預測預訂量的模型，該模型學習預測依賴於P（examined=1 | k，u，q）的P（booking），而這又取決於前一個排名模型所決定的位置k。因此，當前模型將依賴於以前的模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"理想情況下，我們希望模型只關注P（relevant=1 | l，u，q）並僅根據相關性對列表進行排名。爲此，介紹了一種包含兩個關鍵概念的方法："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（1）預測P（examined=1 | k，u，q）的傾向模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（2）用預測傾向的倒數來衡量每個訓練樣本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"當構造傾向模型時，通常需要擾動搜索結果以收集反事實的例子，描述了在不需要額外干預的情況下構造傾向模型的方法。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.2 位置和控制變量"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們的解決方案有兩個關鍵亮點。首先，它是非侵入性的，不需要對搜索結果進行任何隨機化。我們依賴Airbnb搜索結果的一些獨特屬性，這些屬性使得房源出現在不同的位置，即使它們在排名時的相應分數或多或少是不變的："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（1）房源表示在給定日期範圍內只能預訂一次的實體。當房源被預訂並從搜索中消失時，它會改變其餘房源的位置。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（2）每個房源都有自己獨特的日曆可用性，因此不同的房源會出現在不同的位置，以便跨日期範圍進行類似的查詢。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們解決方案的第二個亮點是我們沒有建立一個明確的傾向模型。相反，我們在DNN中引入位置作爲一個特徵，通過dropout正則化。在評分過程中，我們將位置特徵設置爲0。本節的其餘部分描述了爲什麼這是有效的直覺。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們以第2.6節中描述的DNN爲基礎，以查詢、用戶和房源特徵作爲輸入。使用符號q（查詢特性）、u（用戶特性）、l（房源特性）、θ（DNN參數），我們將DNN的輸出表示爲："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6e\/6ecdc0bde45130616453093951d6b1eb.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"反映了[4]中基於位置模型的假設。在這裏，relθ（q，u，l）估計P（relevance=1 | l，u，q），我們稱之爲相關性預測。pbiasθ（q，u，l）估計P（examination=1 | k，u，q），我們稱之爲位置偏差預測。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"很明顯，pbiasθ（q，u，l）缺少房源k作爲輸入的位置，因爲它試圖估計的數量依賴於k。因此，我們的第一步是將k作爲輸入特徵添加到DNN中。由於關聯預測和位置偏差預測都是由DNN輸入提供的，所以在輸入中添加k將把DNN的表示轉換爲："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/c1\/c17f374e5bacc20002110cb88b558c97.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 假設P（examined=1 | k，u，q）與l無關，位置偏差預測對l的任何依賴都可以被視爲誤差。我們假設在訓練數據量足夠大的情況下，學習的參數θ能夠使誤差最小化，並且位置偏差預測在所有實際應用中都與l無關。我們把這個假設理解爲:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/91\/9197af744ca0d0e51906f7d47d10ab31.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"從pbias θ（q，u，l，k）中刪除l。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在打分時，我們將位置特徵k設置爲0。在給定的搜索中，q和u在DNN評分的房源中是不變的。我們使用Q和U來表示特定搜索的查詢和用戶特徵。因此，位置偏差預測變成pbiasθ（Q，U，0），這是特定搜索結果中所有房源的不變量。將不變量β命名爲，評分時方程（5）可改寫爲:"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/af\/af5fcb1acc16d3d97f56338aa5b4ba45.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"這使得兩個房源的分數比較不受位置偏差的影響，並且只依賴於房源的相關性。本質上，我們在排名模型中添加了位置作爲控制變量[18]。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.3 位置Dropout"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在基於位置的模型假設下，加入位置作爲控制變量可以有效地消除房源排名中的位置偏差預測，但這又帶來了一個新的問題。關聯預測現在依賴於位置作爲特徵。這就存在着DNN在訓練過程中依賴位置特徵來預測相關性的風險，但是在位置特徵總是設置爲0的情況下，無法利用這種學習來進行評分。將具有位置作爲特徵的DNN的NDCG與沒有位置特徵的基線相比，我們看到大約-1.3%的下降。因此，直接引入位置作爲控制變量似乎會損害相關性預測。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"爲了減少相關性預測對位置特徵的依賴性，我們使用dropout將其正則化[13]。在訓練期間，我們將房源的位置設置爲0，由dropout率控制。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Dropout 率在無噪聲訪問位置特徵以準確推斷位置偏差與使位置特徵具有噪聲以使其遠離相關性預測之間進行權衡。我們試圖通過以下步驟找到平衡點："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（1）掃描dropout率的範圍，並在一個測試集上計算NDCG的兩個偏好。首先在測試期間將位置設置爲0。它來測量相關性預測並且表示爲NDCG rel。第二種方法通過保持位置特徵來度量相關性和位置偏差預測，即NDCG（rel+pbias）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（2）減去NDCG（rel+pbias）−NDCG（rel）得到位置偏差預測的度量。這裏的直覺是，通過比較有無位置輸入的排名質量，我們可以估計出位置對排名的貢獻。根據NDCG rel繪製它，以獲得圖9中的曲線。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f0\/f04e641f4136cce6514d76c166cb8709.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"（3）爲了在相關性預測和位置偏差預測之間取得平衡，在曲線上選擇位置偏差預測在x軸上足夠提前的點，而不會在y軸上引起太多的相關性預測下降。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"通過這個實驗，我們最終選擇了0.15的輟學率。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4.4 測試結果"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"我們通過一個在線A\/B測試來測試這個想法，其中控制是來自第2.6節的DNN，沒有位置偏差的概念。討論的是相同的DNN，但訓練的特徵是位置作爲特徵，dropout率爲0.15。在在線測試中，我們發現預訂量增加了0.7%。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"在預訂量增加的同時，收入增長了1.8%，這是一個令人驚喜的驚喜。收入的副作用說明了位置偏差是如何在模型的多次迭代中建立起來的。對於排名模型來說，相對容易瞭解價格的影響，因爲它是一個非常乾淨的特徵，而且數據強烈表明人們傾向於更低的價格。質量、位置等的平衡力更難學。因此，最初的簡化模型嚴重依賴於較低的價格。經過多次模型迭代，我們提高了對質量和位置的理解，但到那時，對更便宜價格的偏見已經在訓練中根深蒂固數據。這個粘性使得連續幾個模型高估了低價的偏好。消除位置偏差可以使模型更接近客人的真實偏好，並在價格、質量和位置之間取得更優的平衡。由此帶來的直接後果就是收入增長。最後，爲了結束我們的起點，我們觀察到精品酒店的預訂量增加了+1.1%。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"結論"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"深度學習在Airbnb的搜索排名中繼續蓬勃發展。我們由衷地感謝社區爲我們提供了深入學習的生態系統、開放的思想交流以及通過分享我們自己的經驗加入對話的機會。但我們旅程的亮點在於，我們意識到，要突破DNN的界限，靈感不會來自於某些外部來源。爲此，我們必須聽從用戶的領導。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"論文原文鏈接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/pdf\/2002.05515.pdf","title":null,"type":null},"content":[{"type":"text","text":"https:\/\/arxiv.org\/pdf\/2002.05515.pdf"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]}]}]}