revised note on learing quardatic assignment wiht graph neural networks
關於使用 圖神經網絡 學習二次分配的修訂說明
摘要
| Inverse problems correspond to a certain type of optimization problems formulated over appropriate input distributions. Recently,there has been a growing interest in understanding the computational hardness of these optimization problems, not only in the worst case,but in an average-complexity sense under this same input distribution. | 反問題對應於在適當的輸入分佈上形成的某種類型的優化問題。 最近，人們越來越關注理解這些優化問題的計算硬度，不僅在最壞的情況下，而且在同一輸入分佈下的平均複雜度意義上。 |
| In this revised note, we are interested in studying another aspect of hardness, related to the ability to learn how to solve a problem by simply observing a collection of previously solved instances. These ‘planted solutions’ are used to supervise the training of an appropriate predictive model that parametrizes a broad class of algorithms, with the hope that the resulting model will provide good accuracycomplexity tradeoffs in the average sense. | 在本修訂說明中，我們有興趣研究硬度的另一個方面，與通過簡單地觀察先前解決的實例的集合來學習如何解決問題的能力相關。 這些“種植的解決方案”用於監督適當的預測模型的訓練，該模型參數化廣泛的算法類，希望得到的模型能夠在平均意義上提供良好的精度複雜性權衡。 |
| We illustrate this setup on the Quadratic Assignment Problem,a fundamental problem in Network Science. We observe that datadriven models based on Graph Neural Networks offer intriguingly good performance, even in regimes where standard relaxation based techniques appear to suffer. | 我們在二次分配問題上說明了這種設置，這是網絡科學中的一個基本問題。 我們觀察到基於圖形神經網絡的數據驅動模型提供了有趣的良好性能，即使在基於標準鬆弛的技術似乎受到影響的情況下也是如此。 |
簡介
| Many tasks, spanning from discrete geometry to statistics, are defined in terms of computationally hard optimization problems. Loosely speaking, computational hardness appears when the algorithms to compute the optimum solution scale poorly with the problem size, say faster than any polynomial. For instance, in highdimensional statistics we may be interested in the task of estimating a given object from noisy measurements under a certain generative model. In that case, the notion of hardness contains both a statistical aspect, that asks above which signal-to-noise ratio the estimation is feasible, and a computational one, that restricts the estimation to be computed in polynomial time. An active research area in Theoretical Computer Science and Statistics is to understand the interplay between those statistical and computational detection thresholds;see [1] and references therein for an instance of this program in the community detection problem, or [3, 4, 7] for examples of statistical inference tradeoffs under computational constraints. | 從離散幾何到統計的許多任務都是根據計算上的硬優化問題來定義的。簡而言之，當計算最優解的算法與問題大小一致時，計算硬度會出現，比任何多項式都要快。例如，在高維統計中，我們可能對在某個生成模型下從噪聲測量估計給定對象的任務感興趣。在這種情況下，硬度的概念既包含統計方面，也包括估計可行的信噪比，以及計算方法，其限制在多項式時間內計算估計。理論計算機科學與統計學的一個活躍的研究領域是理解這些統計學和計算檢測閾值之間的相互作用;參見[1]並在其中參考社區檢測問題中該程序的實例，或[3,4,7]例如，計算約束下的統計推斷權衡。 |
| Instead of investigating a designed algorithm for the problem in question, we consider a data-driven approach to learn algorithms from solved instances of the problem. In other words, given a collection (xi, yi)i≤L of problem instances drawn from a certain distribution, we ask whether one can learn an algorithm that achieves good accuracy at solving new instances of the same problem – also being drawn from the same distribution, and to what extent the resulting algorithm can reach those statistical/computational thresholds. | 我們不考慮針對相關問題設計算法，而是考慮採用數據驅動方法從問題的解決實例中學習算法。 換句話說，給定從某個分佈中抽取的問題實例的集合（xi，yi）i≤L，我們會問是否可以學習一種算法，該算法在解決同一問題的新實例時可以獲得良好的準確性 - 也可以從 相同的分佈，以及所得算法達到那些統計/計算閾值的程度。 |
| The general approach is to cast an ensemble of algorithms as neural networks ˆy = f(x; θ) with specific architectures that encode prior knowledge on the algorithmic class, parameterized by θ ∈ RS. The network is trained to minimize the empirical loss L(θ) , for a given measure of error ℓ, using stochastic gradient descent. This leads to yet another notion of learnability hardness, that measures to what extent the problem can be solved with no prior knowledge of the specific algorithm to solve it, but only a vague idea of which operations it should involve. | 一般方法是將算法集合作爲神經網絡y = f（x;θ），使用特定的體系結構編碼算法類的先驗知識，由θ∈RS參數化。 對於給定的誤差量l，使用隨機梯度下降來訓練網絡以最小化經驗損失L（θ）。 這導致了另一個可學習性硬度的概念，即在沒有解決它的特定算法的先驗知識的情況下測量問題可以解決的程度，而只是模糊地考慮它應該涉及哪些操作。 |
| In this revised version of [20] we focus on a particular NP-hard problem, the Quadratic Assignment Problem (QAP), and study datadriven approximations to solve it. Since the problem is naturally formulated in terms of graphs, we consider the so-called Graph Neural Network (GNN) model [27]. This neural network alternates between applying linear combinations of local graph operators – such as the graph adjacency or the graph Laplacian, and pointwise nonlinearities,and has the ability to model some forms of non-linear message passing and spectral analysis, as illustrated for instance in the data-driven Community Detection methods in the Stochastic Block Model [6]. Existing tractable algorithms for the QAP include spectral alignment methods [30] and methods based on semidefinite programming relaxations [10,33]. Our preliminary experiments suggest that the GNN approach taken here may be able to outperform the spectral and SDP counterparts on certain random graph models,at a lower computational budget. We also provide an initial analysis of the learnability hardness by studying the optimization landscape of a simplified GNN architecture. Our setup reveals that, for the QAP, the landscape complexity is controlled by the same concentration of measure phenomena that controls the statistical hardness; see Section 4. | 在[20]的修訂版本中，我們關注特定的NP難問題，二次分配問題（QAP），以及研究數據驅動的近似值來解決它。由於問題是根據圖形自然形成的，我們考慮所謂的圖形神經網絡（GNN）模型[27]。該神經網絡在應用局部圖運算符的線性組合（例如圖鄰接或圖拉普拉斯算子）和逐點非線性之間交替，並且能夠模擬某些形式的非線性消息傳遞和頻譜分析，如例如隨機塊模型中的數據驅動的社區檢測方法[6]。用於QAP的現有易處理算法包括譜對齊方法[30]和基於半定規劃鬆弛的方法[10,33]。我們的初步實驗表明，這裏採用的GNN方法可能能夠在較低的計算預算下優於某些隨機圖模型上的頻譜和SDP對應物。我們還通過研究簡化GNN架構的優化環境，對可學習性硬度進行了初步分析。我們的設置表明，對於QAP，景觀複雜性由控制統計硬度的相同濃度的測量現象控制;見第4節。 |
| The rest of the paper is structured as follows. Section 2 presents the problem set-up and describes existing relaxations of the QAP. Section 3 describes the graph neural network architecture, Section 4 presents our landscape analysis, and Section 5 presents our numerical experiments. Finally, Section 6 describes some open research directions motivated by our initial findings. | 本文的其餘部分的結構如下。 第二部分提出問題，建立並介紹了QAP現有鬆弛。 第3節描述了圖神經網絡結構，第4節描述了我們的景觀分析，第5節介紹了我們的數值實驗。 最後，第6節描述了一些由我們的初步發現驅動的開放式研究方向。 |