0.5%入選率！NeurIPS 2019熱門論文作者獨家專訪

????點擊上方藍字星標“Robinly”，獲取更多重磅AI訪談

Robin.ly 是立足硅谷的視頻內容平臺，服務全球工程師和研究人員，通過與知名人工智能科學家、創業者、投資人和領導者的深度對話和現場交流活動，傳播行業動態和商業技能，打造人才全方位競爭力。

2019年12月8日，神經計算和機器學習領域規模最大的頂會 NeurIPS（神經信息處理系統會議）於加拿大溫哥華拉開帷幕。因註冊人數過多，今年參會門票都要憑運氣抽獎決定。據大會官方統計，今年參會總人數已經突破了 13000 人，與2018年相比，幾乎上漲了50%。

今年大會論文投稿數量也創下了歷史新高，最終，共提交6743 篇有效論文，接收 1428 篇，接受率爲 21.17%。而其中只有36篇獲得15分鐘的口頭報告資格，入選率僅爲0.53%！Robin.ly在大會現場特邀其中20位作者，快速分享論文亮點和應用場景。

今天首先分享三篇來自麻省理工學院不同方向的熱點論文，包括：

Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs
On Robustness of Principal Component Regression
Kernel Instrumental Variable Regression

我們也特別邀請到圖靈獎得主、深度學習三巨頭之一的Yoshua Bengio、兩位大會最佳論文作者、以及多位AI大牛現場深度對話，更多精彩內容將陸續分享，關注我們的公衆號Robinly，及時獲得更新！

長按二維碼或點擊“閱讀原文”

獲取更多NeurIPS英文訪談實錄

類似大腦的物體識別

論文：Brain-Like Object Recognition with High-Performing Shallow Recurrent ANNs

作者：Jonas Kubilius，Martin Schrimpf 等

機構：麻省理工大學、灣區實驗室、紐約大學、哥倫比亞大學、斯坦福大學

論文鏈接：

https://arxiv.org/abs/1909.06161

Martin Schrimpf是麻省理工大學博士研究生，師從James J. DiCarlo教授，從事神經科學與機器學習方向的研究。在論文中，他們提出了可以量化比較大腦系統和人工模型表現的Brain Score，並參照人類大腦的工作原理，重新設計開發了具有循環結構的淺層人工神經網絡CORnet-S，獲得了類似大腦視覺系統的物體識別表現。

論文摘要：Deep convolutional artificial neural networks (ANNs) are the leading class of candidate models of the mechanisms of visual processing in the primate ventral stream. While initially inspired by brain anatomy, over the past years, these ANNs have evolved from a simple eight-layer architecture in AlexNet to extremely deep and branching architectures, demonstrating increasingly better object categorization performance, yet bringing into question how brain-like they still are. In particular, typical deep models from the machine learning community are often hard to map onto the brain's anatomy due to their vast number of layers and missing biologically-important connections, such as recurrence. Here we demonstrate that better anatomical alignment to the brain and high performance on machine learning as well as neuroscience measures do not have to be in contradiction. We developed CORnet-S, a shallow ANN with four anatomically mapped areas and recurrent connectivity, guided by Brain-Score, a new large-scale composite of neural and behavioral benchmarks for quantifying the functional fidelity of models of the primate ventral visual stream. Despite being significantly shallower than most models, CORnet-S is the top model on Brain-Score and outperforms similarly compact models on ImageNet. Moreover, our extensive analyses of CORnet-S circuitry variants reveal that recurrence is the main predictive factor of both Brain-Score and ImageNet top-1 performance. Finally, we report that the temporal evolution of the CORnet-S "IT" neural population resembles the actual monkey IT population dynamics. Taken together, these results establish CORnet-S, a compact, recurrent ANN, as the current best model of the primate ventral visual stream.

圖片來源：Martin Schrimpf

主成分迴歸分析的魯棒性

論文：On Robustness of Principal Component Regression

作者：Anish Agarwal，Devavrat Shah，Dennis Shen，Dogyoon Song

機構：麻省理工學院

論文鏈接：

https://arxiv.org/abs/1902.10920

Anish Agarwal是麻省理工學院電子電氣與計算機學三年級博士研究生，其研究方向是高維數理統計以及數據市場設計。這篇論文通過嚴格的理論分析，證實在工業界廣泛使用的主成分迴歸分析方法（principle component regression；PCR），在處理噪聲、離散、缺失數據方面具有很強的魯棒性，甚至超越現實迴歸。近年來，保護用戶隱私的需求日益增長，產生了大量不完整數據，這種分析方法也越發重要。

論文摘要：Consider the setting of Linear Regression where the observed response variables, in expectation, are linear functions of the p-dimensional covariates. Then to achieve vanishing prediction error, the number of required samples scales faster than pσ2, where σ2 is a bound on the noise variance. In a high-dimensional setting where p is large but the covariates admit a low-dimensional representation (say r ≪ p), then Principal Component Regression (PCR), cf. [36], is an effective approach; here, the response variables are regressed with respect to the principal components of the covariates. The resulting number of required samples to achieve vanishing prediction error now scales faster than rσ2(≪ pσ2). Despite the tremendous utility of PCR, its ability to handle settings with noisy, missing, and mixed (discrete and continuous) valued covariates is not understood and remains an important open challenge, cf. [24]. As the main contribution of this work, we address this challenge by rigorously establishing that PCR is robust to noisy, sparse, and possibly mixed valued covariates. Specifically, under PCR, vanishing prediction error is achieved with the number of samples scaling as r max(σ2, ρ−4 log5(p)), where ρ denotes the fraction of observed (noisy) covariates. We establish generalization error bounds on the performance of PCR, which provides a systematic approach in selecting the correct number of components r in a data-driven manner. The key to our result is a simple, but powerful equivalence between (i) PCR and (ii) Linear Regression with covariate pre-processing via Hard Singular Value Thresholding (HSVT). From a technical standpoint, this work advances the state-of-the-art analysis for HSVT by establishing stronger guarantees with respect to the ∥·∥2,∞-error for the estimated matrix rather than the Frobenius norm/mean-squared error (MSE) as is commonly done in the matrix estimation / completion literature.

圖片來源：Anish Agarwal

內核工具變量回歸

論文：Kernel Instrumental Variable Regression

作者：Rahul Singh，Maneesh Sahani，Arthur Gretton

機構：麻省理工學院、倫敦大學學院

論文鏈接：

https://arxiv.org/abs/1906.00232

Rahul Singh是麻省理工經濟與統計學三年級博士生，其主要研究方向是因果推斷與統計學習理論。本篇文章的綜合運用了計量經濟學與深度學習理論，提出了一種基於非線性關係的核心工具變量回歸方法（kernel instrumental variable regression）。該論文的算法模型僅有三行代碼，卻可以潛在應用於分析具有混雜關係的數據，如市場需求分析 (market demand) 和部分依從（imperfect complaince）的臨牀對照實驗。

論文摘要：Instrumental variable (IV) regression is a strategy for learning causal relationships in observational data. If measurements of input X and output Y are confounded, the causal relationship can nonetheless be identified if an instrumental variable Z is available that influences X directly, but is conditionally independent of Y given X and the unmeasured confounder. The classic two-stage least squares algorithm (2SLS) simplifies the estimation problem by modeling all relationships as linear functions. We propose kernel instrumental variable regression (KIV), a nonparametric generalization of 2SLS, modeling relations among X, Y, and Z as nonlinear functions in reproducing kernel Hilbert spaces (RKHSs). We prove the consistency of KIV under mild assumptions, and derive conditions under which convergence occurs at the minimax optimal rate for unconfounded, single-stage RKHS regression. In doing so, we obtain an efficient ratio between training sample sizes used in the algorithm's first and second stages. In experiments, KIV outperforms state of the art alternatives for nonparametric IV regression.