Hennig P, Schuler C J. Entropy search for information-efficient global optimization[J]. Journal of Machine Learning Research, 2012, 13(1): 1809-1837.
@article{hennig2012entropy,
貝葉斯優化中的 Entropy Search (EI) 方法.
這篇文章關注的是max x ∈ I f ( x ) ,
\max_{x \in I} \:f(x),
x ∈ I max f ( x ) , I I I
一般, 通過高斯過程定義f ( x ) f(x) f ( x ) f ( x ) ∼ N ( u 0 ( x ) , k ( x , x ) ) , y = f ( x ) + ϵ , ϵ ∼ N ( 0 , σ 2 ) .
f(x) \sim \mathcal{N}(u_0(x), k(x,x)),\\
y = f(x) + \epsilon, \: \epsilon \sim \mathcal{N}(0, \sigma^2).
f ( x ) ∼ N ( u 0 ( x ) , k ( x , x ) ) , y = f ( x ) + ϵ , ϵ ∼ N ( 0 , σ 2 ) . X = { x 1 , … , x T } X = \{x_1, \ldots, x_T\} X = { x 1 , … , x T } Y = { y 1 , … , y T } Y = \{y_1, \ldots, y_T\} Y = { y 1 , … , y T } f ( x ∗ ) f(x^*) f ( x ∗ ) μ ( x ∗ ) = μ 0 ( x ∗ ) + k ( x ∗ , X ) T [ k ( X , X ) + σ I ] − 1 ( Y − u 0 ( X ) ) σ ∗ 2 ( x ∗ ) = k ( x ∗ , x ∗ ) − k ( x ∗ , X ) T ( k ( X , X ) + σ I ) − 1 k ( x ∗ , X )
\mu (x^*) = \mu_0(x^*) + k(x^*, X)^T [k(X, X) + \sigma I]^{-1}(Y-u_0(X)) \\
\sigma_*^2(x^*)=k(x^*, x^*) - k(x^*, X)^T (k(X,X)+\sigma I)^{-1}k(x^*,X)
μ ( x ∗ ) = μ 0 ( x ∗ ) + k ( x ∗ , X ) T [ k ( X , X ) + σ I ] − 1 ( Y − u 0 ( X ) ) σ ∗ 2 ( x ∗ ) = k ( x ∗ , x ∗ ) − k ( x ∗ , X ) T ( k ( X , X ) + σ I ) − 1 k ( x ∗ , X )
我們的目的是在已有這些條件的基礎上, 尋找下一個(或多個)評估點.
定義:p m i n ( x ) = p [ x = arg min f ( x ) ] = ∫ f : I → R p ( f ) ∏ x ~ ∈ I , x ~ ≠ x θ [ f ( x ~ ) − f ( x ) ] d f , (1)
\tag{1}
p_{min}(x) =p[x = \arg \min f(x)] = \int_{ f:I \rightarrow R} p(f) \prod_{\tilde{x} \in I, \tilde{x} \not = x} \theta[f(\tilde{x})-f(x)] \mathrm{d}f,
p m i n ( x ) = p [ x = arg min f ( x ) ] = ∫ f : I → R p ( f ) x ~ ∈ I , x ~ = x ∏ θ [ f ( x ~ ) − f ( x ) ] d f , ( 1 ) θ ( x ) = 1 , x ≥ 0 , e l s e 0 \theta(x) = 1, x\ge0, else \: 0 θ ( x ) = 1 , x ≥ 0 , e l s e 0 ∏ \prod ∏ x x x
再定義損失函數(當然損失函數不選擇KL散度也是可以的, 但這是EI的名字的由來):L ( p m i n ) = L K L ( p m i n ) = − ∫ I p m i n ( x ) log p m i n ( x ) b ( x ) d x . (2)
\tag{2}
\mathcal{L}(p_{min}) = \mathcal{L}_{KL}(p_{min})=-\int_Ip_{min}(x) \log \frac{p_{min}(x)}{b(x)} \mathrm{d}x.
L ( p m i n ) = L K L ( p m i n ) = − ∫ I p m i n ( x ) log b ( x ) p m i n ( x ) d x . ( 2 ) b ( x ) b(x) b ( x ) I I I L \mathcal{L} L p m i n p_{min} p m i n f ( x ) f(x) f ( x )
但是這樣還不夠, 我們進一步關心其期望損失(最小化):
⟨ L ⟩ x = ∫ p ( y ∣ x ) L ( p m i n ( ⋅ ∣ Y , X , y , x ) ) d y . (3)
\tag{3}
\langle \mathcal{L} \rangle_{x} = \int p(y|x) \mathcal{L}(p_{min} (\cdot|Y, X, y, x)) \mathrm{d}y.
⟨ L ⟩ x = ∫ p ( y ∣ x ) L ( p m i n ( ⋅ ∣ Y , X , y , x ) ) d y . ( 3 )
通過最小化(3)，我們可以獲得接下來的評估點.
接下來的問題是如果去估計.
p m i n p_{min} p m i n 比較麻煩的是∏ \prod ∏ N N N x ~ = { x ~ 1 , … , x ~ N } \tilde{x} = \{\tilde{x}_1, \ldots, \tilde{x}_N\} x ~ = { x ~ 1 , … , x ~ N } N N N u u u ( X , Y ) (X, Y) ( X , Y ) u ( X , Y ) u(X, Y) u ( X , Y ) x ~ \tilde{x} x ~ u u u p m i n p_{min} p m i n
文中給了倆種方法, 一種直接的方法是p m i n p_{min} p m i n
一下是我猜想的用MC積分的方式(文中未給出具體的形式)"
根據一定策略選取x ~ \tilde{x} x ~
重複J次:
根據概率p ( f ) p(f) p ( f ) f ( x ~ ) , f ( x ) f(\tilde{x}), f(x) f ( x ~ ) , f ( x )
計算∏ \prod ∏
作者選擇的是 Expectation Propagation (EP）的方法, 這種方法能夠估計出x ~ i , i = 1 , … , N \tilde{x}_i, i=1,\ldots,N x ~ i , i = 1 , … , N q m i n ( x i ~ ) q_{min}(\tilde{x_i}) q m i n ( x i ~ ) f m i n f_{min} f m i n x ~ i \tilde{x}_i x ~ i N N N ( N u ( x ~ i ) ) − 1 (Nu(\tilde{x}_i))^{-1} ( N u ( x ~ i ) ) − 1 p m i n ( x ) ≈ q m i n ( x i ) N u ( x ~ i ) Z u , Z u = ∫ u ( x ) d x , x i = arg min x i ∈ x ~ ∥ x − x i ∥ .
p_{min}(x) \approx \frac{q_{min}(x_i)Nu(\tilde{x}_i)}{Z_u}, \: Z_u=\int u(x) \mathrm{d}x, \: x_i = \arg \min_{x_i \in \tilde{x}} \|x-x_i\|.
p m i n ( x ) ≈ Z u q m i n ( x i ) N u ( x ~ i ) , Z u = ∫ u ( x ) d x , x i = arg x i ∈ x ~ min ∥ x − x i ∥ .
這樣我們就完成了p m i n p_{min} p m i n q m i n q_{min} q m i n μ , σ ∗ \mu, \sigma_* μ , σ ∗ Z u Z_u Z u
L K L \mathcal{L}_{KL} L K L p ^ m i n = q m i n \hat{p}_{min}=q_{min} p ^ m i n = q m i n
⟨ Δ L ⟩ \langle \Delta \mathcal{L} \rangle ⟨ Δ L ⟩ arg min X ⟨ L ⟩ X \arg \min_X \langle \mathcal{L} \rangle_X arg min X ⟨ L ⟩ X
最後給出算法:
