python：scipy.optimize.fmin

簡介

優化問題是工程實踐中經常遇到的一種問題。簡單講，就是搜索優化出一組自變量參數，使得目標函數達到極小值(極大值)。

如何搜索出這組參數呢：這就是優化算法做的事情。不同的優化算法適用於不同的優化問題。

本文簡要介紹在python種NM算法來解決局部優化問題。

注意：scipy.optimize中的fmin和minimize都能調用NM算法來優化。兩者區別：

• minimize是更通用的優化算法接口，它不僅包含NM算法，也可以調用其他算法；而fmin就只能用NM算法
• minimize中的NM算法多一個adaptive參數，可以提高高維的優化問題的收斂速度
• fmin的一個好處：支持直接返回優化過程中的迭代參數，有助於可視化。

本文用fmin來重新實現一遍minimize的例子。

接口

實例1：Himmelblau函數

優化流程

代碼

def cost_function(x):
    return (x[0]**2+x[1]-11)**2+(x[0]+x[1]**2-7)**2

x_center = np.array([0,0])
step = 0.5
x0 = np.vstack((x_center, x_center+np.diag((step,step))))
xtol,ftol = 1e-3,1e-3
xopt,fopt,iter,funcalls,warnflags,allvecs = fmin(cost_function,x_center,initial_simplex=x0,xtol=xtol,ftol=ftol,disp=1,retall=1,full_output=1)
print(xopt,fopt)

n = 50
x = np.linspace(-6,6,n)
y = np.linspace(-6,6,n)
z = np.zeros((n,n))
for i,a in enumerate(x):
    for j,b in enumerate(y):
        z[i,j] = cost_function([a,b])

xx, yy = np.meshgrid(x,y)
fig, ax = plt.subplots()
c = ax.pcolormesh(xx,yy,z.T,cmap='jet')
fig.colorbar(c, ax=ax)

t = np.asarray(allvecs)
x_, y_ = t[:,0], t[:,1]
ax.plot(x_,y_,'r',x_[0],y_[0],'go',x_[-1],y_[-1],'y+',markersize=6)

fig2 = plt.figure()
ax1 = plt.subplot(221)
ax2 = plt.subplot(222)
ax3 = plt.subplot(212)
ax1.plot(x_)
ax1.set_title('x')
ax2.plot(y_)
ax2.set_title('y')
ax3.plot(ys)

結果

Optimization terminated successfully.
         Current function value: 0.000002
         Iterations: 35
         Function evaluations: 68
[3.00021471 1.99974856] 1.7007332966814985e-06

對初值敏感

代碼

def cost_function(x):
    return (x[0]**2+x[1]-11)**2+(x[0]+x[1]**2-7)**2

n = 50
x = np.linspace(-6,6,n)
y = np.linspace(-6,6,n)
z = np.zeros((n,n))
for i,a in enumerate(x):
    for j,b in enumerate(y):
        z[i,j] = cost_function([a,b])
xx, yy = np.meshgrid(x,y)
fig, axes = plt.subplots(2, 2, figsize=(12,8))

centers = [[0,0],[-1,0],[0,-1],[-1,-1]]
for i,center in enumerate(centers):
    x_center = np.array(center)
    step = 0.5
    x0 = np.vstack((x_center, x_center+np.diag((step,step))))
    xtol,ftol = 1e-3,1e-3
    xopt,fopt,iter,funcalls,warnflags,allvecs = fmin(cost_function,x_center,initial_simplex=x0,xtol=xtol,ftol=ftol,disp=1,retall=1,full_output=1)
    print(xopt,fopt)

    ii,jj = i//2,i%2
    ax = axes[ii][jj]
    c = ax.pcolormesh(xx,yy,z.T,cmap='jet')
    fig.colorbar(c, ax=ax)

    t = np.asarray(allvecs)
    x_, y_ = t[:,0], t[:,1]
    ax.plot(x_,y_,'r',x_[0],y_[0],'go',x_[-1],y_[-1],'y+',markersize=6)

結果

Optimization terminated successfully.
         Current function value: 0.000002
         Iterations: 35
         Function evaluations: 68
[3.00021471 1.99974856] 1.7007332966814985e-06
Optimization terminated successfully.
         Current function value: 0.000003
         Iterations: 31
         Function evaluations: 60
[-2.80534914  3.13148297] 2.851032148018009e-06
Optimization terminated successfully.
         Current function value: 0.000001
         Iterations: 40
         Function evaluations: 74
[ 3.58440786 -1.8484002 ] 1.1592413943034185e-06
Optimization terminated successfully.
         Current function value: 0.000002
         Iterations: 37
         Function evaluations: 69
[-3.77937271 -3.28299281] 2.2138273459910166e-06