運籌系列32：APMonitor/Gekko套裝

1. 介紹

APM是Advanced Process Monitor，可用於混合整數規劃（LP, QP, NLP, MILP, MINLP）和微分方程求解。而GEKKO是在此之上的升級版本，使用python語言。安裝很簡單pip install gekko即可.
快速入門的例子看這裏：https://gekko.readthedocs.io/en/latest/examples.html
這裏是教程，真是太貼心了：
https://apmonitor.com/wiki/index.php/Main/GekkoPythonOptimization
GEKKO可選的求解器有1: APOPT, 2: BPOPT, 3: IPOPT。此外，也可以調用MINOS、SNOPT等。
IPOPT一般用於問題自由度（自由度=變量個數-約束個數）很大、初始估計較好的情況；BPOPT則是在系統生物應用中使用；APOPT用於問題自由度小於2000的情形。如果有整數約束，則只能使用APOPT。

這裏可以在線嘗試：http://apmonitor.com/。

2. 建模

Gekko常見變量：Constants, Parameters, Variables and Intermediates，顧名思義是常數、參數、變量和顯式變量。
添加約束條件：m.Equations(eqs)
添加目標函數：m.Obj(obj)
connections算是官方的化簡工具，兩個變量如果有直接的關係，則只顯示第一個。
dt()函數表示微分。
time參數可以將所有的約束在每個時間點上都建立起來。
options參數可以設置全局參數。
Array可以方便的創建多維數組，例如x = m.Array(m.Var,(n,p))
solve(disp=True, debug=False)用於求解
max/min有各種編號，具體看文檔。
sos1：specical ordered set。

下面是官方的例子：

from gekko import GEKKO

#Initialize Model
m = GEKKO()

#define parameter
eq = m.Param(value=40)

#initialize variables
x1,x2,x3,x4 = [m.Var(lb=1, ub=5) for i in range(4)]

#initial values
x1.value = 1
x2.value = 5
x3.value = 5
x4.value = 1

#Equations
m.Equation(x1*x2*x3*x4>=25)
m.Equation(x1**2+x2**2+x3**2+x4**2==eq)

#Objective
m.Obj(x1*x4*(x1+x2+x3)+x3)

#Set global options
m.options.IMODE = 3 #steady state optimization

#Solve simulation
m.solve()

#Results
print('')
print('Results')
print('x1: ' + str(x1.value))
print('x2: ' + str(x2.value))
print('x3: ' + str(x3.value))
print('x4: ' + str(x4.value))

返回結果如下：

 ----------------------------------------------------------------
 APMonitor, Version 0.9.2
 APMonitor Optimization Suite
 ----------------------------------------------------------------
 
 
 --------- APM Model Size ------------
 Each time step contains
   Objects      :            0
   Constants    :            0
   Variables    :            6
   Intermediates:            0
   Connections  :            0
   Equations    :            3
   Residuals    :            3
 
 Number of state variables:              5
 Number of total equations: -            2
 Number of slack variables: -            1
 ---------------------------------------
 Degrees of freedom       :              2
 
 solver            3  not supported
 using default solver: APOPT
 ----------------------------------------------
 Steady State Optimization with APOPT Solver
 ----------------------------------------------
 
 Iter    Objective  Convergence
    0  1.67676E+01  1.87500E-01
    1  2.04736E+01  4.88281E-02
    2  1.75530E+01  2.17630E-02
    3  1.70459E+01  9.65302E-03
    4  1.70140E+01  2.04419E-04
    5  1.70140E+01  4.22153E-08
    6  1.70140E+01  6.48259E-13
    7  1.70140E+01  6.48259E-13
 Successful solution
 
 ---------------------------------------------------
 Solver         :  IPOPT (v3.12)
 Solution time  :    2.0000000018626451E-002 sec
 Objective      :    17.014017289155863     
 Successful solution
 ---------------------------------------------------
 


Results
x1: [1.0]
x2: [4.742999637]
x3: [3.8211499845]
x4: [1.3794082931]

3. 深度學習模塊

brain模塊用於人工神經網絡(ANN)的優化求解。gekko優化的原理與常見的梯度下降法不太一樣，使用於中等規模的模型參數優化。這裏直接簡單給一個例子，藍色的是數據點，紅色曲線是神經網絡的預測結果。

from gekko import brain
import numpy as np
import matplotlib.pyplot as plt
b = brain.Brain(remote=False)
b.input_layer(1)
b.layer(linear=2)
b.layer(tanh=2)
b.layer(linear=2)
b.output_layer(1)
x = np.linspace(0,2*np.pi)
y = np.sin(x)
b.learn(x,y)
xp = np.linspace(-2*np.pi,4*np.pi,100)
yp = b.think(xp)
plt.figure()
plt.plot(x,y,'bo')
plt.plot(xp,yp[0],'r-')
plt.show()

結果如下：

 APMonitor, Version 0.9.2
 APMonitor Optimization Suite
 ----------------------------------------------------------------
 
 
 --------- APM Model Size ------------
 Each time step contains
   Objects      :            0
   Constants    :            0
   Variables    :           21
   Intermediates:           14
   Connections  :            0
   Equations    :           15
   Residuals    :            1
 
 Number of state variables:             69
 Number of total equations: -           50
 Number of slack variables: -            0
 ---------------------------------------
 Degrees of freedom       :             19
 
 solver            3  not supported
 using default solver: APOPT
 ----------------------------------------------
 Model Parameter Estimation with APOPT Solver
 ----------------------------------------------
 
 Iter    Objective  Convergence
    0  2.32734E+02  9.62187E-01
    1  1.93339E+02  5.00197E-01
    2  1.11518E+02  1.09380E+00
    3  1.11104E+03  9.86073E-01
    4  3.77756E+02  1.36852E-01
    5  8.80346E+02  1.98341E-01
    6  4.10029E+02  1.27699E-01
    7  3.47998E+02  7.34888E-03
    8  1.49302E+02  4.53653E-02
    9  1.93712E+02  8.73271E-02
 
 Iter    Objective  Convergence
   10  5.74918E+01  3.12551E-01
   11  4.84043E+01  5.68927E-02
   12  4.39362E+01  3.54965E-02
   13  4.32555E+01  1.55739E-02
   14  4.30514E+01  1.66817E-03
   15  4.24926E+01  5.04354E-04
   16  4.19261E+01  3.60617E-02
   17  4.17156E+01  1.53822E-03
   18  4.18934E+01  2.22466E-02
   19  4.17898E+01  2.25828E-02
 
 Iter    Objective  Convergence
   20  4.17632E+01  2.25767E-03
   21  4.17563E+01  1.06715E-03
   22  4.17559E+01  7.26557E-05
   23  4.17556E+01  8.69901E-04
   24  4.17556E+01  5.07237E-04
   25  4.17556E+01  3.89240E-05
   26  4.17556E+01  3.89240E-05
 Successful solution
 
 ---------------------------------------------------
 Solver         :  IPOPT (v3.12)
 Solution time  :    7.6999999815598130E-002 sec
 Objective      :    41.755580638466895     
 Successful solution
 ---------------------------------------------------