A list of the training algorithms that are available in the Neural Network Toolbox software and that use gradient- or Jacobian-based methods, is shown in the following table.
Function |
Algorithm |
---|---|
Levenberg-Marquardt |
|
Bayesian Regularization |
|
BFGS Quasi-Newton |
|
Resilient Backpropagation |
|
Scaled Conjugate Gradient |
|
Conjugate Gradient with Powell/Beale Restarts |
|
Fletcher-Powell Conjugate Gradient |
|
Polak-Ribiére Conjugate Gradient |
|
One Step Secant |
|
Variable Learning Rate Gradient Descent |
|
Gradient Descent with Momentum |
|
Gradient Descent |
Note that:
During training, the progress is constantly updated in the training window. Of most interest are the performance, the magnitude of the gradient of performance and the number of validation checks. The magnitude of the gradient and the number of validation checks are used to terminate the training. The gradient will become very small as the training reaches a minimum of the performance.
Ifthe
magnitude of the gradient is less than 1e-5, the training will stop. This limit can be adjusted by setting the parameter net.trainParam.min_grad
.
Thenumber
of validation checks represents the number of successive iterations that the validation performance fails to decrease. If this number reaches 6 (the default value), the training will stop. In this run, you can see that the training did stop because
of the number of validation checks. You can change this criterion by setting the parameternet.trainParam.max_fail
.
(Note that your results may be different than those shown in the following figure, because of the random setting of the initial weights and biases.)
There are other criteria that can be used to stop network training. They are listed in the following table.
Parameter |
Stopping Criteria |
---|---|
min_grad |
Minimum Gradient Magnitude |
max_fail |
Maximum Number of Validation Increases |
time |
Maximum Training Time |
goal |
Minimum Performance Value |
epochs |
Maximum Number of Training Epochs (Iterations) |
The performance plot shows the value of the performance function versus the iteration number. It plots training, validation, and test performances. The training state plot shows the progress of other training variables, such as the gradient magnitude, the number of validation checks, etc.
The error histogram plot shows the distribution of the network errors.
The regression plot shows a regression between network outputs and network targets.
You
can use the histogram and regression plots to validate network performance
【1】http://cn.mathworks.com/help/nnet/ug/train-and-apply-multilayer-neural-networks.html