optimizer = SGD + Learning Rate scheduler
機器之心：Adagrad & 優化器發展歷程
Paper: An overview of gradient descent optimization algorithms
簡書：Adam
知乎：文本分類問題常見的性能&效果Tricks
引用:
1.在文本分類任務中，有哪些論文中很少提及卻對性能有重要影響的tricks？https://www.zhihu.com/question/265357659
2.正則項相關：
https://www.zhihu.com/question/30231749
L1、L2正則化 https://zhuanlan.zhihu.com/p/35893078
知乎L2 https://www.zhihu.com/question/30231749
L0,L1,L2範數定義 https://zhuanlan.zhihu.com/p/28023308
3.fast ai 注意力和transformer課程
https://www.fast.ai/2019/07/08/fastai-nlp/
4.Adam家族
https://blog.csdn.net/u012535605/article/details/83579214
admw: https://zhuanlan.zhihu.com/p/40814046
admw: https://zhuanlan.zhihu.com/p/63982470
admw & admsgrad: https://www.fast.ai/2018/07/02/adam-weight-decay/
layer-wise adamw batch https://medium.com/nvidia-ai/a-guide-to-optimizer-implementation-for-bert-at-scale-8338cc7f45fd
bert admw源碼https://github.com/google-research/bert/blob/eedf5716ce1268e56f0a50264a88cafad334ac61/optimization.py#L87