Pytorch基礎：神經網絡和優化器
torch.nn是爲神經網絡設計的模塊化接口。nn構建與autograd上，可用來定義和運行神經網絡
nn.functional是神經網絡中使用的一些常用的函數，（不具有可學習參數，如ReLU、pool、DropOut等）
# 導入相關包
import torch
import torch.nn as nn # 一般設置別名爲nn
import torch.nn.functional as F # 一般設置別名爲F
定義一個網絡
Pytorch中已準備好的了現有的網絡模型，只要繼承nn.Module類，並實現forward方法。Pytorch會根據autograd，自動實現backward函數，在forward函數中可使用任何tensor支持的操作及Python語法
class Net(nn.Module):
def __init__(self):
# nn.Module字類函數必須在構建函數中執行父類的構造函數
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 6, 3) # 卷積層， 1爲單通道， 6爲輸出通道， 3爲卷積核3x3
self.fc1 = nn.Linear(1350, 10)
# 正向傳播
def forward(self, x):
print(x.size()) # 結果：[1, 1, 32, 32]
x = self.conv1(x) #根據卷積的尺寸計算公式，計算結果爲30
x = F.relu(x)
print(x.size()) # 結果：[1, 6, 30, 30]
x = F.max_pool2d(x, (2, 2)) # 池化層， 計算結果爲15
x = F.relu(x)
print(x.size()) # 結果：[1, 6, 15, 15]
x = x.view(x.size()[0], -1)
# -1表示自適應，該操作是把[1, 6, 15, 15]壓扁，變爲[-1， 1350]
print(x.size())
x = self.fc1(x)
return x
net = Net()
print(net)
Net(
(conv1): Conv2d(1, 6, kernel_size=(3, 3), stride=(1, 1))
(fc1): Linear(in_features=1350, out_features=10, bias=True)
)
網絡的科學系參數通過.parameters()返回
for param in net.parameters():
print(param)
Parameter containing:
tensor([[[[-0.1501, 0.0207, -0.2991],
[ 0.1171, 0.0988, 0.0631],
[ 0.2022, -0.1330, -0.2333]]],
[[[ 0.2957, -0.2145, -0.2514],
[ 0.1999, -0.0470, -0.0605],
[ 0.2975, 0.1932, 0.0635]]],
[[[ 0.1194, -0.2086, -0.1382],
[ 0.0685, 0.1700, -0.1252],
[-0.3048, -0.0106, 0.1005]]],
[[[ 0.3157, 0.3140, -0.1614],
[ 0.1859, -0.2659, -0.1587],
[-0.2780, -0.2142, -0.0624]]],
[[[ 0.2214, 0.1233, 0.1699],
[-0.2489, -0.1493, -0.3306],
[ 0.2730, 0.1064, -0.0716]]],
[[[ 0.3102, 0.2241, -0.2976],
[ 0.0525, -0.0518, 0.1736],
[ 0.2654, 0.3064, 0.3140]]]], requires_grad=True)
Parameter containing:
tensor([-0.2208, -0.1180, -0.1639, -0.0986, 0.1076, 0.0020],
requires_grad=True)
Parameter containing:
tensor([[ 0.0004, 0.0112, 0.0163, ..., -0.0033, -0.0175, 0.0021],
[-0.0188, 0.0177, -0.0196, ..., -0.0163, -0.0052, -0.0001],
[-0.0009, -0.0209, 0.0002, ..., 0.0217, -0.0135, 0.0113],
...,
[-0.0246, -0.0269, 0.0255, ..., 0.0067, -0.0116, -0.0021],
[ 0.0222, 0.0139, 0.0108, ..., -0.0138, 0.0266, 0.0183],
[ 0.0195, -0.0110, -0.0210, ..., 0.0056, -0.0081, 0.0261]],
requires_grad=True)
Parameter containing:
tensor([ 0.0119, -0.0075, 0.0034, -0.0180, -0.0205, -0.0038, 0.0109, -0.0236,
0.0165, 0.0253], requires_grad=True)
# net.named_parameters可同時返回參數及名稱
for name, param in net.named_parameters():
print('name: {}, parameters: {}'.format(name, param.size()))
name: conv1.weight, parameters: torch.Size([6, 1, 3, 3])
name: conv1.bias, parameters: torch.Size([6])
name: fc1.weight, parameters: torch.Size([10, 1350])
name: fc1.bias, parameters: torch.Size([10])
forward函數輸入和輸出都是Tensor
inputs = torch.randn(1, 1, 32, 32)
outputs = net(inputs)
outputs.size()
torch.Size([1, 1, 32, 32])
torch.Size([1, 6, 30, 30])
torch.Size([1, 6, 15, 15])
torch.Size([1, 1350])
torch.Size([1, 10])
inputs.size()
torch.Size([1, 1, 32, 32])
反向傳播前，首先要將所有的梯度清零
反向傳播是Pytorch自動實現的，只需調用.backward函數即可
net.zero_grad()
outputs.backward(torch.ones(1, 10))
torch.nn只支持batch，不支持一次只輸入一個樣本。
# nn中預設了常用的損失函數
y = torch.arange(0, 10).view(1, 10).float()
criterion = nn.MSELoss()
loss = criterion(outputs, y)
loss.item()
26.876943588256836
優化器
反向傳播計算完所有梯度後，還需要使用優化方法來更新網絡的權重和參數。例如隨機梯度下降
weight = weight - learning_rate * gradient
torch.optim中實現了大多數優化方法
import torch.optim as optim
out = net(inputs)
criterion = nn.MSELoss()
loss = criterion(out, y)
optimizer = optim.SGD(net.parameters(), lr=0.01) # 新建優化器，SGD只需調整參數和學習率
optimizer.zero_grad() # 梯度清零
loss.backward()
optimizer.step() # 更新參數
torch.Size([1, 1, 32, 32])
torch.Size([1, 6, 30, 30])
torch.Size([1, 6, 15, 15])
torch.Size([1, 1350])