深度學習框架PyTorch：入門與實踐 學習（二）

Tensor和Autograd------Tensor

Tensor

創建：

1. Tensor(*size),創建時不會立馬分配空間，使用時纔會分配，而其他均創建時立馬就分配。
2. ones(*sizes)
3. zeros(*sizes)
4. eye(*sizes)對角線爲1其他爲0
5. arange(s,e,step)從s到e步長爲step
6. linspace(s,e,steps)從s到e，均勻切分成steps份
7. rand/randn(*sizes)均勻/標準整體分佈
8. normal(mean,std)/uniform(from,to)
9. randperm(m)隨機排列

利用List創建tensor 

a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
print(a)

b = a.tolist()
print(b)

print(a.size())

print(a.numel())#輸出the num of element

# 創建一個和a一樣大小的Tensor
c = torch.Tensor(a.size())
print(c.size())
# 查看c的形狀
print(c.shape)

常用tensor操作：

1. Tensor.view()可以調整tensor的形狀，但不會修改自身的數據，兩者共享內存

   a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
   print(a)
   b = a.view(3, 2)
   print(b)

   # -1自動計算該維度的大小
   c = a.view(-1, 6)
   print(c)

2. squeeze,unsqueeze減少維度增加維度

   a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
   b = a.view(3, 2)
   # 在1維增加一維
   b = b.unsqueeze(1)
   print(b.size())
   # 減去倒數第二維
   c = b.squeeze(-2)
   print(c.size())

   a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
   print(a.size())
   b = a.view(1, 1, 1, 2, 3)
   print(b.size())
   c = b.squeeze(0)
   print(c.size())
   # 把所有維度爲1的都刪去
   c = c.squeeze()
   print(c.size())

3. resize:調整size，可以修改tensor的尺寸，如果新尺寸大於原尺寸會分配新的內存空間，如果小於原尺寸，數據依舊會被保存。

   a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
   print(a)
   print(a.size())
   b = a.resize_(1, 3)
   print(b.size())
   print(b)
   b = a.resize_(3, 3)
   print(b.size())
   print(b)

4. 索引：

   a = torch.Tensor([[1, 2, 3], [4, 5, 6]])
   print(a)
   # 輸出每個元素是否滿足條件，滿足爲1否則爲0
   print(a > 1)
   # 輸出滿足條件的元素
   print(a[a>1])

5. gather（input, dim, index）:根據index在dim維選取數據，選取的大小跟index一樣。dim=0時，out[i][j]=input[index[i][j]][j],dim=1時，out[i][j]=input[i][index[i][j]]

   a = torch.arange(0, 16).view(4, 4)
   print(a)
   index = torch.LongTensor([[0, 1, 2, 3]])
   b = a.gather(0, index)
   print(b)
   index = torch.LongTensor([[3], [2], [1], [0]])
   c = a.gather(1, index)
   print(c)

6. scatter:gather的逆操作，

   c = torch.zeros((4, 4))
   c.scatter_(0, index, b)
   print(c.size())
   print(c)

7. 高級索引：

   x[[1,2],[1,2],[2,0]]# x[1,1,2],x[2,2,0]
   x[[2,1,0],[0],[1]] #x[2,0,1],x[1,0,1],x[0,0,1]
   

    

線性迴歸

import torch as t
from matplotlib import pyplot as plt
from IPython import display


t.manual_seed(100)

def get_fake_data(batch_size=8):
    x = t.rand(batch_size, 1) * 20
    y = x * 2 + (1 + t.randn(batch_size, 1)) * 3
    return x, y

x, y = get_fake_data()
plt.scatter(x.squeeze().numpy(), y.squeeze().numpy())


w = t.rand(1, 1)
b = t.zeros(1, 1)
lr = 0.001

for ii in range(20000):
    x, y = get_fake_data()
    y_pred = x.mul(w) + b.expand_as(y)
    loss = 0.5 * (y_pred - y) ** 2
    loss = loss.sum()
    dloss = 1
    dy_pred = dloss * (y_pred - y)
    dw = x * dy_pred
    db = dy_pred.sum()
    w.sub_((lr * dw).sum())
    b.sub_(lr * db)

    if ii % 1000 == 0:
        display.clear_output(wait=True)
        x = t.arange(0, 20).view(-1, 1)
        y = x.mul(w) + b.expand_as(x)
        plt.plot(x.numpy(), y.numpy())
        x2, y2 = get_fake_data(batch_size=20)
        plt.scatter(x2.numpy(), y2.numpy())
        plt.xlim(0, 20)
        plt.ylim(0, 41)
        plt.show()
        plt.pause(0.5)

Tensor 和Autograd------Autograd

Variable不支持部分Inplace 函數，因爲這些函數會修改tensor自身，但在反向傳播中，variable需要緩存原來的tensor計算梯度。

torch.autograd.grad(z, y)

輸出z對y的梯度

1. autograd根據用戶對variable的操作構建計算圖，對variable的操作抽象爲function
2. 由用戶創建的節點稱爲葉子節點，葉子節點的grad_fn爲None，葉子節點中需要求導的variable具有accumulateGrad，因爲其梯度是累加的。
3. variable默認requires_grad=false。當一個節點的rrquires_grad設置爲true時，其他依賴它的節點的requires_grad均爲true
4. volatile=True，將所有依賴它的節點全部設置爲vllatile=true，優先級比require_grad=True高，volatile的節點不會求導，也無法進行反向傳播。

用Variable實現線性迴歸

import torch as t
from torch.autograd import Variable as V
from matplotlib import pyplot as plt
from IPython import display

t.manual_seed(1000)


def get_fake_data(batch_size=16):
    x = t.rand(batch_size, 1) * 20
    y = x * 2 + (1 + t.randn(batch_size, 1)) * 3
    return x, y

x, y = get_fake_data()
plt.scatter(x.squeeze().numpy(), y.squeeze().numpy())
w = V(t.rand(1, 1), requires_grad=True)
b = V(t.zeros(1, 1), requires_grad=True)
lr = 0.0001
for ii in range(8000):
    x, y = get_fake_data()
    x = V(x)
    y = V(y)
    y_pred = x.mul(w) + b.expand_as(y)
    loss = 0.5 * (y_pred - y) ** 2
    loss = loss.sum()
    loss.backward()
    dloss = 1

    w.data = w.data - lr * w.grad.data
    b.data = b.data - lr * b.grad.data
    w.grad.data.zero_()
    b.grad.data.zero_()

    if ii % 1000 == 0:
        display.clear_output(wait=True)
        x = t.arange(0, 20).view(-1, 1)
        y = x.mul(w.data) + b.data.expand_as(x)
        plt.plot(x.numpy(), y.numpy())
        x2, y2 = get_fake_data(batch_size=20)
        plt.scatter(x2.numpy(), y2.numpy())
        plt.xlim(0, 20)
        plt.ylim(0, 41)
        plt.show()
        plt.pause(0.5)
print(w.data.squeeze()[0], b.data.squeeze()[0])