Tensor——基本運算

broadcasting

expand、without copying data
key idea

• insert 1 dim ahead
• expand dims with size 1 to same size
• feature maps:[4,32,14,14]
• bias:[32,1,1] =>[1,32,1,1] => [4,32,14,14]

加減乘除

add(+)

>>> import torch
>>> a = torch.rand(3,4)
>>> b = torch.rand(4)
>>> a+b,torch.add(a,b)
(tensor([[1.1636, 1.1454, 1.3575, 1.1242],
        [1.6967, 1.2132, 0.9738, 1.2507],
        [0.9603, 1.6887, 1.3814, 0.7453]]), 
tensor([[1.1636, 1.1454, 1.3575, 1.1242],
        [1.6967, 1.2132, 0.9738, 1.2507],
        [0.9603, 1.6887, 1.3814, 0.7453]]))

可見使用“+”和使用add函數結果是相同的。也可以驗證一下：

>>> import torch
>>> a = torch.rand(3,4)
>>> b = torch.rand(4)
>>> torch.all(torch.eq(a+b,torch.add(a,b)))
tensor(True)

sub(-)
mul(*)
div(/)

矩陣運算

torch.mm( )

只能用於二維矩陣的運算。

>>> import torch
>>> a = torch.full([2,2],3)
>>> b = torch.ones(2,2)
>>> torch.mm(a,b)
tensor([[6., 6.],
        [6., 6.]])

torch.matmul( )/@

可以用於二維矩陣計算，也可以是多維。

二維：

>>> import torch
>>> a = torch.full([2,2],3)
>>> b = torch.ones(2,2)
>>> torch.matmul(a,b)
tensor([[6., 6.],
        [6., 6.]])

>>> import torch
>>> a = torch.full([2,2],3)
>>> b = torch.ones(2,2)
>>> a@b
tensor([[6., 6.],
        [6., 6.]])

降維度

>>> import torch
>>> a = torch.rand(4,784)
>>> x = torch.rand(4,784)
>>> w = torch.rand(512,784)
>>> ([email protected]()).shape
torch.Size([4, 512])

>二維

比如四維，計算的時候就是前兩維不變，後兩維進行計算。

 >>> import torch
>>> a = torch.rand(4,3,28,64)
>>> b = torch.rand(4,3,64,32)
>>> torch.matmul(a,b).shape
torch.Size([4, 3, 28, 32])

broadcast
通道數目不同時，符合broadcast情況：

>>> import torch
>>> a = torch.rand(4,3,28,64)
>>> b = torch.rand(4,1,64,32)
>>> torch.matmul(a,b).shape
torch.Size([4, 3, 28, 32])

維度不同、通道數目也不等時。不符合broadcast時會報錯：

>>> import torch
>>> a = torch.rand(4,3,28,64)
>>> b = torch.rand(4,64,32)
>>> torch.matmul(a,b).shape
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
RuntimeError: The size of tensor a (3) must match the size of tensor b (4) at non-singleton dimension 1

pow

兩種方法：
①任意次冪

>>> import torch
>>> a = torch.full([2,2],6)
>>> a.pow(3)
tensor([[216., 216.],
        [216., 216.]])

②僅限平方

>>> import torch
>>> a = torch.full([2,2],6)
>>> a**2
tensor([[36., 36.],
        [36., 36.]])

sqrt/sqrt

平方根：

>>> import torch
>>> a = torch.full([2,2],1024)
>>> a.sqrt(),a.rsqrt, a**0.5
(tensor([[32., 32.],
        [32., 32.]]),
  tensor([[32., 32.],
        [32., 32.]]))
>>> 

平方根的倒數：

>>> import torch
>>> a = torch.full([2,2],1024)
>>> a.rsqrt()
( tensor([[0.0312, 0.0312],
        [0.0312, 0.0312]]))

exp/log

我變懶了因爲我困了！！三種：exp、log、log2寫在一起吧：

>>> import torch
>>> a = torch.exp(torch.ones(2,2))
>>> a,torch.log(a), torch.log2(a)
(tensor([[2.7183, 2.7183],
        [2.7183, 2.7183]]), 
 tensor([[1., 1.],
        [1., 1.]]), 
 tensor([[1.4427, 1.4427],
        [1.4427, 1.4427]]))

近似值

• .floor()——往下近似
• .ceil()——往上近似
• .trunc()——裁剪爲整數部分
• .frac()——裁剪成小數部分

>>> import torch
>>> a = torch.tensor(3.1415926)
>>> a.floor(), a.ceil(), a.trunc(), a.frac()
(tensor(3.), tensor(4.), tensor(3.), tensor(0.1416))

• torch.round()——四捨五入

>>> import torch
>>> a = torch.tensor(3.499)
>>> b = torch.tensor(4.501)
>>> a.round(), b.round()
(tensor(3.), tensor(5.))

梯度裁剪

最大、最小值：

>>> import torch
>>> grad = torch.rand(2,3)*15
>>> grad.max(), grad.min()
(tensor(12.5018), tensor(4.8845))

.clamp( )

>>> import torch
>>> grad = torch.rand(2,3)*15
>>> grad, grad.clamp(10),grad.clamp(5,10)
(tensor([[11.7561,  0.7950,  7.1729],
        [11.2285,  5.1940, 14.5741]]), 
tensor([[11.7561, 10.0000, 10.0000],
        [11.2285, 10.0000, 14.5741]]),
tensor([[10.0000,  5.0000,  7.1729],
        [10.0000,  5.1940, 10.0000]]))

沒睡午覺真的太困了，最後這點代碼就放在一起打印吧。