nn.CrossEntroyLoss nn.NLLoss nn.LogSoft pytorch損失函數

nn.NNLoss(x,y_target) = -x[y_target]

x=[1,2,3] y_target=[2] 則loss = x[2]=3

nll = nn.NLLLoss()
x = torch.randn(3,3)
y_target = torch.tensor([1,2,0])
cal_my = (x[0][1]+x[1][2] +x[2][0])/3
loss = nll(x,y_target)
print(x)
print(y_target)
print(loss)
print(cal_my)

x爲:
tensor([[ 1.5827,  0.2479, -1.3481],
        [ 1.6454, -1.1276,  0.3725],
        [-0.9897, -1.4498,  2.3366]])

y_target爲:
tensor([1, 2, 0])

loss爲:
tensor(0.1231)

cal_my爲:
tensor(0.1231)

 

function = nn.LogSoftmax (dim=1)

function(x)代表着先讓x經過Softmax，然後再經過torch.log 

首先講一下Softmax的參數設置，在調用Softmax函數時候，需要指定按照哪個維度進行softmax，舉例如下：

x = torch.randn(3,3)
softmax_dim0 = nn.Softmax(dim=0)
softmax_dim1 = nn.Softmax(dim=1)

s_0 = softmax_dim0(x)
s_1 = softmax_dim1(x)
print(x)
print('s_0',s_0)
print('s_1',s_1)

輸出結果爲：
tensor([[ 0.3920,  0.6731,  1.0011],
        [-0.0462,  1.1450, -0.5482],
        [-0.2710,  1.1763, -1.1713]])
s_0 tensor([[0.4628, 0.2349, 0.7540],
        [0.2986, 0.3766, 0.1601],
        [0.2385, 0.3885, 0.0859]])
s_1 tensor([[0.2402, 0.3182, 0.4417],
        [0.2042, 0.6721, 0.1236],
        [0.1767, 0.7514, 0.0718]])

當設置dim=0時候，按照第0維進行softmax 0.4628 + 0.2986+0.2385=1

當設置dim=1時候，按照第1維進行softmax 0.2402+ 0.3182+ 0.4417=1

下面驗證nn.LogSoftmax(dim=1)爲softmax 和log的組合

 

softmax = nn.Softmax(dim=1)
logsoftmax = nn.LogSoftmax(dim=1)

x = torch.randn(3,3)
softmax_value = softmax(x)
cal_logsoftmax_value = torch.log(softmax_value)
logsoftmax_value = logsoftmax(x)

print('x',x)
print('softmax_value',softmax_value)
print('cal_logsoftmax_value',cal_logsoftmax_value)
print('logsoftmax_value',logsoftmax_value)

輸出結果爲：
x tensor([[-0.8330, -0.1150,  0.3054],
        [-0.2180, -1.1248,  0.4393],
        [-0.4476,  0.7618,  0.8175]])
softmax_value tensor([[0.1620, 0.3322, 0.5058],
        [0.3000, 0.1211, 0.5789],
        [0.1267, 0.4245, 0.4488]])
cal_logsoftmax_value tensor([[-1.8200, -1.1021, -0.6816],
        [-1.2040, -2.1108, -0.5467],
        [-2.0662, -0.8568, -0.8011]])
logsoftmax_value tensor([[-1.8200, -1.1021, -0.6816],
        [-1.2040, -2.1108, -0.5467],
        [-2.0662, -0.8568, -0.8011]])

 

function = nn.CrossEntroyLoss()

function(x)爲先讓x經過logsoftmax 再經過nlloss,驗證如下：

import torch
import torch.nn as nn

cel = nn.CrossEntropyLoss()
nll = nn.NLLLoss()
logsoftmax = nn.LogSoftmax(dim=1)

x = torch.randn(3,3)
y_target = torch.tensor([1,2,0])

logsoftmax_value = logsoftmax(x)
nll_value = nll(logsoftmax_value,y_target)
cel_value = cel(x,y_target)

print('x',x)
print('y_target',y_target)
print('logsoftmax_value',logsoftmax_value)
print('nll_value',nll_value)
print('cel_value',cel_value)


輸出結果如下：
x tensor([[ 1.0021,  0.4425, -0.3256],
        [ 0.3272,  0.5255, -0.2736],
        [ 0.5227, -0.6293,  1.4442]])
y_target tensor([1, 2, 0])
logsoftmax_value tensor([[-0.6079, -1.1675, -1.9355],
        [-1.0180, -0.8197, -1.6188],
        [-1.3426, -2.4947, -0.4211]])
nll_value tensor(1.3763)
cel_value tensor(1.3763)