Pytorch框架學習第七次作業——損失函數

1. Lossfunction依舊屬於網絡層的概念，即仍舊是Module的子類，爲了對lossfunction有一個更清晰的概念，需要大家採用步進(Step into)的調試方法從loss_functoin = nn.CrossEntropyLoss() 語句進入函數，觀察從nn.CrossEntropyLoss()到class Module(object)一共經歷了哪些類，記錄其中所有進入的類及函數。

第一步：CrossEntropyLoss類，super(CrossEntropyLoss, self).init
第二步：_WeightedLoss類，super(_WeightedLoss, self).init(size_average, reduce, reduction)
第三步: _Loss類，super(_Loss, self).init()
第四步：進入Module 類

2. 損失函數的reduction有三種模式，它們的作用分別是什麼？
   當inputs和target及weight分別如以下參數時，reduction=’mean’模式時，loss是如何計算得到的？
   inputs = torch.tensor([[1, 2], [1, 3], [1, 3]], dtype=torch.float)
   target = torch.tensor([0, 1, 1], dtype=torch.long)
   weights = torch.tensor([1, 2], dtype=torch.float)

• none：逐個元素計算
• sum：所有元素求和，返回標量
• mean：加權平均，返回標量

    inputs = torch.tensor([[1, 2], [1, 3], [1, 3]], dtype=torch.float)
    target = torch.tensor([0, 1, 1], dtype=torch.long)
    # def loss function
    weights = torch.tensor([1, 2], dtype=torch.float)

    loss_f_none_w = nn.CrossEntropyLoss(weight=weights, reduction='none')
    loss_f_sum = nn.CrossEntropyLoss(weight=weights, reduction='sum')
    loss_f_mean = nn.CrossEntropyLoss(weight=weights, reduction='mean')

    # forward
    loss_none_w = loss_f_none_w(inputs, target)
    loss_sum = loss_f_sum(inputs, target)
    loss_mean = loss_f_mean(inputs, target)

    # view
    print("\nweights: ", weights)
    print(loss_none_w, loss_sum, loss_mean)

將所有元素的loss相加除以所有元素的權重之和（1+2+2=5）