試圖從code snippets 和 pytorch 源代碼 去理解深度學習概念與技巧
返回 總目錄文章
視頻筆記是按時間循序更新的，越往下越新
大部分視頻爭取控制在5-8分鐘以內，極少數時間在10分鐘以上。
pytorch 官方tutorials
tutorials 01-03
- nn.Module, nn.Functional, forward, num_flat_features, inherit, overwrite
- net.parameters, loss.grad_fn.next_functions[0][0], net.zero_grad
- criterion = nn.MSELoss(), loss = criterion(output, target)
- optimizer = optim.SGD(net.parameters(), lr=0.01), optimizer.zero_grad, optimizer.step
net.zero_grad() == 效用== optimizer.zero_grad()
- net.parameters() == 效用== optimizer.param_groups[0]['params']
net 與optimizer調取parameters方式不同
- net.parameters() 生成 generator; 用for in loop調取所有參數
- optimizer.param_groups 生成list, 然後optimizer.param_groups[0]是dict, 然後optimizer.param_groups[0]['params']調取所有參數
- lst = list(net.parameters()) 將generator轉化爲list, 但必須賦值
- p.data.add_(-group['lr'], d_p)
如何調用net.conv1內部的method, attributes
- net.conv1.weight, net.conv2.bias.grad, net.fc1.zero_grad
- optimizer.param_groups[0].keys()
pytorch如何借用THNN計算MSELoss, 覈實是否是THNN在工作
- nn.MSELoss, nn._Loss, nn.module.Module, THNN,
- pytorch/ torch/lib/ THNN/generic/MSECriterion.c, THNN_(MSECriterion_updateOutput)
- `super(Net, self).__init__()`, 在運行super class init中，同時繼承了所有的methods
- Net overwrite `init(), forward()` write a new func `num_flat_features()` for itself
- self._modules.keys(), self.__dict__.keys()
- lst = list(self._buffers.keys()), sorted(keys)
- nn.Conv2d -> nn._ConvND -> nn.Module
- nn._ConvND: init, reset_parameters
- ConvNd = torch._C._functions.ConvNd
self.conv1(x) 先運行__getattr__再運行__call__
如何安裝gdb從而能一路debug from python to C/C++
- 安裝求助
- C/C++ gdb 問題基本解決 部分解決
- gdb python 有待解決 no module libpython 似乎解決了
- 剩下can't read symbols 的warning 沒有真正解決
- MSELoss -> _Loss -> Module
- 包含init, forward, pre_forward_hooks, forward_hooks, backward_hooks
- _functions.thnn.MSELoss.apply(input, target, size_average) 調用torch._C 中的用於計算mseloss函數
- ctx == _ContextMethodMixin, 至於是如何調用的，不清楚過程
- 嘗試理解這種方法的廣泛性
- __init__: 將params, defaults(包含超參數dict)重新打包到self.param_groups裏面
- 方便zero_grad和step 使用
全流程梳理pytorch普通建模 代碼
- part1 part2 part3 part4 part5 part6 part7 part8 part9
- backend1: 從pytorch ConvNd 到Torch.csrc.autograd.functions....ConvForward
- 從pytorch.relu通過backend到torch.Threshold
- 從pytorch.maxpool2d_通過backend_到torch.C.spatialDilatedMaxPooling
- 從pytorch.MSELoss_通過backend_到Torch.mseloss
全流程梳理pytorch 多分類建模 代碼
二元分類問題的Loss設定的注意事項：代碼3
- 如果用BCEWithLogitsLoss
- features, targets的type 要統一爲torch.FloatTensor
- targets的size要規範爲（-1，1）
- 如果用CrossEntryLoss
- targets的type一定要是torch.LongTensor
- 摸索過程：真實發現錯誤和尋找解決方案的過程
- part1, part2, part3
探索keras內部 冗長解讀
- 查看keras內部主要的modules 0:00-7:50
- keras.models.Sequential內部結構 --13:38
- keras.legacy.interfaces...wrapper 讓keras1與keras2互通 -- 15:36
- keras.models.add --22.10
爲什麼pytorch對beginner更友好 解讀
- 更容易一層一層debug, 這個視頻證明用debug方式閱讀keras代碼很難
用pytorch構建自己的數據class 代碼文檔 part1, part2, part3, part4, 總結版
- 存儲自己的數據，transform，batch，shuffle
y.backward()
y.backward(torch.FloatTensor(x.size())
net.conv2.register_forward_hook(printnorm)
net.conv2.register_backward_hook(printgradnorm)
conv2_param_list = list(self.parameters()) # self: conv2
conv2_param_list.__len__() # 2
conv2_param_list[0].size()
transfer_learning_tutorial
data_transforms = {
'train': transforms.Compose([
transforms.RandomSizedCrop(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
]),
'val': transforms.Compose([
transforms.Scale(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
]),
}
ImageFolder如何將圖片folder轉化成模型數據格式
data_dir = '/Users/Natsume/Desktop/data/hymenoptera_data'
image_datasets = {x: datasets.ImageFolder(os.path.join(data_dir, x),
data_transforms[x])
for x in ['train', 'val']}
ax1 = plt.subplot2grid((2, 1), (0, 0), colspan=1, rowspan=1)
ax1.set_title("original close price with mv_avg_volume window %d" %vol_window)
# plot predictions(pct) as color into prices
for start, stop, col in zip(xy[:-1], xy[1:], color_data):
x, y = zip(start, stop)
ax1.plot(x, y, color=uniqueish_color3(col))
dataloders = {x: torch.utils.data.DataLoader(
image_datasets[x], batch_size=4, shuffle=True, num_workers=4)
for x in ['train', 'val']}
def imshow(inp, title=None):
"""Imshow for Tensor."""
inp = inp.numpy().transpose((1, 2, 0))
mean = np.array([0.485, 0.456, 0.406])
std = np.array([0.229, 0.224, 0.225])
inp = std * inp + mean
plt.imshow(inp)
if title is not None:
plt.title(title)
plt.pause(0.001) # pause a bit so that plots are updated
inputs, classes = next(iter(dataloders['train']))
out = torchvision.utils.make_grid(inputs)
imshow(out, title=[class_names[x] for x in classes])
model_ft = models.resnet18(pretrained=True)
model_ft = models.resnet18(pretrained=True)
model_ft.fc = nn.Linear(num_ftrs, 2)
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_ft, step_size=7, gamma=0.1)
for epoch in range(num_epochs):
for phase in ['train', 'val']:
for data in dataloders[phase]:
# -0.26s val
def step(self, epoch=None):
if epoch is None:
epoch = self.last_epoch + 1
self.last_epoch = epoch
for param_group, lr in zip(self.optimizer.param_groups, self.get_lr()):
param_group['lr'] = lr
def get_lr(self):
return [base_lr * self.gamma ** (self.last_epoch // self.step_size)
for base_lr in self.base_lrs]
def train(self, mode=True):
"""Sets the module in training mode.
This has any effect only on modules such as Dropout or BatchNorm.
"""
self.training = mode
for module in self.children():
module.train(mode)
return self
# 阻止計算參數的gradients
param.requires_grad = False
for j in range(inputs.size()[0]):
images_so_far += 1
ax = plt.subplot(num_images//2, 2, images_so_far)
ax.axis('off')
ax.set_title('predicted: {}'.format(class_names[preds[j]]))
imshow(inputs.cpu().data[j])
from torch.utils.data import TensorDataset, DataLoader
train_dataset = TensorDataset(train_features.data, train_targets.data)
train_loader = DataLoader(train_dataset, batch_size=64,
shuffle=True, num_workers=1)
YunJey | pytorch-tutorial
讓pytorch使用tensorboard
1. torchvision.datasets.MNIST()
1. iter(data_loader): 構建iterator
2. tensor.view == np.reshape
3. argmax.squeeze() 去除（n, m, 1)中的1
4. tensor.float(): 改變type
5. logger:
plot curves: loss, acc are scalar;
plot histogram: params, grads, np.array;
plot images: from tensor to (m, h, w)
AI-challenger stock