用Python从零开始构建ResNET

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"近年来，深度学习和计算机视觉领域取得了一系列突破。特别是行业引入了非常深的卷积神经网络后，在这些模型的帮助下，图像识别和图像分类等问题取得了非常好的成果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"因此这些年来，深度学习架构变得越来越深（层越来越多）以解决越来越复杂的任务，这也有助于提高分类和识别任务的性能，并让它们表现稳健。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但当我们继续向神经网络添加更多层时，模型训练起来也越来越困难，模型的准确度开始饱和，然后还会下降。于是ResNet诞生了，让我们摆脱了这种窘境，并能帮助解决这个问题。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"什么是ResNet？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"残差网络（ResNet）是著名的深度学习模型之一，由任少清、何开明、孙健和张翔宇在他们的论文中引入。这篇2015年的论文全名叫“Deep Residual Learning for Image Recognition”[1]。ResNet模型是迄今为止广泛流行和最成功的深度学习模型之一。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"残差块"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"随着这些残差（Residual）块的引入，训练非常深的网络时面临的问题得到了缓解，ResNet模型由这些块组成。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/d7\/a7\/d797d7d91690d15ffcd4d7ef857b71a7.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":10}}],"text":"来源：“图像识别的深度残差学习”论文"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"随着这些残差块的引入，训练非常深的网络时面临的问题得到了缓解，ResNet模型由这些块组成。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在上图中，我们可以注意到的第一件事是跳过模型的某些层的直接连接。这种连接称为“跳过连接”，是残差块的核心。由于存在这种跳过连接，输出是不相同的。如果没有跳过连接，输入‘X将乘以层的权重，然后添加一个偏置项。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然后是激活函数f()，我们得到输出为H(x)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"H(x)=f(wx+b)或H(x)=f(x)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"现在引入了新的跳过连接技术，输出H(x)更改为"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"H(x)=f(x)+x"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但是输入的维度可能与输出的维度不同，这可能发生在卷积层或池化层中。因此，这个问题可以用这两种方法来处理："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用跳过连接填充零以增加其维度。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1×1卷积层被添加到输入以匹配维度。在这种情况下，输出为："}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"H(x)=f(x)+w1.x"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这里添加了一个额外的参数w1，而在使用第一种方法时没有添加额外的参数。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet中的这些跳过连接技术通过梯度流经的替代快捷路径来解决深度CNN中梯度消失的问题。此外，如果有任何层损害了架构的性能，跳过连接也能起作用，它将被正则化跳过。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"ResNet的架构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"架构中有一个34层的普通网络，其灵感来自VGG-19，其中添加了快捷连接或跳过连接。这些跳过连接或残差块将架构转换为残差网络，如下图所示。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/d9\/a4\/d9752fbf94f63f1b708566d5f94517a4.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","marks":[{"type":"size","attrs":{"size":10}}],"text":"来源：“图像识别的深度残差学习”论文"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"将ResNet与Keras结合使用："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Keras是一个开源深度学习库，能够在TensorFlow上运行。Keras Applications提供以下ResNet版本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet50"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet50V2"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet101"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet101V2"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet152"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet152V2"}]}]}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"让我们从零开始构建ResNet："}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/98\/eb\/9895fc6a50db13cdf5279dbdacc7bfeb.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"来源：“图像识别的深度残差学习”论文"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们将上图作为参考，开始构建网络。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"ResNet架构多次使用CNN块，因此我们为CNN块创建一个类，它接受输入通道和输出通道。每个conv层之后都有一个batchnorm2d。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"plain"},"content":[{"type":"text","text":"import torch\nimport torch.nn as nn\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"plain"},"content":[{"type":"text","text":"class block(nn.Module):\ndef __init__(\nself, in_channels, intermediate_channels, identity_downsample=None, stride=1\n):\nsuper(block, self).__init__()\nself.expansion = 4\nself.conv1 = nn.Conv2d(\nin_channels, intermediate_channels, kernel_size=1, stride=1, padding=0, bias=False\n)\nself.bn1 = nn.BatchNorm2d(intermediate_channels)\nself.conv2 = nn.Conv2d(\nintermediate_channels,\nintermediate_channels,\nkernel_size=3,\nstride=stride,\npadding=1,\nbias=False\n)\nself.bn2 = nn.BatchNorm2d(intermediate_channels)\nself.conv3 = nn.Conv2d(\nintermediate_channels,\nintermediate_channels * self.expansion,\nkernel_size=1,\nstride=1,\npadding=0,\nbias=False\n)\nself.bn3 = nn.BatchNorm2d(intermediate_channels * self.expansion)\nself.relu = nn.ReLU()\nself.identity_downsample = identity_downsample\nself.stride = stride\ndef forward(self, x):\nidentity = x.clone()\nx = self.conv1(x)\nx = self.bn1(x)\nx = self.relu(x)\nx = self.conv2(x)\nx = self.bn2(x)\nx = self.relu(x)\nx = self.conv3(x)\nx = self.bn3(x)\nif self.identity_downsample is not None:\nidentity = self.identity_downsample(identity)\nx += identity\nx = self.relu(x)\nreturn x\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然后创建一个ResNet类，它接受许多块、层、图像通道和类数的输入。在下面的代码中，函数‘_make_layer’"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"创建ResNet层，它接受块的输入、残差块数、输出通道和步幅。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"plain"},"content":[{"type":"text","text":"class ResNet(nn.Module):\ndef __init__(self, block, layers, image_channels, num_classes):\nsuper(ResNet, self).__init__()\nself.in_channels = 64\nself.conv1 = nn.Conv2d(image_channels, 64, kernel_size=7, stride=2, padding=3, bias=False)\nself.bn1 = nn.BatchNorm2d(64)\nself.relu = nn.ReLU()\nself.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"plain"},"content":[{"type":"text","text":"# Essentially the entire ResNet architecture are in these 4 lines below\nself.layer1 = self._make_layer(\nblock, layers[0], intermediate_channels=64, stride=1\n)\nself.layer2 = self._make_layer(\nblock, layers[1], intermediate_channels=128, stride=2\n)\nself.layer3 = self._make_layer(\nblock, layers[2], intermediate_channels=256, stride=2\n)\nself.layer4 = self._make_layer(\nblock, layers[3], intermediate_channels=512, stride=2\n)\nself.avgpool = nn.AdaptiveAvgPool2d((1, 1))\nself.fc = nn.Linear(512 * 4, num_classes)\ndef forward(self, x):\nx = self.conv1(x)\nx = self.bn1(x)\nx = self.relu(x)\nx = self.maxpool(x)\nx = self.layer1(x)\nx = self.layer2(x)\nx = self.layer3(x)\nx = self.layer4(x)\nx = self.avgpool(x)\nx = x.reshape(x.shape[0], -1)\nx = self.fc(x)\nreturn x\ndef _make_layer(self, block, num_residual_blocks, intermediate_channels, stride):\nidentity_downsample = None\nlayers = []\n# Either if we half the input space for ex, 56x56 -> 28x28 (stride=2), or channels changes\n# we need to adapt the Identity (skip connection) so it will be able to be added\n# to the layer that's ahead\nif stride != 1 or self.in_channels != intermediate_channels * 4:\nidentity_downsample = nn.Sequential(\nnn.Conv2d(\nself.in_channels,\nintermediate_channels * 4,\nkernel_size=1,\nstride=stride,\nbias=False\n),\nnn.BatchNorm2d(intermediate_channels * 4),\n)\nlayers.append(\nblock(self.in_channels, intermediate_channels, identity_downsample, stride)\n)\n# The expansion size is always 4 for ResNet 50,101,152\nself.in_channels = intermediate_channels * 4\n# For example for first resnet layer: 256 will be mapped to 64 as intermediate layer,\n# then finally back to 256. Hence no identity downsample is needed, since stride = 1,\n# and also same amount of channels.\nfor i in range(num_residual_blocks - 1):\nlayers.append(block(self.in_channels, intermediate_channels))\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"返回nn.Sequential(*layers)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然后定义不同版本的ResNet"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于ResNet50，层序列为[3,4,6,3]。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于ResNet101，层序列为[3,4,23,3]。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于ResNet152，层序列为[3,8,36,3]。（请参阅“图像识别的深度残差学习”论文）"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"def ResNet50(img_channel=3, num_classes=1000):return ResNet(block, [3, 4, 6, 3], img_channel, num_classes)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"```plain\ndef ResNet101(img_channel=3, num_classes=1000):\nreturn ResNet(block, [3, 4, 23, 3], img_channel, num_classes)\ndef ResNet152(img_channel=3, num_classes=1000):\nreturn ResNet(block, [3, 8, 36, 3], img_channel, num_classes)\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然后编写一个小的测试来检查模型是否工作正常。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"plain"},"content":[{"type":"text","text":"def test():\nnet = ResNet101(img_channel=3, num_classes=1000)\ndevice = \"cuda\" if torch.cuda.is_available() else \"cpu\"\ny = net(torch.randn(4, 3, 224, 224)).to(device)\nprint(y.size())\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"plain"},"content":[{"type":"text","text":"test()\n"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"对于上面的测试用例，输出应该是："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/resource\/image\/58\/ed\/587c9d3f45430d79c49e1b13ec4799ed.png","alt":null,"title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"全部代码可以在这里访问："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/github.com\/BakingBrains\/Deep_Learning_models_implementation_from-scratch_using_pytorch_\/blob\/main\/ResNet_.py?fileGuid=45ZbZ1uTOtQ8SANF","title":"","type":null},"content":[{"type":"text","text":"https:\/\/github.com\/BakingBrains\/Deep_Learning_models_implementation_from-scratch_using_pytorch_\/blob\/main\/ResNet_.py"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"[1]：Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun: Deep Residual Learning for Image Recognition, Dec 2015, DOI:"},{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/abs\/1512.03385?fileGuid=45ZbZ1uTOtQ8SANF","title":"","type":null},"content":[{"type":"text","text":"https:\/\/arxiv.org\/abs\/1512.03385"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"原文链接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"https:\/\/www.analyticsvidhya.com\/blog\/2021\/06\/build-resnet-from-scratch-with-python\/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+AnalyticsVidhya+%28Analytics+Vidhya%29"}]}]}