Concurrent Spatial and Channel Squeeze & Excitation in Fully Convolutional Networks
PDF: https://arxiv.org/pdf/1803.02579v2.pdf
PyTorch代码: https://github.com/shanglianlm0525/PyTorch-Networks
1 概述
本文对SE模块进行了改进,设计了三种 SE 变形结构cSE、sSE、scSE,在 MRI 脑分割 和 CT 器官分割任务上取得了可观的改进。
2 Spatial Squeeze and Channel Excitation Block (cSE)
即原始的SE Block , 详细见 Attention论文:Squeeze-and-Excitation Networks及其PyTorch实现
PyTorch代码:
class SE_Module(nn.Module):
def __init__(self, channel,ratio = 16):
super(SE_Module, self).__init__()
self.squeeze = nn.AdaptiveAvgPool2d(1)
self.excitation = nn.Sequential(
nn.Linear(in_features=channel, out_features=channel // ratio),
nn.ReLU(inplace=True),
nn.Linear(in_features=channel // ratio, out_features=channel),
nn.Sigmoid()
)
def forward(self, x):
b, c, _, _ = x.size()
y = self.squeeze(x).view(b, c)
z = self.excitation(y).view(b, c, 1, 1)
return x * z.expand_as(x)
3 Channel Squeeze and Spatial Excitation Block (sSE)
PyTorch代码:
class sSE_Module(nn.Module):
def __init__(self, channel):
super(sSE_Module, self).__init__()
self.spatial_excitation = nn.Sequential(
nn.Conv2d(in_channels=channel, out_channels=1, kernel_size=1,stride=1,padding=0),
nn.Sigmoid()
)
def forward(self, x):
z = self.spatial_excitation(x)
return x * z.expand_as(x)
4 Spatial and Channel Squeeze & Excitation Block (scSE)
PyTorch代码:
class scSE_Module(nn.Module):
def __init__(self, channel,ratio = 16):
super(scSE_Module, self).__init__()
self.cSE = cSE_Module(channel,ratio)
self.sSE = sSE_Module(channel)
def forward(self, x):
return self.cSE(x) + self.sSE(x)