The GAN-based method is used to generate high-resolution remote sensing for data augmentation and image classification.
https://github.com/weihancug/GAN-based-HRRS-Sample-Generation-for-Image-Classification代碼位置
下載後一看是pytorch的，配置環境，gpu服務器一直被佔用，於是在cpu上運行：python2.7 tensorflow1.5+pytorch0.4.1
一、分割數據、處理數據：
dataset_split_preprocess.py,修改代碼路徑，並建立對應的目錄，放入數據集。執行ucm_preprocess.py
二、執行對抗生成：
wacgan-ucm.py執行時候 35行 N_GPUS = 1 原來爲2.
三、執行classifier-torch.py分類
注意路徑。
在cpu上運行 對抗生成，結果出現問題（對抗生成時候）：
InvalidArgumentError (see above for traceback): CPU BiasOp only supports NHWC.
[[Node: Generator.1.Shortcut/BiasAdd = BiasAdd[T=DT_FLOAT, data_format="NCHW", _device="/job:localhost/replica:0/task:0/cpu:0"](Generator.1.Shortcut/Conv2D, Generator.1.Shortcut/Generator.1.Shortcut.Biases/read)]]
說cpu下不支持nchw要改成nhwc，改完還不行。
於是到gpu執行，（多人使用環境），先查看pytorch，發現 2.7.1python有，3.6.7 conda虛擬環境下也有，於是先後執行，因代碼python2.7下寫的，3.6下執行時候print函數需要加括號。並且執行對抗生成時候 3.6版本有個地方出錯 （異常處理那裏）
所以用2.7，但是2.7對應的tensorflow是1.12版本，太高，提示問題，要降級到1.9以下。怕影響整個環境，所以沒有修改（不是conda環境的，怕出錯）。後來想到直接在自己的虛擬環境python2.7下再裝一下pytorch即可。於是下午開始配置，過程如下：
Installing with CUDA 9
conda install pytorch=0.4.1 torchvision cuda90 -c pytorch 安裝成功後繼續運行，終於成功了。ok
/home/gis/anaconda3/envs/Pytguo27/bin/python /home/gis/PycharmProjects/guo/GAN-based-HRRS-Sample-Generation-for-Image-Classification-master/classifier-torch.py
./UCM/train-128-20/train/
./UCM/train-128-20/test/
_VGG(
(features): Sequential(
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace)
(3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(4): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(5): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(6): ReLU(inplace)
(7): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(8): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(9): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(10): ReLU(inplace)
(11): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(12): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(13): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace)
(15): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(16): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace)
(18): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
(19): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(20): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(21): ReLU(inplace)
(22): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(23): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(24): ReLU(inplace)
(25): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
(classifier): Sequential(
(0): Dropout(p=0.5)
(1): Linear(in_features=8192, out_features=2048, bias=True)
(2): ReLU(inplace)
(3): Dropout(p=0.5)
(4): Linear(in_features=2048, out_features=2048, bias=True)
(5): ReLU(inplace)
(6): Linear(in_features=2048, out_features=21, bias=True)
)
)
learning rate is :0.010000
train model with train data
/home/gis/PycharmProjects/guo/GAN-based-HRRS-Sample-Generation-for-Image-Classification-master/classifier-torch.py:198: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
losses.update(loss.data[0], input.size(0))
/home/gis/PycharmProjects/guo/GAN-based-HRRS-Sample-Generation-for-Image-Classification-master/classifier-torch.py:199: UserWarning: invalid index of a 0-dim tensor. This will be an error in PyTorch 0.5. Use tensor.item() to convert a 0-dim tensor to a Python number
top1.update(prec1[0], input.size(0))
Epoch: [0][0/14] 2.474 (2.474) DatTimea 0.192 (0.192) Loss 3.0642 (3.0642) Prec@1 1.562 (1.562)
Epoch: [0][1/14] 0.116 (1.295) DatTimea 0.012 (0.102) Loss 3.0555 (3.0599) Prec@1 8.594 (5.078)
Epoch: [0][2/14] 0.106 (0.899) DatTimea 0.098 (0.101) Loss 3.0671 (3.0623) Prec@1 3.906 (4.688)
Epoch: [0][3/14] 0.146 (0.710) DatTimea 0.137 (0.110) Loss 3.0442 (3.0578) Prec@1 2.344 (4.102)
Epoch: [0][4/14] 0.146 (0.598) DatTimea 0.138 (0.116) Loss 3.0371 (3.0536) Prec@1 3.906 (4.062)
Epoch: [0][5/14] 0.144 (0.522) DatTimea 0.136 (0.119) Loss 3.0450 (3.0522) Prec@1 3.906 (4.036)
Epoch: [0][6/14] 0.145 (0.468) DatTimea 0.137 (0.122) Loss 3.0287 (3.0488) Prec@1 6.250 (4.353)
Epoch: [0][7/14] 0.145 (0.428) DatTimea 0.137 (0.124) Loss 3.0168 (3.0448) Prec@1 9.375 (4.980)
Epoch: [0][8/14] 0.145 (0.396) DatTimea 0.138 (0.125) Loss 3.0252 (3.0427) Prec@1 3.906 (4.861)
Epoch: [0][9/14] 0.145 (0.371) DatTimea 0.137 (0.126) Loss 3.0335 (3.0417) Prec@1 3.125 (4.688)
Epoch: [0][10/14] 0.145 (0.351) DatTimea 0.137 (0.127) Loss 3.0218 (3.0399) Prec@1 4.688 (4.688)
Epoch: [0][11/14] 0.146 (0.334) DatTimea 0.138 (0.128) Loss 3.0119 (3.0376) Prec@1 9.375 (5.078)
Epoch: [0][12/14] 0.144 (0.319) DatTimea 0.137 (0.129) Loss 3.0032 (3.0349) Prec@1 10.156 (5.469)
Epoch: [0][13/14] 0.824 (0.355) DatTimea 0.137 (0.129) Loss 3.0157 (3.0348) Prec@1 6.250 (5.476)
train model with generated data
Traceback (most recent call last):
File "/home/gis/PycharmProjects/guo/GAN-based-HRRS-Sample-Generation-for-Image-Classification-master/classifier-torch.py", line 300, in <module>
main()
File "/home/gis/PycharmProjects/guo/GAN-based-HRRS-Sample-Generation-for-Image-Classification-master/classifier-torch.py", line 131, in main
train(GtrainLoader, vgg, criterion, optimizer, epoch)
File "/home/gis/PycharmProjects/guo/GAN-based-HRRS-Sample-Generation-for-Image-Classification-master/classifier-torch.py", line 184, in train
output = model(input_var)
File "/home/gis/anaconda3/envs/Pytguo27/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/gis/PycharmProjects/guo/GAN-based-HRRS-Sample-Generation-for-Image-Classification-master/classifier-torch.py", line 75, in forward
output = self.features(input)
File "/home/gis/anaconda3/envs/Pytguo27/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/gis/anaconda3/envs/Pytguo27/lib/python2.7/site-packages/torch/nn/modules/container.py", line 91, in forward
input = module(input)
File "/home/gis/anaconda3/envs/Pytguo27/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
result = self.forward(*input, **kwargs)
File "/home/gis/anaconda3/envs/Pytguo27/lib/python2.7/site-packages/torch/nn/modules/conv.py", line 301, in forward
self.padding, self.dilation, self.groups)
RuntimeError: CUDA error: out of memory
這個錯誤怎麼辦？
pytorch模型提示超出內存cuda runtime error(2): out of memory
復現github代碼顯示內存錯誤，我想不對啊，人家訓練好的模型我復現怎麼會內存溢出呢? 查了一下，batch_size太大了電腦抗不住，改小一點就能繼續運行了 原來128 後來改爲 64？？甚至更小。