MobileNet 訓練檢測網實驗總結

MobileNet 訓練檢測網實驗總結

MobileNet做目標檢測的包在tensorflow/models/object_detection裏。先下載一個ssd_mobilenet_v1的預訓練模型。

1.數據製作

• 準備VOC格式的數據集

• 使用 腳本 ../object_detection/cre

  ate_pascal_tf_record.py 把VOC數據轉化成 tf.record文件。

# DATA
python bread_pascal_record.py --data_dir=/home/wrz/zifu/VOCdevkit2007/VOC2007 --year=VOC2007 --output_path=/hyw/zifu.record --label_map_path=/hyw/zifu_label_map.pbtxt

• 這個腳本需要提供 標籤文件label_map.txt ,在object_detection/data/下有很多label_map.txt的例子
• 這個腳本對取xml文件，每張圖片生成tf.example的數據結構，如果報錯找不到圖片，可以在主函數中print data，看下data[dir]和data[filename]，在函數dict_to_tf_example中對應修改img_path
• 函數dict_to_tf_example中展示了對xml文件讀取和創建tf.example類型的過程，可以在這裏根據對label的需要做改動

2.訓練

• 網絡結構文件 ../object_detection/samples/configs/下有mobilenet_ssd的訓練用的網絡結構文件ssd_mobilenet_v1_pets.config，修改下面幾處：

  • num_class
  • fine_tune_checkpoint: “/home/wrz/MobileNet-master/models-master/object_detection/ssd_mobilenet_v1_coco_11_06_2017/model.ckpt
  • input_path: “/home/wrz/MobileNet-master/models-master/object_detection/hyw/zifu.record”
  • label_map_path: “/home/wrz/MobileNet-master/models-master/object_detection/hyw/zifu_label_map.pbtxt”
  • train_config: batch_size;learning_rate

  關於網絡結構文件,其結構如下

  --model:{}
  
  --train_config:{
    --optimizer: {}
    --data_augmentation_options:{}
    ...
     }
  
  --train_input_reader: {}
  --eval_config: {}
  --eval_input_reader: {}

• 類似caffe.proto對layers的參數定義，tensorflow在object_detect/protos/ 下有對config文件各個部分的參數定義。例如在optimizer.proto中可以找到learning_rate，在train_config.proto中有batch_size：

  syntax = "proto2";
  
  package object_detection.protos;
  
  import "object_detection/protos/optimizer.proto";
  import "object_detection/protos/preprocessor.proto";
  
  message TrainConfig {
    // Input queue batch size.
    optional uint32 batch_size = 1 [default=32];
  
    // Data augmentation options.
    repeated PreprocessingStep data_augmentation_options = 2;
  
    // Whether to synchronize replicas during training.
    optional bool sync_replicas = 3 [default=false];
  
    // How frequently to keep checkpoints.
    optional uint32 keep_checkpoint_every_n_hours = 4 [default=1000];
  
    // Optimizer used to train the DetectionModel.
    optional Optimizer optimizer = 5;
    ......

  train_config.proto中data_augmentation_options = 2，在preprocessor.proto中對應可以看到數據增強的各種模式選擇，默認值是2,即水平反轉增強：

  syntax = "proto2";
  
  package object_detection.protos;
  
  // Message for defining a preprocessing operation on input data.
  // See: //object_detection/core/preprocessor.py
  message PreprocessingStep {
    oneof preprocessing_step {
      NormalizeImage normalize_image = 1;
      RandomHorizontalFlip random_horizontal_flip = 2;
      RandomPixelValueScale random_pixel_value_scale = 3;
      RandomImageScale random_image_scale = 4;
      RandomRGBtoGray random_rgb_to_gray = 5;
      RandomAdjustBrightness random_adjust_brightness = 6;
      RandomAdjustContrast random_adjust_contrast = 7;
      RandomAdjustHue random_adjust_hue = 8;
      RandomAdjustSaturation random_adjust_saturation = 9;
      RandomDistortColor random_distort_color = 10;
      RandomJitterBoxes random_jitter_boxes = 11;
      RandomCropImage random_crop_image = 12;
      RandomPadImage random_pad_image = 13;
      RandomCropPadImage random_crop_pad_image = 14;
      RandomCropToAspectRatio random_crop_to_aspect_ratio = 15;
      RandomBlackPatches random_black_patches = 16;
      RandomResizeMethod random_resize_method = 17;
      ScaleBoxesToPixelCoordinates scale_boxes_to_pixel_coordinates = 18;
      ResizeImage resize_image = 19;
      SubtractChannelMean subtract_channel_mean = 20;
      SSDRandomCrop ssd_random_crop = 21;
      SSDRandomCropPad ssd_random_crop_pad = 22;
      SSDRandomCropFixedAspectRatio ssd_random_crop_fixed_aspect_ratio = 23;
    }
  }

  ​

  ​

• 訓練命令**：

  python train.py --logtostderr --train_dir=hyw/output/zifu --pipeline_config_path=hyw/ssd_mobilenet_v1_pets.config

• 模型保存：

  在trainer.py中，可以看到模型保存主要用到函數：

  saver = tf.train.Saver(keep_checkpoint_every_n_hours=keep_checkpoint_every_n_hours)

  在tesorflow官方文檔中有對Saver的具體描述：https://www.tensorflow.org/api_docs/python/tf/train/Saver

  defined in "tensorflow/python/training/saver.py"

  __init__(
    var_list=None,
    reshape=False,
    sharded=False,
    max_to_keep=5,
    keep_checkpoint_every_n_hours=10000.0,
    name=None,
    restore_sequentially=False,
    saver_def=None,
    builder=None,
    defer_build=False,
    allow_empty=False,
    write_version=tf.train.SaverDef.V2,
    pad_step_number=False,
    save_relative_paths=False,
    filename=None
  )

  Args:
  
    var_list: A list of Variable/SaveableObject, or a dictionary mapping names to SaveableObjects. If None, defaults to the list of all saveable objects.
    reshape: If True, allows restoring parameters from a checkpoint where the variables have a different shape.
    sharded: If True, shard the checkpoints, one per device.
    max_to_keep: Maximum number of recent checkpoints to keep. Defaults to 5.
    keep_checkpoint_every_n_hours: How often to keep checkpoints. Defaults to 10,000 hours.
    name: String. Optional name to use as a prefix when adding operations.
    restore_sequentially: A Bool, which if true, causes restore of different variables to happen sequentially within each device. This can lower memory usage when restoring very large models.
    saver_def: Optional SaverDef proto to use instead of running the builder. This is only useful for specialty code that wants to recreate a Saver object for a previously built Graph that had a Saver. The saver_def proto should be the one returned by the as_saver_def() call of the Saver that was created for that Graph.
    builder: Optional SaverBuilder to use if a saver_def was not provided. Defaults to BaseSaverBuilder().
    defer_build: If True, defer adding the save and restore ops to the build() call. In that case build() should be called before finalizing the graph or using the saver.
    allow_empty: If False (default) raise an error if there are no variables in the graph. Otherwise, construct the saver anyway and make it a no-op.
    write_version: controls what format to use when saving checkpoints. It also affects certain filepath matching logic. The V2 format is the recommended choice: it is much more optimized than V1 in terms of memory required and latency incurred during restore. Regardless of this flag, the Saver is able to restore from both V2 and V1 checkpoints.
    pad_step_number: if True, pads the global step number in the checkpoint filepaths to some fixed width (8 by default). This is turned off by default.
    save_relative_paths: If True, will write relative paths to the checkpoint state file. This is needed if the user wants to copy the checkpoint directory and reload from the copied directory.
    filename: If known at graph construction time, filename used for variable loading/saving.
  

​ 其中max_to_keep=5默認保存最近5次的模型，設置max_to_keep=0爲保存所有模型

• 訓練參數

  訓練時用到的函數是trainer.py中：

  slim = tf.contrib.slim
  
  slim.learning.train(
        train_tensor,
        logdir=train_dir,
        master=master,
        is_chief=is_chief,
        session_config=session_config,
        startup_delay_steps=train_config.startup_delay_steps,
        init_fn=init_fn,
        summary_op=summary_op,
        number_of_steps=(
            train_config.num_steps if train_config.num_steps else None),
        save_summaries_secs=120,
        sync_optimizer=sync_optimizer,
        saver=saver)     

• 其中 tensorflow/contrib/slim/python/slim/learning.py中，對於train的定義:

  def train(train_op,
          logdir,
          train_step_fn=train_step,
          train_step_kwargs=_USE_DEFAULT,
          log_every_n_steps=1,
          graph=None,
          master='',
          is_chief=True,
          global_step=None,
          number_of_steps=None,
          init_op=_USE_DEFAULT,
          init_feed_dict=None,
          local_init_op=_USE_DEFAULT,
          init_fn=None,
          ready_op=_USE_DEFAULT,
          summary_op=_USE_DEFAULT,
          save_summaries_secs=600,
          summary_writer=_USE_DEFAULT,
          startup_delay_steps=0,
          saver=None,
          save_interval_secs=600,
          sync_optimizer=None,
          session_config=None,
          trace_every_n_steps=None):

​ 其中number_of_steps定義最大迭代次數，save_summaries_secs=600默認每隔多少秒存儲一次

3.檢測

• 在 ../object_detection/目錄下，命令行輸入

  jupyter notebook

  打開../models/object_detection/object_detection_tutorial.ipynb

  運行即可

• 這個腳本運行使用的模型爲.pb文件，tensorflow訓練得到的是.meta 和 .ckpt文件，分別存儲了圖結構和參數值，需要先轉化。用腳本../object_detection/export_inference_graph.py：

  python export_inference_graph.py --input_type image_tensor --pipeline_config_path ./hyw/ssd_mobilenet_v1_pets.config --checkpoint_path ./hyw/output/zifu/model.ckpt-34940 --inference_graph_path ./hyw/output/model-34940.pb

• 最後說明，檢測耗時包括圖像轉化成np.array，讀取和裝載模型，檢測圖片。如果要保證檢測速度，則需要保存 當前使用的graph和sess。

• i5 6300HQ上測試，單張檢測圖片大概0.1秒，但是圖片中包含的目標貌似和檢測時間成倍數關係。賽揚j3160上測試，1秒一張圖，桌面版i7 自己編譯的tensorflow要比pip安裝的快，速度大概0.04秒多一張圖，這個和作者聲稱的時間一致。