在使用tensorflow構建神經網絡時，經常看到tf.nn，tf.layers，tf.contrib這三個模塊，它們有很多函數的功能是相同的，部分可以混用，這就導致使用者很容易被迷惑，本文在此做一下筆記以理清思路。
tf.nn：用於原始神經網絡（NN）操作的包裝程序，屬於基礎操作，常見包括卷積（conv、conv_transpose、dilation2D、separable_conv）、池化（avg_pool，max_pool）、歸一化（batch_normalization）、部分loss（ctc_loss）、激活函數（relu，crelu、swish）、分類操作（softmax）、Dropout、RNN（rnn_cell）等。
tf.layers：提供公共API構建網絡，可看作是比nn更高層的封裝，每個函數有更多可選的輸入參數設置，主要是卷積（Conv）、全連接（Dense）、展平（Flatten）、Dropout、池化（Pool）、空間約束（Layer、InputSpec）、歸一化（BatchNormalization）等。
tf.contrib: 各類模塊的實驗代碼，包括各類高級操作的API，在構建網絡的高級操作API中，包含tf.contrib.layers、tf.contrib.nn和tf.contrib.slim（好用的模塊），屬於實驗代碼，不穩定。但在TF2.0移除了contrib模塊。
個人：由於tensorflow已經發展到2.0以上，1.x移植到2.x還是存在不兼容，建議採用tf.keras, 採用tensorflow作爲後端。
以下爲各模塊對應的子模塊、成員與函數：
tf.nn：
Modules
rnn_cell module: 構建RNN.
Functions
all_candidate_sampler(...): 生成所有類的集合.
atrous_conv2d(...): 卷積（又稱帶孔卷積或膨脹卷積）.
atrous_conv2d_transpose(...): atrous_conv2d的轉置.
avg_pool(...): 平均池化.
avg_pool1d(...): 平均池化.
avg_pool2d(...): 平均池化.
avg_pool3d(...): 平均池化.
avg_pool_v2(...): 平均池化.
batch_norm_with_global_normalization(...): 批量歸一化.
batch_normalization(...): 批量歸一化.
bias_add(...): 添加偏置.
bidirectional_dynamic_rnn(...): 創建雙向遞歸神經網絡的動態版本. (deprecated)
collapse_repeated(...): 將重複的標籤合併爲單個標籤.
compute_accidental_hits(...): 計算與true_classes匹配的sampled_candidates中的位置ID.
compute_average_loss(...): 使用sample_weights縮放每個示例的損失並計算其平均值.
conv1d(...): 給定3D輸入和濾波器張量，計算1D卷積。. (deprecated argument values) (deprecated argument values)
conv1d_transpose(...): 轉置 conv1d.
conv2d(...): 給定4-D輸入和濾波器張量，計算2-D卷積。.
conv2d_backprop_filter(...): 計算相對於濾波器的卷積梯度.
conv2d_backprop_input(...): 計算相對於輸入的卷積梯度.
conv2d_transpose(...): 轉置 of conv2d.
conv3d(...): 給定5-D輸入和濾波器張量，計算3-D卷積.
conv3d_backprop_filter(...): 計算3D卷積相對於濾波器的梯度.
conv3d_backprop_filter_v2(...): 計算3D卷積相對於濾波器的梯度.
conv3d_transpose(...): 轉置 of conv3d.
conv_transpose(...): 轉置 of convolution.
convolution(...): 計算N-D卷積的和 (actually cross-correlation).
crelu(...): 計算級聯的ReLU.
ctc_beam_search_decoder(...): 對輸入中給出的對數執行波束搜索解碼.
ctc_beam_search_decoder_v2(...): 對輸入中給出的對數執行波束搜索解碼.
ctc_greedy_decoder(...): 對輸入中給出的對數執行貪婪解碼（最佳路徑）.
ctc_loss(...): 計算CTC（連接器時間分類）損失.
ctc_loss_v2(...): 計算CTC（連接器時間分類）損失.
ctc_unique_labels(...): 獲取tf.nn.ctc_loss的批處理標籤的唯一標籤和索引 .
depth_to_space(...): T型張量的DepthToSpace.
depthwise_conv2d(...): Depthwise 2-D convolution.
depthwise_conv2d_backprop_filter(...): 計算相對於濾波器的深度卷積的梯度.
depthwise_conv2d_backprop_input(...): 計算相對於輸入的深度卷積的梯度.
depthwise_conv2d_native(...): 給定4-D輸入和濾波器張量，計算二維深度卷積.
depthwise_conv2d_native_backprop_filter(...): 計算相對於濾波器的深度卷積的梯度.
depthwise_conv2d_native_backprop_input(...): 計算相對於輸入的深度卷積的梯度.
dilation2d(...): 計算4D輸入和3D濾波器張量的灰度膨脹.
dropout(...): Computes dropout. (deprecated arguments)
dynamic_rnn(...): 創建由RNNCell指定的遞歸神經網絡 cell. (deprecated)
elu(...):計算指數線性: exp(features) - 1 if < 0, features otherwise.
embedding_lookup(...): 在嵌入張量列表中查找id.
embedding_lookup_sparse(...): 計算給定id和權重的嵌入.
erosion2d(...): 計算4-D值和3-D內核張量的灰度腐蝕.
fixed_unigram_candidate_sampler(...): 使用提供的（固定的）基本分發對一組類進行採樣.
fractional_avg_pool(...): 對輸入執行分數平均池. (deprecated)
fractional_max_pool(...): 對輸入執行分數最大池化. (deprecated)
fused_batch_norm(...): Batch normalization.
in_top_k(...): 判斷目標是否在前K個預測中.
l2_loss(...): L2 Loss.
l2_normalize(...): 使用L2範數沿尺寸軸標準化. (deprecated arguments)
leaky_relu(...): 計算 Leaky ReLU 激活函數.
learned_unigram_candidate_sampler(...): 從訓練期間學習到的分佈中抽樣一組類別.
local_response_normalization(...): Local Response Normalization.
log_poisson_loss(...): 計算給定的對數泊松損失 log_input.
log_softmax(...): 計算log softmax activations. (deprecated arguments)
log_uniform_candidate_sampler(...): 使用對數均勻（Zipfian）基本分佈對一組類進行採樣.
lrn(...): Local Response Normalization.
max_pool(...): 最大池化.
max_pool1d(...): 最大池化.
max_pool2d(...): 最大池化.
max_pool3d(...): 最大池化.
max_pool_v2(...): 最大池化.
max_pool_with_argmax(...): 對輸入執行最大池化，並輸出最大值和索引.
moments(...): 計算x的均值和方差.
nce_loss(...): 計算並返回噪聲對比估計訓練損失.
normalize_moments(...): 根據足夠的統計量計算的均值和方差.
pool(...): 執行N-D池化操作.
quantized_avg_pool(...): 產生量化類型的輸入張量的平均池.
quantized_conv2d(...): 給定量化的4D輸入和濾波器張量，計算2D卷積.
quantized_max_pool(...): 爲量化類型產生輸入張量的最大池.
quantized_relu_x(...): 計算量化整流線性X: min(max(features, 0), max_value)
raw_rnn(...): 創建由RNNCell單元指定的RNN和循環函數loop_fn.
relu(...): 計算整流線性: max(features, 0).
relu6(...): 計算整流線性6: min(max(features, 0), 6).
relu_layer(...): 計算Relu(x * weight + biases).
safe_embedding_lookup_sparse(...): 查找嵌入結果，說明無效的ID和空白功能.
sampled_softmax_loss(...): 計算並返回採樣的softmax訓練損失。
scale_regularization_loss(...): 按副本數縮放給定正則化損失的總和.
selu(...): 計算比例指數線性: scale * alpha * (exp(features) - 1)
separable_conv2d(...): 帶可分離濾波器的二維卷積.
sigmoid(...): 計算x元素的S形.
sigmoid_cross_entropy_with_logits(...): 計算給定logits的S形交叉熵 .
softmax(...): 計算softmax 激活. (deprecated arguments)
softmax_cross_entropy_with_logits(...): 計算S形交叉熵 between logits and labels. (deprecated)
softmax_cross_entropy_with_logits_v2(...): 計算S形交叉熵 between logits and labels. (deprecated arguments)
softplus(...): 計算 softplus: log(exp(features) + 1).
softsign(...): 計算 softsign: features / (abs(features) + 1).
space_to_batch(...): T型4維張量的SpaceToBatch.
space_to_depth(...): T型張量的SpaceToDepth.
sparse_softmax_cross_entropy_with_logits(...): 計算稀疏softmax交叉熵 between logitsand labels.
static_bidirectional_rnn(...): 創建雙向遞歸神經網絡. (deprecated)
static_rnn(...): 創建一個指定的遞歸神經網絡 by RNNCell cell. (deprecated)
static_state_saving_rnn(...): RNN接受狀態保存程序以進行時間截斷的RNN計算. (deprecated)
sufficient_statistics(...): 計算x的均值和方差的足夠統計量.
swish(...): 計算 Swish activation function: x * sigmoid(x).
tanh(...): 計算 hyperbolic tangent of x element-wise.
top_k(...): 查找最後一個維度的k個最大條目的值和索引.
uniform_candidate_sampler(...): 使用統一的基本分佈對一組類進行採樣.
weighted_cross_entropy_with_logits(...): 計算加權交叉熵. (deprecated arguments)
weighted_moments(...): 返回x的頻率加權均值和方差.
with_space_to_batch(...): 對輸入的空分批表示執行運算.
xw_plus_b(...): Computes matmul(x, weights) + biases.
zero_fraction(...): 返回值中零的分數.
ers
tf.layers：
Modules
experimental module: Public API for tf.layers.experimental namespace.
Classes
class AveragePooling1D: Average Pooling layer for 1D inputs.
class AveragePooling2D: Average pooling layer for 2D inputs (e.g. images).
class AveragePooling3D: Average pooling layer for 3D inputs (e.g. volumes).
class BatchNormalization: Batch Normalization layer from http://arxiv.org/abs/1502.03167.
class Conv1D: 1D convolution layer (e.g. temporal convolution).
class Conv2D: 2D convolution layer (e.g. spatial convolution over images).
class Conv2DTranspose: Transposed 2D convolution layer (sometimes called 2D Deconvolution).
class Conv3D: 3D convolution layer (e.g. spatial convolution over volumes).
class Conv3DTranspose: Transposed 3D convolution layer (sometimes called 3D Deconvolution).
class Dense: Densely-connected layer class.
class Dropout: Applies Dropout to the input.
class Flatten: Flattens an input tensor while preserving the batch axis (axis 0).
class InputSpec: Specifies the ndim, dtype and shape of every input to a layer.
class Layer: Base layer class.
class MaxPooling1D: Max Pooling layer for 1D inputs.
class MaxPooling2D: Max pooling layer for 2D inputs (e.g. images).
class MaxPooling3D: Max pooling layer for 3D inputs (e.g. volumes).
class SeparableConv1D: Depthwise separable 1D convolution.
class SeparableConv2D: Depthwise separable 2D convolution.
Functions
average_pooling1d(...): Average Pooling layer for 1D inputs. (deprecated)
average_pooling2d(...): Average pooling layer for 2D inputs (e.g. images). (deprecated)
average_pooling3d(...): Average pooling layer for 3D inputs (e.g. volumes). (deprecated)
batch_normalization(...): Functional interface for the batch normalization layer. (deprecated)
conv1d(...): Functional interface for 1D convolution layer (e.g. temporal convolution). (deprecated)
conv2d(...): Functional interface for the 2D convolution layer. (deprecated)
conv2d_transpose(...): Functional interface for transposed 2D convolution layer. (deprecated)
conv3d(...): Functional interface for the 3D convolution layer. (deprecated)
conv3d_transpose(...): Functional interface for transposed 3D convolution layer. (deprecated)
dense(...): Functional interface for the densely-connected layer. (deprecated)
dropout(...): Applies Dropout to the input. (deprecated)
flatten(...): Flattens an input tensor while preserving the batch axis (axis 0). (deprecated)
max_pooling1d(...): Max Pooling layer for 1D inputs. (deprecated)
max_pooling2d(...): Max pooling layer for 2D inputs (e.g. images). (deprecated)
max_pooling3d(...): Max pooling layer for 3D inputs (e.g. (deprecated)
separable_conv1d(...): Functional interface for the depthwise separable 1D convolution layer. (deprecated)
separable_conv2d(...): Functional interface for the depthwise separable 2D convolution layer. (deprecated)
tf.contrib：
Modules
autograph module: This is the legacy module for AutoGraph, kept for backward compatibility.
batching module: Ops and modules related to batch.
bayesflow module: Ops for representing Bayesian computation.
checkpoint module: Tools for working with object-based checkpoints.
cloud module: Module for cloud ops.
cluster_resolver module: Standard imports for Cluster Resolvers.
compiler module: A module for controlling the Tensorflow/XLA JIT compiler.
constrained_optimization module: A library for performing constrained optimization in TensorFlow.
copy_graph module: Functions to copy elements between graphs.
crf module: Linear-chain CRF layer.
cudnn_rnn module: Ops for fused Cudnn RNN models.
data module: Experimental API for building input pipelines.
deprecated module: Non-core alias for the deprecated tf.X_summary ops.
distribute module: A distributed computation library for TF.
distributions module: Classes representing statistical distributions and ops for working with them.
eager module: TensorFlow Eager execution prototype.
estimator module: estimator python module.
factorization module: Ops and modules related to factorization.
feature_column module: Experimental utilities for tf.feature_column.
ffmpeg module: Working with audio using FFmpeg.
framework module: Framework utilities.
graph_editor module: TensorFlow Graph Editor.
grid_rnn module: GridRNN cells
image module: Ops for image manipulation.
input_pipeline module: Ops and modules related to input_pipeline.
integrate module: Integration and ODE solvers.
keras module: Implementation of the Keras API meant to be a high-level API for TensorFlow.
kernel_methods module: Ops and estimators that enable explicit kernel methods in TensorFlow.
labeled_tensor module: Labels for TensorFlow.
layers module: Ops for building neural network layers, regularizers, summaries, etc.
learn module: High level API for learning (DEPRECATED).
legacy_seq2seq module: Deprecated library for creating sequence-to-sequence models in TensorFlow.
linear_optimizer module: Ops for training linear models.
lookup module: Ops for lookup operations.
losses module: Ops for building neural network losses.
memory_stats module: Ops for memory statistics.
metrics module: Ops for evaluation metrics and summary statistics.
mixed_precision module: Library for mixed precision training.
model_pruning module: Model pruning implementation in tensorflow.
nn module: Module for variants of ops in tf.nn.
opt module: A module containing optimization routines.
optimizer_v2 module: Distribution-aware version of Optimizer.
periodic_resample module: Custom op used by periodic_resample.
predictor module: Modules for Predictors.
proto module: Ops and modules related to proto.
quantization module: Ops for building quantized models.
quantize module: Functions for rewriting graphs for quantized training.
receptive_field module: Module that declares the functions in tf.contrib.receptive_field's API.
recurrent module: Recurrent computations library.
reduce_slice_ops module: reduce by slice
remote_fused_graph module: Remote fused graph ops python library.
resampler module: Ops and modules related to resampler.
rnn module: RNN Cells and additional RNN operations.
rpc module: Ops and modules related to RPC.
saved_model module: SavedModel contrib support.
seq2seq module: Ops for building neural network seq2seq decoders and losses.
signal module: Signal processing operations.
slim module: Slim is an interface to contrib functions, examples and models.
solvers module: Ops for representing Bayesian computation.
sparsemax module: Module that implements sparsemax and sparsemax loss, see [1].
specs module: Init file, giving convenient access to all specs ops.
staging module: contrib module containing StagingArea.
stat_summarizer module: Exposes the Python wrapper for StatSummarizer utility class.
stateless module: Stateless random ops which take seed as a tensor input.
summary module: TensorFlow Summary API v2.
tensor_forest module: Random forest implementation in tensorflow.
tensorboard module: tensorboard module containing volatile or experimental code.
testing module: Testing utilities.
tfprof module: tfprof is a tool that profile various aspect of TensorFlow model.
timeseries module: A time series library in TensorFlow (TFTS).
tpu module: Ops related to Tensor Processing Units.
training module: Training and input utilities.
util module: Utilities for dealing with Tensors.