目录
参考博客: https://cuijiahua.com/blog/2018/02/dl_6.html
A standard convolution both filters and combines inputs into a new set of outputs in one step. The depthwise separable convolution splits this into two layers, a separate layer for filtering and a separate layer for combining.
一个卷积核处理输入数据时的计算量为（有Padding）：
$D_K ?D_K ?M?D_F ?D_F$
M为输入的通道数
$D_K$为卷积核的宽和高
$D_F$为输入feature map的宽和高
在某一层如果使用N个卷积核，这一个卷积层的计算量为：
$D_K ?D_K ?M?N*D_F ?D_F$
如果采用 Depthwise Convolutional Filters,标准交卷:
而深度可分离卷积:
一组和输入通道数相同的2D卷积核的运算量为：
$D_KD_kMD_FD_F$
3D的1x1卷积核的计算量为:
$NMD_F*D_F$
因此这种组合方式的计算量:
$D_KD_kMD_FD_F+NMD_F*D_F$
相比较，Depthwise Separable Convolution 的计算量:
举一个具体的例子，给定输入图像的为 3 通道的 224x224 的图像，VGG16网络的第3个卷积层conv2_1输入的是尺寸为 112 的特征图，通道数为 64 ，卷积核尺寸为 3，卷积核个数为 128，传统卷积运算量就是：
而将传统3D卷积替换为deep-wise结果1x1方式的卷积，计算量为:
可见这一层，计算量比例:
传统的3D卷积常见的使用方式如下图左侧所示，deep-wise卷积的使用方式如下图右边所示:
Deepwise结合 1x1 的卷积方式代替传统卷积不仅在理论上会更高效，而且由于大量使用 1x1 的卷积，可以直接使用高度优化的数学库来完成这个操作。以Caffe为例，如果要使用这些数学库，要首先使用 im2col 的方式来对数据进行重新排布，从而确保满足此类数学库的输入形式；但是 1x1 方式的卷积不需要这种预处理。
在MobileNet中，有95%的计算量和75%的参数属于1x1卷积:
宽度因子
宽度因子α是一个属于(0,1]之间的数，附加于网络的通道数。简单来说就是新网络中每一个模块要使用的卷积核数量相较于标准的MobileNet比例。对于deep-wise结合1x1方式的卷积核，计算量为：
分辨率因子
分辨率因子β的取值范围在(0,1]之间，是作用于每一个模块输入尺寸的约减因子，简单来说就是将输入数据以及由此在每一个模块产生的特征图都变小了，结合宽度因子α，deep-wise结合1x1方式的卷积核计算量为：
在代码中，并没有实现分辨率因子,而是多了一个depth_multiplier参数:
"""MobileNet v1 models for Keras.MobileNet is a general architecture and can be used for multiple use cases.Depending on the use case, it can use different input layer size anddifferent width factors. This allows different width models to reducethe number of multiply-adds and therebyreduce inference cost on mobile devices.MobileNets support any input size greater than 32 x 32, with larger image sizesoffering better performance.The number of parameters and number of multiply-addscan be modified by using the `alpha` parameter,which increases/decreases the number of filters in each layer.By altering the image size and `alpha` parameter,all 16 models from the paper can be built, with ImageNet weights provided.The paper demonstrates the performance of MobileNets using `alpha` values of1.0 (also called 100 % MobileNet), 0.75, 0.5 and 0.25.For each of these `alpha` values, weights for 4 different input image sizesare provided (224, 192, 160, 128).The following table describes the size and accuracy of the 100% MobileNeton size 224 x 224:----------------------------------------------------------------------------Width Multiplier (alpha) | ImageNet Acc | Multiply-Adds (M) | Params (M)----------------------------------------------------------------------------| 1.0 MobileNet-224 | 70.6 % | 529 | 4.2 || 0.75 MobileNet-224 | 68.4 % | 325 | 2.6 || 0.50 MobileNet-224 | 63.7 % | 149 | 1.3 || 0.25 MobileNet-224 | 50.6 % | 41 | 0.5 |----------------------------------------------------------------------------The following table describes the performance ofthe 100 % MobileNet on various input sizes:------------------------------------------------------------------------ Resolution | ImageNet Acc | Multiply-Adds (M) | Params (M)------------------------------------------------------------------------| 1.0 MobileNet-224 | 70.6 % | 529 | 4.2 || 1.0 MobileNet-192 | 69.1 % | 529 | 4.2 || 1.0 MobileNet-160 | 67.2 % | 529 | 4.2 || 1.0 MobileNet-128 | 64.4 % | 529 | 4.2 |------------------------------------------------------------------------The weights for all 16 models are obtained and translatedfrom Tensorflow checkpoints found athttps://github.com/tensorflow/models/blob/master/slim/nets/mobilenet_v1.md# Reference- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://arxiv.org/pdf/1704.04861.pdf))"""from keras.models import Modelfrom keras.layers import Input, Activation, Dropout, Reshape, BatchNormalization, GlobalAveragePooling2D, GlobalMaxPooling2Dfrom keras.layers import Conv2D, DepthwiseConv2Dfrom keras.utils import plot_modelfrom keras import backend as K def relu6(x): return K.relu(x, max_value=6) def _make_divisiable(v, divisor=8, min_value=8): """分段函数,保证能够被divisor整除，最小数是min_value""" if min_value is None: min_value = divisor new_v = max(min_value, int(v + divisor/2) // divisor * divisor) # Make sure that round down does not go down by more than 10%. if new_v < 0.9 * v: new_v += divisor return new_v def _conv_bolck(inputs, filters, alpha, kernel=(3, 3), strides=(1, 1), bn_epsilon=1e-3, bn_momentum=0.99, block_id=1): """ Adds an initial convolution layer (with batch normalization and relu6). Args: inputs: Input tensor of shape `(rows, cols, 3)` (with `channels_last` data format) or (3, rows, cols) (with `channels_first` data format). It should have exactly 3 inputs channels, and width and height should be no smaller than 32. E.g. `(224, 224, 3)` would be one valid value. filters: Integer, the dimensionality of the output space. (i.e. the number output of filters in the convolution). alpha: controls the width of the network. - If `alpha` < 1.0, proportionally decreases the number of filters in each layer. - If `alpha` > 1.0, proportionally increases the number of filters in each layer. - If `alpha` = 1, default number of filters from the paper are used at each layer. kernel: An integer or tuple/list of 2 integers, specifying the width and height of the 2D convolution window. Can be a single integer to specify the same value for all spatial dimensions. strides: An integer or tuple/list of 2 integers, specifying the strides of the convolution along the width and height. Can be a single integer to specify the same value for all spatial dimensions. Specifying any stride value != 1 is incompatible with specifying any `dilation_rate` value != 1. bn_epsilon: Epsilon value for BatchNormalization bn_momentum: Momentum value for BatchNormalization block_id: Integer, a unique identification designating the block number. Returns: Output tensor of block Input shape: 4D tensor with shape: `(samples, channels, rows, cols)` if data_format=‘channels_first‘ or `(samples, rows, cols, channels)` if data_format=‘channels_last‘. Output shape: 4D tensor with shape: `(samples, filters, new_rows, new_cols)` if data_format=‘channels_first‘ or `(samples, new_rows, new_cols, filters)` if data_format=‘channels_last‘. `rows` and `cols` values might have changed due to stride. """ channel_axis = 1 if K.image_data_format() == ‘channels_first‘ else -1 filters = _make_divisiable(filters * alpha) # 乘以宽度因子后的卷积核数量，可能不能被divisor=8整除 x = Conv2D(filters, kernel, use_bias=False, strides=strides, name=‘conv{}‘.format(block_id))(inputs) x = BatchNormalization(axis=channel_axis, momentum=bn_momentum, epsilon=bn_epsilon, name=‘conv{}_bn‘.format(block_id))(x) return Activation(relu6, name=‘conv{}_relu‘.format(block_id))(x) def _depthwise_conv_block(inputs, pointwise_conv_filters, alpha, depth_multiplier=1, strides=(1, 1), bn_epsilon=1e-3, block_id=1): """Adds a depthwise convolution block. A depthwise convolution block consists of a depthwise conv, batch normalization, relu6, pointwise convolution, batch normalization and relu6 Args: inputs: Input tensor of shape `(rows, cols, channels)`(with `channels_last` data format) or (channels, rows, cols)(with `channels_first` data format) pointwise_conv_filters: Integer, the dimensionality of the output space (i.e. the number output of filters in the pointwise convolution). alpha: controls the width of the network. - If `alpha` < 1.0, proportionally decreases the number of filters in each layer. - If `alpha` > 1.0, proportionally increases the number of filters in each layer. - If `alpha` = 1, default number of filters from the paper are used at each layer. depth_multiplier: The number of depthwise convolution output channels for each channel. The total number of depthwise convolution output channels will be equal to `filters_in * depth_multiplier`. 每个通道的深度卷积输出通道的数量 strides: An integer or tuple/list of 2 integers, specifying the strides of the convolution along the width and height. Can be a single integer to specify the same value for all spatial dimensions. Specifying any stride value != 1 is incompatible with specifying any `dilation_rate` value != 1. bn_epsilon: Epsilon value for BatchNormalization block_id: Integer, a unique identification designating the block number. Returns: Output tensor of block Input shape: 4D tensor with shape: `(batch, channels, rows, cols)` if data_format=‘channels_first‘ or `(batch, rows, cols, channels)` if data_format=‘channels_last‘. Output shape: 4D tensor with shape: `(batch, filters, new_rows, new_cols)` if data_format=‘channels_first‘ or `(batch, new_rows, new_cols, filters)` if data_format=‘channels_last‘. `rows` and `cols` values might have changed due to stride. """ channel_axis = 1 if K.image_data_format() == ‘channels_first‘ else -1 pointwise_conv_filters = _make_divisiable(pointwise_conv_filters * alpha) # Depthwise Conv2D # 只有depth_multiplier个卷积核，其将卷积操作分解，实际上卷积核shape: 3 x 3 x input_channels x depth_multiplier # 以下面为例,DepthwiseConv2D输出的tensor的shape: (batch, rows, cols, input_channels * depth_multiplier) x = DepthwiseConv2D(kernel_size=(3, 3), padding=‘same‘, depth_multiplier=depth_multiplier, strides=strides, use_bias=False, name=‘conv_dw_{}‘.format(block_id))(inputs) x = BatchNormalization(axis=channel_axis, epsilon=bn_epsilon, name=‘conv_dw_{}_bn‘.format(block_id))(x) x = Activation(relu6, name=‘conv_dw_{}_relu‘.format(block_id))(x) # Pointwise Conv2D pointwise_conv_filters控制最终out_channels x = Conv2D(pointwise_conv_filters, kernel_size=(1, 1), padding=‘same‘, use_bias=False, strides=(1, 1), name=‘conv_pw_{}‘.format(block_id))(x) x = BatchNormalization(axis=channel_axis, epsilon=bn_epsilon, name=‘conv_pw_{}_bn‘.format(block_id))(x) return Activation(relu6, name=‘conv_pw_{}_relu‘.format(block_id))(x) def mobilenetv1(input_shape, alpha=1.0, depth_multiplier=1, dropout=1e-3, classes=1000): """Instantiates the MobileNet architecture. Args: input_shape: optional shape tuple, only to be specified if `include_top` is False. (otherwise the input shape has to be `(224, 224, 3)` (with `channels_last` data format) or (3, 224, 224) (with `channels_first` data format). It should have exactly 3 inputs channels, and width and height should be no smaller than 32. E.g. `(200, 200, 3)` would be one valid value. alpha: controls the width of the network. - If `alpha` < 1.0, proportionally decreases the number of filters in each layer. - If `alpha` > 1.0, proportionally increases the number of filters in each layer. - If `alpha` = 1, default number of filters from the paper are used at each layer. depth_multiplier: depth multiplier for depthwise convolution dropout: dropout rate classes: optional number of classes to classify images into Returns: A Keras model instance. Raises: ValueError: in case of invalid argument for `weights`, or invalid input shape. RuntimeError: If attempting to run this model with a backend that does not support separable convolutions. """ x_input = Input(shape=input_shape) x = _conv_bolck(x_input, 32, alpha, strides=(2, 2)) x = _depthwise_conv_block(x, 64, alpha, depth_multiplier, block_id=1) x = _depthwise_conv_block(x, 128, alpha, depth_multiplier, strides=(2, 2), block_id=2) x = _depthwise_conv_block(x, 128, alpha, depth_multiplier, block_id=3) x = _depthwise_conv_block(x, 256, alpha, depth_multiplier, strides=(2, 2), block_id=4) x = _depthwise_conv_block(x, 256, alpha, depth_multiplier, block_id=5) x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, strides=(2, 2),block_id=6) x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=7) x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=8) x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=9) x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=10) x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, block_id=11) x = _depthwise_conv_block(x, 512, alpha, depth_multiplier, strides=(2, 2),block_id=12) x = _depthwise_conv_block(x, 1024, alpha, depth_multiplier, block_id=13) shape = (1, 1, int(1024 * alpha)) x = GlobalAveragePooling2D()(x) x = Reshape(shape, name=‘reshape_1‘)(x) x = Dropout(dropout, name=‘dropout‘)(x) x = Conv2D(classes, (1, 1), padding=‘same‘, name=‘conv_preds‘)(x) x = Activation(‘softmax‘, name=‘act_sotmax‘)(x) x = Reshape((classes,), name=‘reshape_2‘)(x) return Model(x_input, x) if __name__ == ‘__main__‘: alpha = 1 depth_multiplier = 1 mobilenet = mobilenetv1(input_shape=(224, 224, 3), alpha=alpha, depth_multiplier=depth_multiplier) mobilenet.summary() plot_model(mobilenet, show_shapes=True, to_file=‘mobilenet_alpha{}_depth_multiplier_{}.png‘.format(alpha, depth_multiplier))
