MMCV Operators¶
To make custom operators in MMCV more standard, precise definitions of each operator are listed in this document.
MMCV Operators
MMCVBorderAlign
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVCARAFE
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVCAWeight
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVCAMap
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVCornerPool
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVDeformConv2d
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVModulatedDeformConv2d
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVDeformRoIPool
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVMaskedConv2d
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVPSAMask
Description
Parameters
Inputs
Outputs
Type Constraints
NonMaxSuppression
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVRoIAlign
Description
Parameters
Inputs
Outputs
Type Constraints
MMCVRoIAlignRotated
Description
Parameters
Inputs
Outputs
Type Constraints
grid_sampler*
Description
Parameters
Inputs
Outputs
Type Constraints
cummax*
Description
Parameters
Inputs
Outputs
Type Constraints
cummin*
Description
Parameters
Inputs
Outputs
Type Constraints
Reminders
MMCVBorderAlign¶
Description¶
Applies border_align
over the input feature based on predicted bboxes.
For each border line (e.g. top, left, bottom or right) of each box, border_align does the following:
uniformly samples
pool_size
+1 positions on this line, involving the start and end points.the corresponding features on these points are computed by bilinear interpolation.
max pooling over all the
pool_size
+1 positions are used for computing pooled feature.
Read BorderDet: Border Feature for Dense Object Detection for more detailed information.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
pool_size |
number of positions sampled over the boxes' borders(e.g. top, bottom, left, right). |
Inputs¶
- input: T
- Features with shape [N,4C,H,W]. Channels ranged in [0,C), [C,2C), [2C,3C), [3C,4C) represent the top, left, bottom, right features respectively
- boxes: T
- Boxes with shape [N,H*W,4]. Coordinate format (x1,y1,x2,y2).
Outputs¶
- output: T
- Pooled features with shape [N,C,H*W,4]. The order is(top,left,bottom,right) for the last dimension.
Type Constraints¶
T:tensor(float32)
MMCVCARAFE¶
Description¶
CARAFE operator performs feature upsampling.
Read CARAFE: Content-Aware ReAssembly of FEatures for more detailed information.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
kernel_size |
reassemble kernel size, should be odd integer |
int |
group_size |
reassemble group size |
float |
scale_factor |
upsample ratio(>=1) |
Inputs¶
- features: T
- Input features. 4-D tensor of shape (N, C, H, W). N is the batch size.
- masks: T
- The input mask
Outputs¶
- output: T
- The upsampled features. 4-D tensor of shape (N, C, H * scale_factor, W * scale_factor). N is the batch size.
Type Constraints¶
T:tensor(float32)
MMCVCAWeight¶
Description¶
Operator for Criss-Cross Attention Read CCNet: Criss-Cross Attention for SemanticSegmentation for more detailed information.
Parameters¶
None
Inputs¶
- t: T
- The query matrix of shape (N, C', H, W).
- f: T
- The key matrix of shape (N, C', H, W).
Outputs¶
- weight: T
- The attention map of shape (N, H+W-1, H, W).
Type Constraints¶
T:tensor(float32)
MMCVCAMap¶
Description¶
Operator for Criss-Cross Attention Read CCNet: Criss-Cross Attention for SemanticSegmentation for more detailed information.
Parameters¶
None
Inputs¶
- weight: T
- Output from the operator MMCVCAWeight.
- value: T
- The value matrix of shape (N, C, H, W).
Outputs¶
- output: T
- Output tensor of aggregated contextual information
Type Constraints¶
T:tensor(float32)
MMCVCornerPool¶
Description¶
Perform CornerPool on input
features. Read CornerNet – Detecting Objects as Paired Keypoints for more details.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
mode |
corner pool mode, (0: top , 1: bottom , 2: left , 3: right ) |
Inputs¶
- input: T
- Input features. 4-D tensor of shape (N, C, H, W). N is the batch size.
Outputs¶
- output: T
- The pooled features. 4-D tensor of shape (N, C, H, W).
Type Constraints¶
T:tensor(float32)
MMCVDeformConv2d¶
Description¶
Applies a deformable 2D convolution over an input signal composed of several input planes.
Read Deformable Convolutional Networks for detail.
Parameters¶
Type | Parameter | Description |
---|---|---|
list of ints |
stride |
The stride of the convolving kernel, (sH, sW). Defaults to (1, 1) . |
list of ints |
padding |
Paddings on both sides of the input, (padH, padW). Defaults to (0, 0) . |
list of ints |
dilation |
The spacing between kernel elements (dH, dW). Defaults to (1, 1) . |
int |
groups |
Split input into groups. input_channel should be divisible by the number of groups. Defaults to 1 . |
int |
deformable_groups |
Groups of deformable offset. Defaults to 1 . |
int |
bias |
Whether to add a learnable bias to the output. 0 stands for False and 1 stands for True . Defaults to 0 . |
int |
im2col_step |
Groups of deformable offset. Defaults to 32 . |
Inputs¶
- input: T
- Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the number of channels, inH and inW are the height and width of the data.
- offset: T
- Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW are the height and width of weight, outH and outW is the height and width of offset and output.
- weight: T
- Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
Outputs¶
- output: T
- Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
Type Constraints¶
T:tensor(float32, Linear)
MMCVModulatedDeformConv2d¶
Description¶
Perform Modulated Deformable Convolution on input feature, read Deformable ConvNets v2: More Deformable, Better Results for detail.
Parameters¶
Type | Parameter | Description |
---|---|---|
list of ints |
stride |
The stride of the convolving kernel. (sH, sW) |
list of ints |
padding |
Paddings on both sides of the input. (padH, padW) |
list of ints |
dilation |
The spacing between kernel elements. (dH, dW) |
int |
deformable_groups |
Groups of deformable offset. |
int |
groups |
Split input into groups. input_channel should be divisible by the number of groups. |
Inputs¶
- feature: T
- Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the number of channels, inH and inW are the height and width of the data.
- offset: T
- Input offset; 4-D tensor of shape (N, deformable_group* 2* kH* kW, outH, outW), where kH and kW are the height and width of weight, outH and outW are the height and width of offset and output.
- mask: T
- Input mask; 4-D tensor of shape (N, deformable_group* kH* kW, outH, outW), where kH and kW are the height and width of weight, outH and outW are the height and width of offset and output.
- weight]: T
- Input weight; 4-D tensor of shape (output_channel, input_channel, kH, kW).
- bias: T, optional
- Input bias; 1-D tensor of shape (output_channel).
Outputs¶
- output: T
- Output feature; 4-D tensor of shape (N, output_channel, outH, outW).
Type Constraints¶
T:tensor(float32, Linear)
MMCVDeformRoIPool¶
Description¶
Deformable roi pooling layer
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
output_height |
height of output roi |
int |
output_width |
width of output roi |
float |
spatial_scale |
used to scale the input boxes |
int |
sampling_ratio |
number of input samples to take for each output sample. 0 means to take samples densely for current models. |
float |
gamma |
gamma |
Inputs¶
- input: T
- Input feature map; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
- rois: T
- RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of input.
- offset: T
- offset of height and width. Defaults to a tensor of zero
Outputs¶
- feat: T
- RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element feat[r-1] is a pooled feature map corresponding to the r-th RoI RoIs[r-1].
Type Constraints¶
T:tensor(float32)
MMCVMaskedConv2d¶
Description¶
Performs a masked 2D convolution from PixelRNN Read Pixel Recurrent Neural Networks for more detailed information.
Parameters¶
Type | Parameter | Description |
---|---|---|
list of ints |
stride |
The stride of the convolving kernel. (sH, sW). Only support stride=1 in mmcv |
list of ints |
padding |
Paddings on both sides of the input. (padH, padW). Defaults to (0, 0) . |
Inputs¶
- features: T
- Input features; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
- mask: T
- Input mask; 3D tensor of shape (N, H, W)
- weight: T
- The learnable weights of the module
- bias: T
- The learnable bias of the module
Outputs¶
- output: T
- The output convolved feature
Type Constraints¶
T:tensor(float32)
MMCVPSAMask¶
Description¶
An operator from PSANet.
Read PSANet: Point-wise Spatial Attention Network for Scene Parsing for more detailed information.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
psa_type |
0 means collect and 1 means distribute |
list of ints |
mask_size |
The size of mask |
Inputs¶
- input: T
- Input feature; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
Outputs¶
- output: T
- Output tensor of shape (N, H * W, H, W)
Type Constraints¶
T:tensor(float32)
NonMaxSuppression¶
Description¶
Filter out boxes has high IoU overlap with previously selected boxes or low score. Output the indices of valid boxes.
Note this definition is slightly different with onnx: NonMaxSuppression
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
center_point_box |
0 - the box data is supplied as [y1, x1, y2, x2], 1-the box data is supplied as [x_center, y_center, width, height]. |
int |
max_output_boxes_per_class |
The maximum number of boxes to be selected per batch per class. Default to 0, number of output boxes equal to number of input boxes. |
float |
iou_threshold |
The threshold for deciding whether boxes overlap too much with respect to IoU. Value range [0, 1]. Default to 0. |
float |
score_threshold |
The threshold for deciding when to remove boxes based on score. |
int |
offset |
0 or 1, boxes' width or height is (x2 - x1 + offset). |
Inputs¶
- boxes: T
- Input boxes. 3-D tensor of shape (num_batches, spatial_dimension, 4).
- scores: T
- Input scores. 3-D tensor of shape (num_batches, num_classes, spatial_dimension).
Outputs¶
- indices: tensor(int32, Linear)
- Selected indices. 2-D tensor of shape (num_selected_indices, 3) as [[batch_index, class_index, box_index], ...].
- num_selected_indices=num_batches* num_classes* min(max_output_boxes_per_class, spatial_dimension).
- All invalid indices will be filled with -1.
Type Constraints¶
T:tensor(float32, Linear)
MMCVRoIAlign¶
Description¶
Perform RoIAlign on output feature, used in bbox_head of most two-stage detectors.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
output_height |
height of output roi |
int |
output_width |
width of output roi |
float |
spatial_scale |
used to scale the input boxes |
int |
sampling_ratio |
number of input samples to take for each output sample. 0 means to take samples densely for current models. |
str |
mode |
pooling mode in each bin. avg or max |
int |
aligned |
If aligned=0 , use the legacy implementation in MMDetection. Else, align the results more perfectly. |
Inputs¶
- input: T
- Input feature map; 4D tensor of shape (N, C, H, W), where N is the batch size, C is the numbers of channels, H and W are the height and width of the data.
- rois: T
- RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of input.
Outputs¶
- feat: T
- RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element feat[r-1] is a pooled feature map corresponding to the r-th RoI RoIs[r-1].
Type Constraints¶
T:tensor(float32)
MMCVRoIAlignRotated¶
Description¶
Perform RoI align pooling for rotated proposals
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
output_height |
height of output roi |
int |
output_width |
width of output roi |
float |
spatial_scale |
used to scale the input boxes |
int |
sampling_ratio |
number of input samples to take for each output sample. 0 means to take samples densely for current models. |
str |
mode |
pooling mode in each bin. avg or max |
int |
aligned |
If aligned=0 , use the legacy implementation in MMDetection. Else, align the results more perfectly. |
int |
clockwise |
If aligned=0 , use the legacy implementation in MMDetection. Else, align the results more perfectly. |
Inputs¶
- features: T
- Input feature map; 4D tensor of shape (N, C, H, W)
- rois: T
- RoIs (Regions of Interest) to pool over; 2-D tensor of shape (num_rois, 5) given as [[batch_index, x1, y1, x2, y2], ...]. The RoIs' coordinates are the coordinate system of input.
Outputs¶
- RoI pooled output, 4-D tensor of shape (num_rois, C, output_height, output_width). The r-th batch element feat[r-1] is a pooled feature map corresponding to the r-th RoI RoIs[r-1].
Type Constraints¶
T:tensor(float32)
grid_sampler*¶
Description¶
Perform sample from input
with pixel locations from grid
.
Check torch.nn.functional.grid_sample for more information.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
interpolation_mode |
Interpolation mode to calculate output values. (0: bilinear , 1: nearest ) |
int |
padding_mode |
Padding mode for outside grid values. (0: zeros , 1: border , 2: reflection ) |
int |
align_corners |
If align_corners=1 , the extrema (-1 and 1 ) are considered as referring to the center points of the input's corner pixels. If align_corners=0 , they are instead considered as referring to the corner points of the input's corner pixels, making the sampling more resolution agnostic. |
Inputs¶
- input: T
- Input feature; 4-D tensor of shape (N, C, inH, inW), where N is the batch size, C is the numbers of channels, inH and inW are the height and width of the data.
- grid: T
- Input offset; 4-D tensor of shape (N, outH, outW, 2), where outH and outW are the height and width of offset and output.
Outputs¶
- output: T
- Output feature; 4-D tensor of shape (N, C, outH, outW).
Type Constraints¶
T:tensor(float32, Linear)
cummax*¶
Description¶
Returns a tuple (values
, indices
) where values
is the cumulative maximum elements of input
in the dimension dim
. And indices
is the index location of each maximum value found in the dimension dim
. Read torch.cummax for more details.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
dim |
the dimension to do the operation over |
Inputs¶
- input: T
- The input tensor with various shapes. Tensor with empty element is also supported.
Outputs¶
- output: T
- Output the cumulative maximum elements of `input` in the dimension `dim`, with the same shape and dtype as `input`.
- indices: tensor(int64)
- Output the index location of each cumulative maximum value found in the dimension `dim`, with the same shape as `input`.
Type Constraints¶
T:tensor(float32)
cummin*¶
Description¶
Returns a tuple (values
, indices
) where values
is the cumulative minimum elements of input
in the dimension dim
. And indices
is the index location of each minimum value found in the dimension dim
. Read torch.cummin for more details.
Parameters¶
Type | Parameter | Description |
---|---|---|
int |
dim |
the dimension to do the operation over |
Inputs¶
- input: T
- The input tensor with various shapes. Tensor with empty element is also supported.
Outputs¶
- output: T
- Output the cumulative minimum elements of `input` in the dimension `dim`, with the same shape and dtype as `input`.
- indices: tensor(int64)
- Output the index location of each cumulative minimum value found in the dimension `dim`, with the same shape as `input`.
Type Constraints¶
T:tensor(float32)
Reminders¶
Operators endwith
*
are defined in Torch and are included here for the conversion to ONNX.