MMSegmentation从0.x迁移到1.x

New dependencies

1. MMSegmentation 1.x 依赖于一些新的软件包，建议准备一个新的干净环境,然后根据安装教程重新安装。
2. 手动安装以下软件包。

• MMEngine：MMEngine 是 OpenMMLab 2.0 架构的核心，我们将许多与计算机视觉无关的内容从 MMCV 拆分到 MMEngine 中。

• MMCV：OpenMMLab 的计算机视觉包。这不是一个新的依赖，但您需要将其升级到 2.0.0 或以上的版本。

• MMClassification（可选）：OpenMMLab 的图像分类工具箱和基准。这不是一个新的依赖，但您需要将其升级到 1.0.0rc6 版本。

• MMDetection(可选): OpenMMLab 的目标检测工具箱和基准。这不是一个新的依赖，但您需要将其升级到 3.0.0 或以上的版本。

Train launch

The main improvement of OpenMMLab 2.0 is releasing MMEngine which provides universal and powerful runner for unified interfaces to launch training jobs.

Test launch

Here’s the provided information formatted into a Markdown table:

Configuration file

Model settings

No changes in model.backbone, model.neck, model.decode_head and model.losses fields.
model.backbone, model.neck, model.decode_head and model.losses和MMSeg0.x相同

Add model.data_preprocessor field to configure the DataPreProcessor, including:

相对于MMSeg0.x添加了model.data_preprocessor 字段

• mean (Sequence, optional): The pixel mean of R, G, B channels. Defaults to None.

• std (Sequence, optional): The pixel standard deviation of R, G, B channels. Defaults to None.

• size (Sequence, optional): Fixed padding size.

• size_divisor (int, optional): The divisor of padded size.

• seg_pad_val (float, optional): Padding value of segmentation map. Default: 255.

• padding_mode (str): Type of padding. Default: ‘constant’.

• constant: pads with a constant value, this value is specified with pad_val.

• bgr_to_rgb (bool): whether to convert image from BGR to RGB.Defaults to False.

• rgb_to_bgr (bool): whether to convert image from RGB to BGR. Defaults to False.

Dataset settings

Changes in data:

The original data field is split to train_dataloader, val_dataloader and test_dataloader. This allows us to configure them in fine-grained. For example, you can specify different sampler and batch size during training and test. The samples_per_gpu is renamed to batch_size. The workers_per_gpu is renamed to num_workers.

data字段拆分为train_dataloader, val_dataloader, test_dataloader三个字段，可以在训练集，测试集分别指定不同的sampler和batch_size, samples_per_gpu重命名为batch_size， workers_per_gpu重命名为num_workers

• Original

1
2
3
4
5
6
7

data = dict(
    samples_per_gpu=4,
    workers_per_gpu=4,
    train=dict(...),
    val=dict(...),
    test=dict(...),
)

• New

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

train_dataloader = dict(
    batch_size=4,
    num_workers=4,
    dataset=dict(...),
    sampler=dict(type='DefaultSampler', shuffle=True)  # necessary
)

val_dataloader = dict(
    batch_size=4,
    num_workers=4,
    dataset=dict(...),
    sampler=dict(type='DefaultSampler', shuffle=False)  # necessary
)

test_dataloader = val_dataloader

Changes in pipeline:

• The original formatting transforms ToTensor、ImageToTensor、Collect are combined as PackSegInputs. 将ToTensor, ImageToTensor, Collect组合为一个字段PackSegInputs.

• We don’t recommend to do Normalize and Pad in the dataset pipeline. Please remove it from pipelines and set it in the data_preprocessor field. Normalize和Pad修改为在data_preprocessor中设置

• The original Resize in MMSeg 1.x has been changed to RandomResize and the input arguments img_scale is renamed to scale, and the default value of keep_ratio is modified to False. Resize修改为RandomResize，不知道这两种Resize方式是否相同，img_scale重命名为scale,功能应该是相同的，keep_ration的默认值修改为False.

• The original test_pipeline combines single-scale test and multi-scale test together, in MMSeg 1.x we separate it into test_pipeline and tta_pipeline.

train_pipeline:

• Original

1
2
3
4
5
6
7
8
9
10
11
12

train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations', reduce_zero_label=True),
    dict(type='Resize', img_scale=(2560, 640), ratio_range=(0.5, 2.0)),
    dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.75),
    dict(type='RandomFlip', prob=0.5),
    dict(type='PhotoMetricDistortion'),
    dict(type='Normalize', **img_norm_cfg),
    dict(type='Pad', size=crop_size, pad_val=0, seg_pad_val=255),
    dict(type='DefaultFormatBundle'),
    dict(type='Collect', keys=['img', 'gt_semantic_seg']),
]

• new:

1
2
3
4
5
6
7
8
9
10
11
12
13

train_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='LoadAnnotations', reduce_zero_label=True),
    dict(
        type='RandomResize',
        scale=(2560, 640),
        ratio_range=(0.5, 2.0),
        keep_ratio=True),
    dict(type='RandomCrop', crop_size=crop_size, cat_max_ratio=0.75),
    dict(type='RandomFlip', prob=0.5),
    dict(type='PhotoMetricDistortion'),
    dict(type='PackSegInputs')
]

test_pipeline:

• Original:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(
        type='MultiScaleFlipAug',
        img_scale=(2560, 640),
        # img_ratios=[0.5, 0.75, 1.0, 1.25, 1.5, 1.75],
        flip=False,
        transforms=[
            dict(type='Resize', keep_ratio=True),
            dict(type='RandomFlip'),
            dict(type='Normalize', **img_norm_cfg),
            dict(type='ImageToTensor', keys=['img']),
            dict(type='Collect', keys=['img']),
        ])
]

• New

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

test_pipeline = [
    dict(type='LoadImageFromFile'),
    dict(type='Resize', scale=(2560, 640), keep_ratio=True),
    dict(type='LoadAnnotations', reduce_zero_label=True),
    dict(type='PackSegInputs')
]
img_ratios = [0.5, 0.75, 1.0, 1.25, 1.5, 1.75]
tta_pipeline = [
    dict(type='LoadImageFromFile', backend_args=None),
    dict(
        type='TestTimeAug',
        transforms=[
            [
                dict(type='Resize', scale_factor=r, keep_ratio=True)
                for r in img_ratios
            ],
            [
                dict(type='RandomFlip', prob=0., direction='horizontal'),
                dict(type='RandomFlip', prob=1., direction='horizontal')
            ], [dict(type='LoadAnnotations')], [dict(type='PackSegInputs')]
        ])
]

将transfor从test_pipeline中拎出来放到tta_pipeline(Test Time Augmentation Pipeline，用于提高模型预测性能的模块。这个模块通过对输入图像进行一系列的变换和增强，然后将这些变换后的图像输入到模型中，最后将这些预测结果进行融合，以得到最终的预测结果。这样做可以有效地提高模型的鲁棒性和准确性。）中。

Changes in evaluation:

• The evaluation field is split to val_evaluator and test_evaluator. And it won’t support interval and save_best arguments. The interval is moved to train_cfg.val_interval, and the save_best is moved to default_hooks.checkpoint.save_best. pre_eval has been removed.单一的evaluation拆分为val_evaluator和test_evaluator同时删除了interval和save_best参数，interval放到了train_cfg.val_interval下，save_best放到了default_hooks.checkpoint.save_best下，删掉了pre_eval

• ‘mIoU’ has been changed to ‘IoUMetric’.

Original：

1

evaluation = dict(interval=2000, metric='mIoU', pre_eval=True)

New:

1
2

val_evaluator = dict(type='IoUMetric', iou_metrics=['mIoU'])
test_evaluator = val_evaluator

Optimizer and Schedule settings

Changes in optimizer and optimizer_config:

• Now we use optim_wrapper field to specify all configuration about the optimization process. And the optimizer is a sub field of optim_wrapper now.将optimizer封装到optim_wrapper下。

• paramwise_cfg is also a sub field of optim_wrapper, instead of optimizer.原本在optimizer下的optim_wrapper移动到了paramwise_cfg下。

• optimizer_config is removed now, and all configurations of it are moved to optim_wrapper. 删除了optimizer_config。

grad_clip is renamed to clip_grad. grad_clip重命名为clip_grad。

Original

1
2

optimizer = dict(type='AdamW', lr=0.0001, weight_decay=0.0005)
optimizer_config = dict(grad_clip=dict(max_norm=1, norm_type=2))

New

1
2
3
4

optim_wrapper = dict(
    type='OptimWrapper',
    optimizer=dict(type='AdamW', lr=0.0001, weight_decay=0.0005),
    clip_grad=dict(max_norm=1, norm_type=2))

Changes in lr_config:

• The lr_config field is removed and we use new param_scheduler to replace it. 用param_scheduler替换掉了lr_config, 不知道内部是否做了修改

• The warmup related arguments are removed, since we use schedulers combination to implement this functionality. 删除了warmup相关的参数，改用schedulers combination实现对应的功能。

The new schedulers combination mechanism is very flexible, and you can use it to design many kinds of learning rate / momentum curves. See the tutorial for more details. 新的schedulers combination可以实现多种learning rate / momentum curves。

Original：

1
2
3
4
5
6
7
8

lr_config = dict(
    policy='poly',
    warmup='linear',
    warmup_iters=1500,
    warmup_ratio=1e-6,
    power=1.0,
    min_lr=0.0,
    by_epoch=False)

New:

1
2
3
4
5
6
7
8
9
10
11
12

param_scheduler = [
    dict(
        type='LinearLR', start_factor=1e-6, by_epoch=False, begin=0, end=1500),
    dict(
        type='PolyLR',
        power=1.0,
        begin=1500,
        end=160000,
        eta_min=0.0,
        by_epoch=False,
    )
]

Changes in runner:

Most configuration in the original runner field is moved to train_cfg, val_cfg and test_cfg, which configure the loop in training, validation and test.大部分原来的runner字段被移动到了train_cfg，val_cfg和test_cfg中

Original：

1

runner = dict(type='IterBasedRunner', max_iters=20000)

New:

1
2
3
4

# The `val_interval` is the original `evaluation.interval`.
train_cfg = dict(type='IterBasedTrainLoop', max_iters=20000, val_interval=2000)
val_cfg = dict(type='ValLoop') # Use the default validation loop.
test_cfg = dict(type='TestLoop') # Use the default test loop.

In fact, in OpenMMLab 2.0, we introduced Loop to control the behaviors in training, validation and test. The functionalities of Runner are also changed. You can find more details of runner tutorial in MMEngine.

Runtime settings

Changes in checkpoint_config and log_config:

The checkpoint_config are moved to default_hooks.checkpoint and the log_config are moved to default_hooks.logger. And we move many hooks settings from the script code to the default_hooks field in the runtime configuration. checkpoint_config被移动到了default_hooks.checkpoint， log_config 被移动到了 default_hooks.logger，从script code移动了很多hooks settings到runtime configuration中的default_hooks。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

default_hooks = dict(
    # record the time of every iterations.
    timer=dict(type='IterTimerHook'),

    # print log every 50 iterations.
    logger=dict(type='LoggerHook', interval=50, log_metric_by_epoch=False),

    # enable the parameter scheduler.
    param_scheduler=dict(type='ParamSchedulerHook'),

    # save checkpoint every 2000 iterations.
    checkpoint=dict(type='CheckpointHook', by_epoch=False, interval=2000),

    # set sampler seed in distributed environment.
    sampler_seed=dict(type='DistSamplerSeedHook'),

    # validation results visualization.
    visualization=dict(type='SegVisualizationHook'))

In addition, we split the original logger to logger and visualizer. The logger is used to record information and the visualizer is used to show the logger in different backends, like terminal and TensorBoard. 将original logger划分为logger和visualizer. logger用于记录信息， visualizer被用于以不同的后端显示logger（terminal和TensorBoard）

Original：

1
2
3
4
5
6

log_config = dict(
    interval=100,
    hooks=[
        dict(type='TextLoggerHook'),
        dict(type='TensorboardLoggerHook'),
    ])

New:

1
2
3
4
5
6
7
8

default_hooks = dict(
    ...
    logger=dict(type='LoggerHook', interval=100),
)
vis_backends = [dict(type='LocalVisBackend'),
                dict(type='TensorboardVisBackend')]
visualizer = dict(
    type='SegLocalVisualizer', vis_backends=vis_backends, name='visualizer')

Changes in load_from and resume_from:

The resume_from is removed. And we use resume and load_from to replace it. 删除了resume_from，使用resume和load_from替代，不知道是否需要修改代码以适配。

• If resume=True and load_from is not None, resume training from the checkpoint in load_from.

• If resume=True and load_from is None, try to resume from the latest checkpoint in the work directory.

• If resume=False and load_from is not None, only load the checkpoint, not resume training.

• If resume=False and load_from is None, do not load nor resume.

Changes in dist_params: The dist_params field is a sub field of env_cfg now. And there are some new configurations in the env_cfg.

Changes in dist_params: The dist_params field is a sub field of env_cfg now. And there are some new configurations in the env_cfg. dist_params作为一个子字段放到了env_cfg下。

1
2
3
4
5
6
7
8
9
10

env_cfg = dict(
    # whether to enable cudnn benchmark
    cudnn_benchmark=False,

    # set multi process parameters
    mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0),

    # set distributed parameters
    dist_cfg=dict(backend='nccl'),
)

Changes in workflow: workflow related functionalities are removed. workflow相关的功能被删除。

New field visualizer: The visualizer is a new design in OpenMMLab 2.0 architecture. We use a visualizer instance in the runner to handle results & log visualization and save to different backends. See the visualization tutorial for more details. visualizer在OpenMMLab2.0中是一个新设计。

New field default_scope: The start point to search module for all registries. The default_scope in MMSegmentation is mmseg. See the registry tutorial for more details. default_scope是一个新加的字段 在MMSegmentation中default_scope是mmseg。