Data Transform Migration¶

Introduction¶

In MMOCR version 0.x, we implemented a series of Data Transform methods in mmocr/datasets/pipelines/xxx_transforms.py. However, these modules are scattered all over the place and lack a standardized design. Therefore, we refactored all the data transform modules in MMOCR version 1.x. According to the task type, they are now defined in ocr_transforms.py, textdet_transforms.py, and textrecog_transforms.py, respectively, under mmocr/datasets/transforms. Specifically, ocr_transforms.py implements the data augmentation methods for OCR-related tasks in general, while textdet_transforms.py and textrecog_transforms.py implement data augmentation transforms related to text detection and text recognition tasks, respectively.

Since some of the modules were renamed, merged or separated during the refactoring process, the new interface and default parameters may be inconsistent with the old version. Therefore, this migration guide will introduce how to configure the new data transforms to achieve the identical behavior as the old version.

Configuration Migration Guide¶

Data Formatting Related Data Transforms¶

Collect + CustomFormatBundle -> PackTextDetInputs/PackTextRecogInputs

PackxxxInputs implements both Collect and CustomFormatBundle functions, and no longer has key parameters, the generation of training targets is moved to be done in loss modules.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
    type='CustomFormatBundle',
    keys=['gt_shrink', 'gt_shrink_mask', 'gt_thr', 'gt_thr_mask'],
    meta_keys=['img_path', 'ori_shape', 'img_shape'],
    visualize=dict(flag=False, boundary_key='gt_shrink')),
dict(
    type='Collect',
    keys=['img', 'gt_shrink', 'gt_shrink_mask', 'gt_thr', 'gt_thr_mask'])

dict(
  type='PackTextDetInputs',
  meta_keys=('img_path', 'ori_shape', 'img_shape'))

Data Augmentation Related Data Transforms¶

ResizeOCR -> Resize, RescaleToHeight, PadToWidth

The original ResizeOCR is now split into three data augmentation modules.

When keep_aspect_ratio=False, it is equivalent to Resize in version 1.x. Its configuration can be modified as follows.

MMOCR 0.x Configuration	MMOCR 1.x Configuration
dict( type='ResizeOCR', height=32, min_width=100, max_width=100, keep_aspect_ratio=False)	dict( type='Resize', scale=(100, 32), keep_ratio=False)

When keep_aspect_ratio=True and max_width=None. The image will be rescaled to a fixed size alongside the height while keeping the aspect ratio the same as the origin.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='ResizeOCR',
  height=32,
  min_width=32,
  max_width=None,
  width_downsample_ratio = 1.0 / 16
  keep_aspect_ratio=True)

dict(
  type='RescaleToHeight',
  height=32,
  min_width=32,
  max_width=None,
  width_divisor=16),

When keep_aspect_ratio=True and max_width is a fixed value. The image will be rescaled to a fixed size alongside the height while keeping the aspect ratio the same as the origin. Then, the width will be padded or cropped to max_width. That is to say, the shape of the output image is always (height, max_width).

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='ResizeOCR',
  height=32,
  min_width=32,
  max_width=100,
  width_downsample_ratio = 1.0 / 16,
  keep_aspect_ratio=True)

dict(
  type='RescaleToHeight',
  height=32,
  min_width=32,
  max_width=100,
  width_divisor=16),
dict(
  type='PadToWidth',
  width=100)

RandomRotateTextDet & RandomRotatePolyInstances -> RandomRotate

We implemented all random rotation-related data augmentation in RandomRotate in version 1.x. Its default behavior is identical to the RandomRotateTextDet in version 0.x.

Note

The default value of “max_angle” might be different from the old version, so the users are suggested to manually set the number.

MMOCR 0.x Configuration	MMOCR 1.x Configuration
dict(type='RandomRotateTextDet')	dict(type='RandomRotate', max_angle=10)

For RandomRotatePolyInstances，it is supposed to set use_canvas=True。

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='RandomRotatePolyInstances',
  rotate_ratio=0.5, # Specify the execution probability
  max_angle=60,
  pad_with_fixed_color=False)

# Wrap the data transforms with RandomApply and specify the execution probability
dict(
  type='RandomApply',
  transforms=[
    dict(type='RandomRotate',
    max_angle=60,
    pad_with_fixed_color=False,
    use_canvas=True)],
  prob=0.5) # Specify the execution probability

Note

In version 0.x, some data augmentation methods specified execution probability by defining an internal variable “xxx_ratio”, such as “rotate_ratio”, “crop_ratio”, etc. In version 1.x, these parameters have been removed. Now we can use “RandomApply” to wrap different data transforms and specify their execution probabilities.

RandomCropFlip -> TextDetRandomCropFlip

Currently, only the method name has been changed, and other parameters remain the same.
RandomCropPolyInstances -> RandomCrop

In MMOCR version 1.x, crop_ratio and instance_key are removed. The gt_polygons is now used as the target for cropping.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='RandomCropPolyInstances',
  instance_key='gt_masks',
  crop_ratio=0.8, # Specify the execution probability
  min_side_ratio=0.3)

# Wrap the data transforms with RandomApply and specify the execution probability
dict(
  type='RandomApply',
  transforms=[dict(type='RandomCrop', min_side_ratio=0.3)],
  prob=0.8) # Specify the execution probability

RandomCropInstances -> TextDetRandomCrop

In MMOCR version 1.x, crop_ratio and instance_key are removed. The gt_polygons is now used as the target for cropping.

MMOCR 0.x Configuration	MMOCR 1.x Configuration
dict( type='RandomCropInstances', target_size=(800，800), instance_key='gt_kernels')	dict( type='TextDetRandomCrop', target_size=(800，800))

EastRandomCrop -> RandomCrop + Resize + mmengine.Pad

EastRandomCrop was implemented by applying cropping, scaling and padding to the input image. Now, the same effect can be achieved by combining three data transforms.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='EastRandomCrop',
  max_tries=10,
  min_crop_side_ratio=0.1,
  target_size=(640, 640))

dict(type='RandomCrop', min_side_ratio=0.1),
dict(type='Resize', scale=(640,640), keep_ratio=True),
dict(type='Pad', size=(640,640))

RandomScaling -> mmengine.RandomResize

The RandomScaling is now replaced with mmengine.RandomResize.

MMOCR 0.x Configuration	MMOCR 1.x Configuration
dict( type='RandomScaling', size=800, scale=(0.75, 2.5))	dict( type='RandomResize', scale=(800, 800), ratio_range=(0.75, 2.5), keep_ratio=True)

Note

By default, the data pipeline will search for the corresponding data transforms from the register of the current scope, and if that data transform does not exist, it will continue to search in the upstream library, such as MMCV and MMEngine. For example, the RandomResize transform is not implemented in MMOCR, but it can be directly called in the configuration, as the program will automatically search for it from MMCV. In addition, you can also specify scope by adding a prefix. For example, mmengine.RandomResize will force it to use RandomResize implemented in MMEngine, which is useful when a method of the same name exists in both upstream and downstream libraries. It is noteworthy that all of the data transforms implemented in MMCV are registered to MMEngine, that is why we use mmengine.RandomResize but not mmcv.RandomResize.

SquareResizePad -> Resize + SourceImagePad

SquareResizePad implements two branches and uses one of them randomly based on the pad_ratio. Specifically, one branch first resizes the image and then pads it to a certain size; while the other branch only resizes the image. To enhance the reusability of the different modules, we split this data transform into a combination of Resize + SourceImagePad in version 1.x, and control the branches via RandomChoice.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='SquareResizePad',
  target_size=800,
  pad_ratio=0.6)

dict(
  type='RandomChoice',
  transforms=[
    [
      dict(
        type='Resize',
        scale=800,
        keep_ratio=True),
      dict(
        type='SourceImagePad',
        target_scale=800)
    ],
    [
      dict(
        type='Resize',
        scale=800,
        keep_ratio=False)
    ]
  ],
  prob=[0.4, 0.6]), # Probability of selection of two combinations

Note

In version 1.x, the random choice wrapper “RandomChoice” replaces “OneOfWrapper”, allowing random selection of data transform combinations.

RandomWrapper -> mmengine.RandomApply

In version 1.x, the RandomWrapper wrapper has been replaced with RandomApply in MMEngine, which is used to specify the probability of performing a data transform. And the probability p is now named prob.

MMOCR 0.x Configuration	MMOCR 1.x Configuration
dict( type='RandomWrapper', p=0.25, transforms=[ dict(type='PyramidRescale'), ])	dict( type='RandomApply', prob=0.25, transforms=[ dict(type='PyramidRescale'), ])

OneOfWrapper -> mmengine.RandomChoice

The random choice wrapper is now renamed to RandomChoice and is used in exactly the same way as before.
ScaleAspectJitter -> ShortScaleAspectJitter, BoundedScaleAspectJitter

The ScaleAspectJitter implemented several different image size jittering strategies, which has now been split into several independent data transforms.

When resize_type='indep_sample_in_range', it is equivalent to RandomResize.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='ScaleAspectJitter',
  img_scale=None,
  keep_ratio=False,
  resize_type='indep_sample_in_range',
  scale_range=(640, 2560))

 dict(
  type='RandomResize',
  scale=(640, 640),
  ratio_range=(1.0, 4.125),
  resize_type='Resize',
  keep_ratio=True))

When resize_type='long_short_bound', we implemented BoundedScaleAspectJitter, which randomly rescales the image so that the long and short sides of the image are around the bound; then jitters the aspect ratio.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='ScaleAspectJitter',
  img_scale=[(3000, 736)],  # Unused
  ratio_range=(0.7, 1.3),
  aspect_ratio_range=(0.9, 1.1),
  multiscale_mode='value',
  long_size_bound=800,
  short_size_bound=480,
  resize_type='long_short_bound',
  keep_ratio=False)

dict(
  type='BoundedScaleAspectJitter',
  long_size_bound=800,
  short_size_bound=480,
  ratio_range=(0.7, 1.3),
  aspect_ratio_range=(0.9, 1.1))

When resize_type='round_min_img_scale', we implemented ShortScaleAspectJitter, which rescales the image for its shorter side to reach the short_size and then jitters its aspect ratio, finally rescales the shape guaranteed to be divided by scale_divisor.

MMOCR 0.x Configuration

MMOCR 1.x Configuration

dict(
  type='ScaleAspectJitter',
  img_scale=[(3000, 640)],
  ratio_range=(0.7, 1.3),
  aspect_ratio_range=(0.9, 1.1),
  multiscale_mode='value',
  keep_ratio=False)

dict(
  type='ShortScaleAspectJitter',
  short_size=640,
  ratio_range=(0.7, 1.3),
  aspect_ratio_range=(0.9, 1.1),
  scale_divisor=32),