In OpenMMLab, all the inference operations are unified into a new interface - Inferencer. Inferencer is designed to expose a neat and simple API to users, and shares very similar interface across different OpenMMLab libraries.

In MMOCR, Inferencers are constructed in different levels of task abstraction.

  • Standard Inferencer: Following OpenMMLab’s convention, each fundamental task in MMOCR has a standard Inferencer, namely TextDetInferencer (text detection), TextRecInferencer (text recognition), TextSpottingInferencer (end-to-end OCR), and KIEInferencer (key information extraction). They are designed to perform inference on a single task, and can be chained together to perform inference on a series of tasks. They also share very similar interface, have standard input/output protocol, and overall follow the OpenMMLab design.

  • MMOCRInferencer: We also provide MMOCRInferencer, a convenient inference interface only designed for MMOCR. It encapsulates and chains all the Inferencers in MMOCR, so users can use this Inferencer to perform a series of tasks on an image and directly get the final result in an end-to-end manner. However, it has a relatively different interface from other standard Inferencers, and some of standard Inferencer functionalities might be sacrificed for the sake of simplicity.

For new users, we recommend using MMOCRInferencer to test out different combinations of models.

If you are a developer and wish to integrate the models into your own project, we recommend using standard Inferencers, as they are more flexible and standardized, equipped with full functionalities.

Basic Usage

As of now, MMOCRInferencer can perform inference on the following tasks:

  • Text detection

  • Text recognition

  • OCR (text detection + text recognition)

  • Key information extraction (text detection + text recognition + key information extraction)

  • OCR (text spotting) (coming soon)

For convenience, MMOCRInferencer provides both Python and command line interfaces. For example, if you want to perform OCR inference on demo/demo_text_ocr.jpg with DBNet as the text detection model and CRNN as the text recognition model, you can simply run the following command:

>>> from mmocr.apis import MMOCRInferencer
>>> # Load models into memory
>>> ocr = MMOCRInferencer(det='DBNet', rec='SAR')
>>> # Perform inference
>>> ocr('demo/demo_text_ocr.jpg', show=True)

The resulting OCR output will be displayed in a new window:


If you are running MMOCR on a server without GUI or via SSH tunnel with X11 forwarding disabled, the show option will not work. However, you can still save visualizations to files by setting out_dir and save_vis=True arguments. Read Dumping Results for details.

Depending on the initialization arguments, MMOCRInferencer can run in different modes. For example, it can run in KIE mode if it is initialized with det, rec and kie specified.

>>> kie = MMOCRInferencer(det='DBNet', rec='SAR', kie='SDMGR')
>>> kie('demo/demo_kie.jpeg', show=True)

The output image should look like this:

You may have found that the Python interface and the command line interface of MMOCRInferencer are very similar. The following sections will use the Python interface as an example to introduce the usage of MMOCRInferencer. For more information about the command line interface, please refer to Command Line Interface.


Each Inferencer must be initialized with a model. You can also choose the inference device during initialization.

Model Initialization

For each task, MMOCRInferencer takes two arguments in the form of xxx and xxx_weights (e.g. det and det_weights) for initialization, and there are many ways to initialize a model for inference. We will take det and det_weights as an example to illustrate some typical ways to initialize a model.

  • To infer with MMOCR’s pre-trained model, passing its name to the argument det can work. The weights will be automatically downloaded and loaded from OpenMMLab’s model zoo. Check Weights for available model names.

    >>> MMOCRInferencer(det='DBNet')
  • To load custom config and weight, you can pass the path to the config file to det and the path to the weight to det_weights.

    >>> MMOCRInferencer(det='path/to/', det_weights='path/to/dbnet.pth')

You may click on the “Standard Inferencer” tab to find more initialization methods.


Each Inferencer instance is bound to a device. By default, the best device is automatically decided by MMEngine. You can also alter the device by specifying the device argument. For example, you can use the following code to create an Inferencer on GPU 1.

>>> inferencer = MMOCRInferencer(det='DBNet', device='cuda:1')

To create an Inferencer on CPU:

>>> inferencer = MMOCRInferencer(det='DBNet', device='cpu')

Refer to torch.device for all the supported forms.


Once the Inferencer is initialized, you can directly pass in the raw data to be inferred and get the inference results from return values.


Input can be either of these types:

  • str: Path/URL to the image.

    >>> inferencer('demo/demo_text_ocr.jpg')
  • array: Image in numpy array. It should be in BGR order.

    >>> import mmcv
    >>> array = mmcv.imread('demo/demo_text_ocr.jpg')
    >>> inferencer(array)
  • list: A list of basic types above. Each element in the list will be processed separately.

    >>> inferencer(['img_1.jpg', 'img_2.jpg])
    >>> # You can even mix the types
    >>> inferencer(['img_1.jpg', array])
  • str: Path to the directory. All images in the directory will be processed.

    >>> inferencer('tests/data/det_toy_dataset/imgs/test/')


By default, each Inferencer returns the prediction results in a dictionary format.

  • visualization contains the visualized predictions. But it’s an empty list by default unless return_vis=True.

  • predictions contains the predictions results in a json-serializable format. As presented below, the contents are slightly different depending on the task type.

        'predictions' : [
          # Each instance corresponds to an input image
            'det_polygons': [...],  # 2d list of length (N,), format: [x1, y1, x2, y2, ...]
            'det_scores': [...],  # float list of length (N,)
            'det_bboxes': [...],   # 2d list of shape (N, 4), format: [min_x, min_y, max_x, max_y]
            'rec_texts': [...],  # str list of length (N,)
            'rec_scores': [...],  # float list of length (N,)
            'kie_labels': [...],  # node labels, length (N, )
            'kie_scores': [...],  # node scores, length (N, )
            'kie_edge_scores': [...],  # edge scores, shape (N, N)
            'kie_edge_labels': [...]  # edge labels, shape (N, N)
        'visualization' : [
          array(..., dtype=uint8),

If you wish to get the raw outputs from the model, you can set return_datasamples to True to get the original DataSample, which will be stored in predictions.

Dumping Results

Apart from obtaining predictions from the return value, you can also export the predictions/visualizations to files by setting out_dir and save_pred/save_vis arguments.

>>> inferencer('img_1.jpg', out_dir='outputs/', save_pred=True, save_vis=True)

Results in the directory structure like:

├── preds
│   └── img_1.json
└── vis
    └── img_1.jpg

The filename of each file is the same as the corresponding input image filename. If the input image is an array, the filename will be a number starting from 0.

Batch Inference

You can customize the batch size by setting batch_size. The default batch size is 1.


Here are extensive lists of parameters that you can use.


Arguments Type Default Description
det str or Weights, optional None Pretrained text detection algorithm. It's the path to the config file or the model name defined in metafile.
det_weights str, optional None Path to the custom checkpoint file of the selected det model. If it is not specified and "det" is a model name of metafile, the weights will be loaded from metafile.
rec str or Weights, optional None Pretrained text recognition algorithm. It’s the path to the config file or the model name defined in metafile.
rec_weights str, optional None Path to the custom checkpoint file of the selected rec model. If it is not specified and “rec” is a model name of metafile, the weights will be loaded from metafile.
kie [1] str or Weights, optional None Pretrained key information extraction algorithm. It’s the path to the config file or the model name defined in metafile.
kie_weights str, optional None Path to the custom checkpoint file of the selected kie model. If it is not specified and “kie” is a model name of metafile, the weights will be loaded from metafile.
device str, optional None Device used for inference, accepting all allowed strings by torch.device. E.g., 'cuda:0' or 'cpu'. If None, the available device will be automatically used. Defaults to None.

[1]: kie is only effective when both text detection and recognition models are specified.


Arguments Type Default Description
inputs str/list/tuple/np.array required It can be a path to an image/a folder, an np array or a list/tuple (with img paths or np arrays)
return_datasamples bool False Whether to return results as DataSamples. If False, the results will be packed into a dict.
batch_size int 1 Inference batch size.
det_batch_size int, optional None Inference batch size for text detection model. Overwrite batch_size if it is not None.
rec_batch_size int, optional None Inference batch size for text recognition model. Overwrite batch_size if it is not None.
kie_batch_size int, optional None Inference batch size for KIE model. Overwrite batch_size if it is not None.
return_vis bool False Whether to return the visualization result.
print_result bool False Whether to print the inference result to the console.
show bool False Whether to display the visualization results in a popup window.
wait_time float 0 The interval of show(s).
out_dir str results/ Output directory of results.
save_vis bool False Whether to save the visualization results to out_dir.
save_pred bool False Whether to save the inference results to out_dir.

Command Line Interface


This section is only applicable to MMOCRInferencer.

You can use tools/ to perform inference through MMOCRInferencer. Its general usage is as follows:

python tools/ INPUT_PATH [--det DET] [--det-weights ...] ...

where INPUT_PATH is a required field, which should be a path to an image or a folder. Command-line parameters follow the mapping relationship with the Python interface parameters as follows:

  • To convert the Python interface parameters to the command line ones, you need to add two -- in front of the Python interface parameters, and replace the underscore _ with the hyphen -. For example, out_dir becomes --out-dir.

  • For boolean type parameters, putting the parameter in the command is equivalent to specifying it as True. For example, --show will specify the show parameter as True.

In addition, the command line will not display the inference result by default. You can use the --print-result parameter to view the inference result.

Here is an example:

python tools/ demo/demo_text_ocr.jpg --det DBNet --rec SAR --show --print-result

Running this command will give the following result:

{'predictions': [{'rec_texts': ['CBank', 'Docbcba', 'GROUP', 'MAUN', 'CROBINSONS', 'AOCOC', '916M3', 'BOO9', 'Oven', 'BRANDS', 'ARETAIL', '14', '70<UKN>S', 'ROUND', 'SALE', 'YEAR', 'ALLY', 'SALE', 'SALE'],
'rec_scores': [0.9753464579582214, ...], 'det_polygons': [[551.9930285844646, 411.9138765335083, 553.6153911653112,
383.53195309638977, 620.2410061195247, 387.33785033226013, 618.6186435386782, 415.71977376937866], ...], 'det_scores': [0.8230461478233337, ...]}]}
Read the Docs v: dev-1.x
On Read the Docs
Project Home

Free document hosting provided by Read the Docs.