How to Optimize QR Recognition Performance by Image Preprocessing and Parameter Tuning

When using a Barcode reader or scanner SDK, we primarily focus on the detection speed and recognition accuracy. There are many factors that may affect the performance of the Barcode detection SDK. The factors can be approximately classified into two categories: image quality and algorithm performance. In this article, we take Dynamsoft Barcode Reader Python edition as the example to discuss how to optimize the performance of the Barcode detection SDK by preprocessing input QR code images and adjusting the algorithm parameters on desktop platforms.

Prerequisites

• OpenCV Python
  

  pip install opencv-python
  pip install opencv-contrib-python
  

• Dynamsoft Barcode Reader SDK
  

  pip install dbr
  

• Download some QR code images by Google Image Search

Preprocessing QR Images

To evaluate the SDK performance, we use color image, grayscale image, and binary image as the input images.

Color Image

Let's get started with a color image of a QR code.

The following code loads and displays the image using OpenCV and reads barcode information using Dynamsoft Barcode Reader.

import cv2
import numpy as np
import dbr
import time

reader = dbr.BarcodeReader()
reader.init_license("LICENSE-KEY") # https://www.dynamsoft.com/customer/license/trialLicense?product=dbr
image = cv2.imread('Air-Force-Reserve-San-Diego.jpg')

def detect(windowName, image, pixel_format):
    try:
        buffer = image.tobytes()
        height = image.shape[0]
        width = image.shape[1]
        stride = image.strides[0]
        start = time.time()
        results = reader.decode_buffer_manually(buffer, width, height, stride, pixel_format, "")
        end = time.time()
        print("Time taken: {:.3f}".format(end - start) + " seconds")
        cv2.putText(image, "Time taken: {:.3f}".format(end - start) + " seconds", (10, 30), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        if results != None:
            for result in results:
                print("Barcode Format : ")
                print(result.barcode_format_string)
                print("Barcode Text : ")
                print(result.barcode_text)

                points = result.localization_result.localization_points
                data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
                cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

                x = min(points[0][0], points[1][0], points[2][0], points[3][0])
                y = min(points[0][1], points[1][1], points[2][1], points[3][1])
                cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

        cv2.imshow(windowName, image)
    except BarcodeReaderError as bre:
        print(bre)

detect('Color', image, dbr.EnumImagePixelFormat.IPF_RGB_888)
cv2.waitKey(0)

We initialize Dynamsoft Barcode Reader with the license key.

reader = dbr.BarcodeReader()
reader.init_license("LICENSE-KEY")

By default, OpenCV loads images in BGR format. The corresponding enum value of pixel format provided by Dynamsoft Barcode Reader is dbr.EnumImagePixelFormat.IPF_RGB_888.

Then we call the decode_buffer_manually function to decode the image.

buffer = image.tobytes()
height = image.shape[0]
width = image.shape[1]
stride = image.strides[0]
results = reader.decode_buffer_manually(buffer, width, height, stride, dbr.EnumImagePixelFormat.IPF_RGB_888, "")

The function returns a list of BarcodeResult objects. Each BarcodeResult object contains the barcode information and the localization result. The drawContours function and putText function can be used to draw the barcode information and localization result on the image.

points = result.localization_result.localization_points
data = np.array([[points[0][0], points[0][1]], [points[1][0], points[1][1]], [points[2][0], points[2][1]], [points[3][0], points[3][1]]])
cv2.drawContours(image=image, contours=[data], contourIdx=-1, color=(0, 255, 0), thickness=2, lineType=cv2.LINE_AA)

x = min(points[0][0], points[1][0], points[2][0], points[3][0])
y = min(points[0][1], points[1][1], points[2][1], points[3][1])
cv2.putText(image, result.barcode_text, (x, y), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)

Grayscale Image

Grayscale image only carries intensity information. The barcode recognition API internally converts a color image to a grayscale image. So
theoretically, if the input image is a grayscale image, the recognition speed is faster.

We run the following code to see the time cost:

grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
detect('Grayscale', grayscale_image, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)

The pixel format is changed to dbr.EnumImagePixelFormat.IPF_GRAYSCALED accordingly. As expected, decoding the grayscale image converted from the original color image saves a little time.

Binary Image

Thresholding the grayscale image to binary image can make it easier to analyze. We can also try it out by using the OpenCV function threshold:

# https://docs.opencv.org/4.5.1/d7/d4d/tutorial_py_thresholding.html
blur = cv2.GaussianBlur(grayscale_image,(5,5),0)
ret, thresh = cv2.threshold(blur, 0,255,cv2.THRESH_BINARY + cv2.THRESH_OTSU)
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)

We use Gaussian blur to reduce noise and smooth the image, and then use the OpenCV function threshold to convert the grayscale image to binary image. The Otsu's method is used to automatically determine the optimal global threshold value from the image histogram.

More Tests

Let's take a look at some complicated cases.

Perspective Distortion

Here is a perspective distorted QR code image.

If we use the code above, there is no barcode recognized, no matter color image, grayscale image or binary image. However, it does not mean the barcode image cannot be recognized. I managed to recognize the QR code by manually adjusting and setting the threshold value:

image = cv2.imread('perspective-distortion-qr-code.jpg')
grayscale_image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(grayscale_image, 125, 255,cv2.THRESH_BINARY) 
detect('Binary', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)

Time taken: 0.0080 seconds
Barcode Format :
QR_CODE
Barcode Text :
http://goo.by/wb0Ric

Inverted Color

If we try to read the following QR code image printed with inverted color, it will fail to do the recognition.

After converting the image to binary image, we can see the black and white are inverted for QR code:

To make the QR code recognizable, we use OpenCV function bitwise_not:

ret, thresh = cv2.threshold(grayscale_image, 150,255,cv2.THRESH_BINARY )
cv2.bitwise_not(thresh, thresh) 
detect('Inverted', thresh, dbr.EnumImagePixelFormat.IPF_GRAYSCALED)

Time taken: 0.0541 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/

Adjusting the Algorithm Parameters

So far, we have tried some image preprocessing ways to read QR code. Actually, the barcode SDK contains more image processing and computer vision algorithms than the ones we have tried. To cover all cases, we need to know how to tune the parameters of the barcode SDK.

We use the following Python code to print the runtime parameters in console:

print(reader.get_runtime_settings().__dict__)

{'terminate_phase': 32, 'timeout': 10000, 'max_algorithm_thread_count': 4, 'expected_barcodes_count': 0, 'barcode_format_ids': -31457281, 'barcode_format_ids_2': 0, 'pdf_raster_dpi': 300, 'scale_down_threshold': 2300, 'binarization_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'localization_modes': [2, 16, 4, 8, 0, 0, 0, 0], 'colour_clustering_modes': [0, 0, 0, 0, 0, 0, 0, 0], 
'colour_conversion_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'grayscale_transformation_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'region_predetection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'image_preprocessing_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'texture_detection_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'text_filter_modes': [2, 0, 0, 0, 0, 0, 0, 0], 'dpm_code_reading_modes': [0, 0, 0, 0, 0, 0, 
0, 0], 'deformation_resisting_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_complement_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'barcode_colour_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'text_result_order_modes': [1, 2, 4, 0, 0, 0, 0, 0], 'text_assisted_correction_mode': 2, 'deblur_level': 9, 'intermediate_result_types': 0, 'intermediate_result_saving_mode': 1, 'result_coordinate_type': 1, 'return_barcode_zone_clarity': 0, 'region_top': 0, 'region_bottom': 0, 'region_left': 0, 'region_right': 0, 'region_measured_by_percentage': 0, 'min_barcode_text_length': 0, 'min_result_confidence': 0, 'scale_up_modes': [1, 0, 0, 0, 0, 0, 0, 0], 'accompanying_text_recognition_modes': [0, 0, 0, 0, 0, 0, 0, 0], 'pdf_reading_mode': 1, 'deblur_modes': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'barcode_zone_min_distance_to_image_borders': 0}

There are quite a lot of parameters to tune, but you cannot find a crash course on how to tune these parameters. Fortunately, we can leverage the parameter tuning tool of Dynamsoft Barcode Reader online demo to customize parameters for specific images.

There are five modes available for selection. You can click Advanced Settings to expand parameters of a mode. For better studying the parameter differences among different modes, we can scroll down the page and save all parameters to a json file.

There is a tradeoff between the barcode decoding speed and recognition accuracy. The more algorithms are enabled for barcode recognition, the more time it takes.

Best coverage vs. Best speed

Now, we upload the perspective distorted QR image to see the difference between the best speed and best coverage.

Best speed
It takes 15 milliseconds but fails to recognize the QR code.

Best coverage
It successfully recognizes the QR code but takes 828 milliseconds.

Both of them are not ideal according to our test in Python code. To guarantee the best performance:

1. Go to Advanced Settings > Binarization mode > Threshold.
2. Change the threshold value to 125.

Now the QR code is recognized extremely fast.

We can then export the template to a JSON file and load it in our Python code:

error = reader.init_runtime_settings_with_file('perspective-qr.json')
image = cv2.imread('perspective-distortion-qr-code.jpg')
detect('Color', image_copy, dbr.EnumImagePixelFormat.IPF_RGB_888)

The result is satisfying.

Time taken: 0.0080 seconds
Barcode Format :
QR_CODE
Barcode Text :
http://goo.by/wb0Ric

As for the inverted QR code, the setting is as follows:

The performance is even better when reading the inverted QR code in Python.

Time taken: 0.0160 seconds
Barcode Format : 
QR_CODE
Barcode Text :
https://www.jalna.com.au/

In a nutshell, to get best performance of Dynamsoft Barcode Reader SDK, you should start with the simplest settings and then gradually increase and tune algorithm parameters.

Source Code