Project-Based Learning (PBL) Project

Team members: Prabhat Kumar, Atul Panwar, Ankita Jaiswal

Assistant Professor/Project Guide: Yogita Sharma.

A model that will detect a car in a live stream or video and recognize characters on the number plate of the car.
Secondly, it will use the characters and fetch the owner's information using RTO APIs.
Steps
1. License Plate Detection
Let’s take a sample image of a car and start with detecting the License Plate on that car. We will then use the same image for Character Segmentation and Character Recognition as well. If you want to jump straight into the code without explanation then you can scroll down to the bottom of this page, where the complete code is provided. The test image that I am using for this tutorial is shown below.

image source link: https://rb.gy/lxmiuv

Step 1: Resize the image to the required size and then grayscale it. The code for the same is given below.

img = cv2.resize(img, (620,480) )
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) #convert to grey scale

Resizing help us to avoid any problems with bigger resolution images, make sure the number plate still remains in the frame after resizing. Gray scaling is common in all image processing steps. This speeds up other following processes sine we no longer have to deal with the color details when processing an image. The image would be transformed something like this when this step is done

Step 2: Every image will have useful and useless information, in this case for us only the license plate is the useful information the rest are pretty much useless for our program. This useless information is called noise. Normally using a bilateral filter (Blurring) will remove the unwanted details from an image. The code for the same is blurred.

gray = cv2.bilateralFilter(gray, 13, 15, 15)

Syntax is destination_image = cv2.bilateralFilter(source_image, diameter of pixel, sigmaColor, sigmaSpace). You can increase the sigma color and sigma space from 15 to higher values to blur out more background information, but be careful that the useful part does not get blurred. The output image is shown below, as you can see the background details (tree and building) are blurred in this image. This way we can avoid the program from concentrating on these regions later.

Step 3: The next step is interesting where we perform edge detection. There are many ways to do it, the most easy and popular way is to use the canny edge method from OpenCV. The line to do the same is shown below.

edged = cv2.Canny(gray, 30, 200) #Perform Edge detection

The syntax will be destination_image = cv2.Canny(source_image, threshold value 1, threshold value

2). The Threshold Vale 1 and Threshold Value 2 are the minima and maximum threshold values. Only the edges that have an intensity gradient more than the minimum threshold value and less than the maximum threshold value will be displayed. The resulting image is shown below.

Step 4: Now we can start looking for contours on our image

contours=cv2.findContours(edged.copy(),cv2.RETR_TREE,
cv2.CHAIN_APPROX_SIMPLE)
contours = imutils.grab_contours(contours)
contours = sorted(contours,key=cv2.contourArea, reverse = True)[:10]
screenCnt = None

Once the counters have been detected we sort them from big to small and consider only the first 10 results ignoring the others. In our image, the counter could be anything that has a closed surface but of all the obtained results the license plate number will also be there since it is also a closed surface.
To filter the license plate image among the obtained results, we will loop through all the results and check which has a rectangle shape contour with four sides and a closed figure. Since a license plate would definitely be a rectangle four-sided figure.

for c in cnts:
# approximate the contour
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.018 * peri, True)
# if our approximated contour has four points, then
# we can assume that we have found our screen
if len(approx) == 4:
screenCnt = approx
break

Once we have found the right counter we save it in a variable called screenCnt and then draw a rectangle box around it to make sure we have detected the license plate correctly.

Step 5: Now that we know where the number plate is, the remaining information is pretty much useless for us. So we can proceed with masking the entire picture except for the place where the number plate is. The code to do the same is shown below

# Masking the part other than the number plate
mask = np.zeros(gray.shape,np.uint8)
new_image = cv2.drawContours(mask,[screenCnt],0,255,-1,)
new_image = cv2.bitwise_and(img,img,mask=mask)

The masked new image will appear something like below.

2. Character Segmentation
The next step in Number Plate Recognition is to segment the license plate out of the image by cropping it and saving it as a new image. We can then use this image to detect the character in it. The code to crop the roi (Region of interest) image form the main image is shown below

# Now crop
(x, y) = np.where(mask == 255)
(topx, topy) = (np.min(x), np.min(y))
(bottomx, bottomy) = (np.max(x), np.max(y))
Cropped = gray[topx:bottomx+1, topy:bottomy+1]

The resulting image is shown below. Normally added to cropping the image, we can also gray it and edge it if required. This is done to improve the character recognition in the next step. However, I found that it works fine even with the original image.

3. Character Recognition
The final step in this Number Plate Recognition is to actually read the number plate information from the segmented image. We will use the pytesseract package to read characters from images, just like we did in the previous tutorial. The code for the same is given below.

#Read the number plate
text = pytesseract.image_to_string(Cropped, config=’ — psm 11’)
print(“Detected license plate Number is:”,text)

As you can see the original image had the number “CZ20FSE” on it and our program has detected it printed the same value on the Jupiter notebook.
Fail Cases in Number Plate Recognition
The complete code of this License Plate Recognition can be downloaded from here, it contains the program and the test images that we used to check our program. Without being said, it is to be remembered that the results from this method will not be accurate. The accuracy depends on the clarity of the image, orientation, light exposure, etc. To get better results you can try implementing Machine learning algorithms along with this.

As you can see, our program was able to detect the license plate correctly and crop it. But the Tesseract library has failed to recognize the characters properly. Instead of the actual “MH 13 CD 0096” the OCR has recognized it to be “MH13CD 0036”. Problems like this can be corrected by either using better orientation images or by configuring the Tesseract engine.
Other Successful Examples
Most of the time of the image quality and orientation are correct, the program was able to identify the license plate and read the number from it. The below snapshots show the successful results obtained.