Realtime Number Plate Detection using Deep Learning

real-time-vehicle-or-car-number-plate-detection-using-deep-learning-and-python

In this tutorial, I will show you how to detect or recognize number pate in vehicles (especially cars) using deep learning in Python. To make this number plate detection system, I will use a deep learning model called YOLO v8.

Let me now break down the entire process of making vehicle number plate detection system into some steps:

Before we start, if you are new to computer vision I will suggest you to check out this Udemy course: Python for Computer Vision with OpenCV and Deep Learning.

Step1: Download Number Plate Dataset

For this tutorial, I am going to use Ainzyo ANPR dataset from roboflow. You can download that dataset from this link. This dataset has 7497 images of vehicles, especially for cars.

Since I will use YOLOv8 model for car number plate detection, I am going to download this dataset in YOLOv8 format. For the training purpose this comes with Train, Test and Validation set.

dataset-to-train-automatic-car-number-plate-detection-model

You need to download that dataset and change the name of the folder to “datasets” and then place it in your working directory. So now folder structure of your working directory should looks like below.

Woking directory
    ├── datasets
    	   ├── train
    	   |     ├── images
    	   |     └── labels
    	   ├── test
    	   |     ├── images
    	   |     └── labels
    	   ├── valid
    	   |     ├── images
    	   |     └── labels
    	   └── data.yaml
    

Step2: Setup Environment

Since we are going to use YOLO v8 model for our vehicle number plate detection model, we need to configure our system environment. It is always a good practice to create an isolated conda environment for this kind of project.

After creating fresh virtual environment, to set up entire your environment to train YOLO v8 model, please read this tutorial: Train YOLOv8 on Custom dataset in Windows GPU. It will take no more than 5 minutes to install all required libraries.

Step3: Train Model on Custom Dataset

Once you configured your environment variable, create a Python file or open a Jupyter notebook in your root working directory. In my case, I am creating a Jupyter notebook. So now my folder structure looks like below:

Woking directory
    ├── Untitled.ipynb
    ├── datasets
    	   ├── train
    	   |     ├── images
    	   |     └── labels
    	   ├── test
    	   |     ├── images
    	   |     └── labels
    	   ├── valid
    	   |     ├── images
    	   |     └── labels
    	   └── data.yaml
    

Now in your Jupyter notebook, first check whether all your configurations are correctly installed or not. First we will check ultralytics library is installed correctly or not.

from IPython import display
display.clear_output()

import ultralytics
ultralytics.checks()
Ultralytics YOLOv8.0.145  Python-3.7.12 torch-1.13.1 CUDA:0 (NVIDIA GeForce GTX 1050 Ti, 4096MiB)
Setup complete  (12 CPUs, 23.8 GB RAM, 237.5/914.6 GB disk)

As you can see in the output, ultralytics library is correctly installed and using GPU (CUDA) as its backend. So now we are ready to train our vehicle license plate detection model using YOLOv8. Below is the python code to do that.

%%time

import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'

from ultralytics import YOLO

# Load a model
model = YOLO('yolov8n.pt')  # load a pretrained model

# Train the model
model.train(data='datasets/data.yaml', epochs=30, imgsz=640, device=0)
Output
optimizer: AdamW(lr=0.002, momentum=0.9) with parameter groups 57 weight(decay=0.0), 64 weight(decay=0.0005), 63 bias(decay=0.0)
Image sizes 640 train, 640 val
Using 8 dataloader workers
Logging results to runs\detect\train
Starting training for 30 epochs...

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
       1/30      2.04G     0.9921      1.367      1.291         30        640: 100%|██████████| 351/351 [02:58<00:00,  
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 35/35 [00:12
                   all       1095       1203      0.877       0.88      0.936       0.69

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
       2/30      2.03G       1.03      0.911      1.306         30        640: 100%|██████████| 351/351 [02:48<00:00,  
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 35/35 [00:11
                   all       1095       1203      0.876       0.86      0.924       0.67

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
       3/30      2.05G      1.019     0.8174      1.307         31        640: 100%|██████████| 351/351 [02:46<00:00,  
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 35/35 [00:11
                   all       1095       1203      0.918      0.908      0.957      0.714

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
       4/30      2.04G     0.9855     0.7526       1.28         49        640: 100%|██████████| 351/351 [02:46<00:00,  
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 35/35 [00:11
                   all       1095       1203      0.921      0.914      0.952      0.703

      Epoch    GPU_mem   box_loss   cls_loss   dfl_loss  Instances       Size
       5/30      2.04G     0.9693     0.7039      1.266         42        640: 100%|██████████| 351/351 [02:45<00:00,  
                 Class     Images  Instances      Box(P          R      mAP50  mAP50-95): 100%|██████████| 35/35 [00:11
                   all       1095       1203       0.95      0.904      0.966      0.747
.....
.....
.....
.....

In this code line 3 & 4 is to avoid error this: OMP: Error #15: Initializing libiomp5md.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results.

Also Read:  Install OpenCV GPU with CUDA for Windows 10

I have used 30 epochs to train my vehicle detector YOLOv8 model, you can also try different numbers. If you have any issues understanding the above code, I will suggest you to first read below listed tutorials:

In my system (4 GB NVIDIA 1050 Ti graphics) it took around 1 hour to complete training for 30 epochs.

Once you successfully train YOLOv8 model for your custom dataset, a folder called “runs” will be created inside your working directory. best.pt is your final trained YOLOv8 model. So now your folder structure should looks like below:

Woking directory
	├── runs
	|    └── detect
	|         └── train
	|              ├── Some statistical images
	|              └── weights
	|                   ├── best.pt
        |	            └── last.pt
	|
        ├── datasets
        |	 ├── train
        |	 |     ├── images
        |	 |     └── labels
        |	 ├── test
        |	 |     ├── images
        |	 |     └── labels
        |	 ├── valid
        |	 |     ├── images
        |	 |     └── labels
        |	 └── data.yaml
	└── Untitled.ipynb

Step4: Inference on Image

Now we just need to call that best.pt custom trained YOLOv8 car detector model for a new image to see the output. Let’s do it in the below Python code.

from ultralytics import YOLO
 
# Load our custom number plate detection model
model = YOLO("runs/detect/train/weights/best.pt")
 
# Use the model to detect object - car number plate
model.predict(source="test_img1.jpg", save=True, show=True)
automatically-detect-or-recognize-car-license-plate-using-deep-learning-and-python

You can find output image inside runs -> detect -> predict folder. As you can see in the above output, our model is correctly detecting license plate from our input car image.

Step5: Realtime Inference on Video

This is fun part. Since we have successfully trained our vehicle detector YOLOv8 model and it is working correctly for input images. Let’s now implement this model to real-time video and see how our car number plate detection model is performing.

# For video
from ultralytics import YOLO
 
# Load custom trained YOLOv8 model
model = YOLO("runs/detect/train/weights/best.pt")
 
# Use the model to detect object
model.predict(source="test_video_4.mp4", show=True)

As you can see in the above output video, our realtime vehicle or car number plate or license plate detection model works smoothly with good FPS (frame per second).

Also Read:  Use Opencv with GPU with just 2 lines of code

Final Thought

In this tutorial, I made a simple real-time number plate detection system by training YOLOv8 deep learning model in Python. This project is just to show you how to detect number plates in a vehicle.

You can take this project to the next level by adding ocr or image to text to extract number from those detected plates. For that, you just need to crop those detected number plates and pass it to OCR (for example PyTesseract) after doing some morphological analysis.

Maybe I will cover that in my next tutorial but this is it for this tutorial, If you have any questions or suggestions regarding this tutorial, please let me know in the comment section below.

If you are new to computer vision I will suggest you to check out this Udemy course: Python for Computer Vision with OpenCV and Deep Learning.

Similar Read:

Leave a comment