Social distancing detector using OpenCV and Python

In this tutorial, I will show you how to implement a COVID-19 social distancing detector using OpenCV, Computer Vision and Deep Learning.

Here, I’m going to give you a starting point for your own social distancing detector. You can then modify it to develop your own projects may be you can make some advanced social distance detector.

Before reading this tutorial I will highly recommend you to read below articles. You should have your basics clear.

Must Read:

Approach to develop Social distance detection

I will break this whole project into three codes:

  1. Save an image frame from video footage
  2. Convert actual distance to pixels apart
  3. Detect person and find distance between them

Save an image frame from video footage

For this tutorial I am using CCTV video footage of Airport. You can find the video link below which I have used in this tutorial.

Input video for this tutorial

Now reason to save a image frame from this video is is related to the next step (to convert actual distance to pixels apart).

# Import opencv library
import cv2

# Read video frames
cap = cv2.VideoCapture('data/cctv.mp4')

while True:
    # Read video frame by frame
    read_ok, img = cap.read()
    cv2.imshow("Image", img)

    # Save last frame of the video as image using OpenCV
    cv2.imwrite("last_frame.jpg", img)

    # Close video window by pressing 'x'
    if cv2.waitKey(1) & 0xFF == ord('x'):
        break
How to save a video frame as image

Convert actual distance to pixels apart

In OpenCV you can only calculate how many pixels are there (pixels apart) in between two objects or person. Now to convert this pixels apart to actual distance, you need to have a reference object. We must know geometric value (like height, width, area etc.) of that reference object.

Let’s say our reference object in this video (or image frame) is tiles. We know height and width of each tile.

  • Width of a tile: 3 feet
  • Height of a tile: 6 feet
reference object for Social distancing detector using OpenCV and Python
Hight and Width of a tile

Now let’s say we want to make a social distancing detector which can mark or detect those persons if distance between those person is less than 9 feet.

Also Read:  Warp perspective and Transform OpenCV python

To do that we have to know amount of pixel difference for 9 feet in our video frame. In below code we are doing exactly same thing. We are converting actual distance to pixel distance.

In this code we will use output (last_frame.jpg) of previous step as input image.

import cv2
import numpy as np
import math

# Create point matrix get coordinates of mouse click on image
point_matrix = np.zeros((2,2),np.int)

counter = 0
# Function to store coordinate of mouse click in image
def mousePoints(event,x,y,flags,params):
    global counter
    # Left button click
    if event == cv2.EVENT_LBUTTONDOWN:
        point_matrix[counter] = x,y
        counter = counter + 1

# Function for Euclidean Distance between two points
def calculateDistance(x1,y1,x2,y2):
    dist = math.sqrt((x2 - x1)**2 + (y2 - y1)**2)
    return dist

# Loading image which was saved in previous step
img = cv2.imread("last_frame.jpg")

while True:
    for x in range (0,2):
        cv2.circle(img,(point_matrix[x][0],point_matrix[x][1]),3,(0,0,255),cv2.FILLED)

    # Stop when we have two mouse click
    if counter == 2:
        starting_x = point_matrix[0][0]
        starting_y = point_matrix[0][1]

        ending_x = point_matrix[1][0]
        ending_y = point_matrix[1][1]

        # Draw line between two mouse clicked points
        cv2.line(img, (starting_x, starting_y), (ending_x, ending_y), (0, 255, 0), thickness=2)

        # Show length of the line on image
        line_length = calculateDistance(starting_x, starting_y, ending_x, ending_y)

        # Original width of a tile is 2 feet
        # So width of three tiles will be 6 feet in total
        font = cv2.FONT_HERSHEY_DUPLEX
        blue_color = (255,0,0)
        cv2.putText(img, f'{"Pixel Distance: ", round(line_length, 2)}', (starting_x - 25, starting_y + 70), font, 1, blue_color, 2)
        cv2.putText(img, f'{"Original: ", "6 ft"}', (starting_x - 25, starting_y + 100), font, 1, blue_color, 2)

    # Showing original image
    cv2.imshow("Original Image ", img)
    # Mouse click event on original image
    cv2.setMouseCallback("Original Image ", mousePoints)
    # Refreshing window all time
    cv2.waitKey(1)

In the above code we have used mouse click event to select two points and find pixel distance between two points by Euclidean distance. Since width of one tile is 3 feet, we need to know pixel difference between 3 tiles which is 9 feet.

Convert actual distance to pixel diference for Social distancing detector using OpenCV and Python
Width of 3 tiles is 9 feet

As you can see 9 feet (width of 3 tiles) is equal to 217 pixel for our video frame. It varies image to image. For the video you are working it may be different.

Detect person and find distance between them

In my last post I have shown you how to use pretrained yolo model to detect object. I am going to use same code for this social distance detection.

Also Read:  Realtime Number Plate Detection using Deep Learning

Must Read:

import cv2
import numpy as np

from itertools import combinations
import math

# Euclidean Distance between two points
def calculateDistance(x1,y1,x2,y2):
    dist = math.sqrt((x2 - x1)**2 + (y2 - y1)**2)
    return dist

# Load Yolo
yolo_weight = "data/model/yolov3.weights"
yolo_config = "data/model/yolov3.cfg"
coco_labels = "data/model/coco.names"
net = cv2.dnn.readNet(yolo_weight, yolo_config)

net.setPreferableBackend(cv2.dnn.DNN_BACKEND_CUDA)
net.setPreferableTarget(cv2.dnn.DNN_TARGET_CUDA)

classes = []
with open(coco_labels, "r") as f:
    classes = [line.strip() for line in f.readlines()]
layer_names = net.getLayerNames()
output_layers = [layer_names[i[0] - 1] for i in net.getUnconnectedOutLayers()]
colors = np.random.uniform(0, 255, size=(len(classes), 3))

# Defining desired shape
fWidth = 256
fHeight = 256

# Below function will read video frames
cap = cv2.VideoCapture('data/cctv.mp4')

while True:
    read_ok, img = cap.read()

    height, width, channels = img.shape

    # Detecting objects
    blob = cv2.dnn.blobFromImage(img, 0.00392, (416, 416), (0, 0, 0), True, crop=False)
    net.setInput(blob)
    outs = net.forward(output_layers)

    # Showing information on the screen
    class_ids = []
    confidences = []
    boxes = []
    center_points = []
    for out in outs:
        for detection in out:
            scores = detection[5:]
            class_id = np.argmax(scores)
            confidence = scores[class_id]

            # class_id = 0 means we will only detect persons from video
            if confidence > 0.5 and class_id == 0:
                # Object detected
                center_x = int(detection[0] * width)
                center_y = int(detection[1] * height)
                w = int(detection[2] * width)
                h = int(detection[3] * height)
                # Rectangle coordinates
                x = int(center_x - w / 2)
                y = int(center_y - h / 2)
                boxes.append([x, y, w, h, center_x, center_y])
                center_points.append([center_x, center_y])
                confidences.append(float(confidence))
                class_ids.append(class_id)

    indexes = cv2.dnn.NMSBoxes(boxes, confidences, 0.5, 0.4)

    # Create combination list of center points between each detected person bounding box
    combination_points = list(combinations(center_points, 2))

    font = cv2.FONT_HERSHEY_DUPLEX
    for i in range(len(boxes)):
        if i in indexes:
            x, y, w, h, box_center_x, box_center_y = boxes[i]

            color = colors[i]
            cv2.circle(img, (center_x, center_y), 3, (0, 0, 255), cv2.FILLED)

            for points in combination_points:
                # Find Distance between two person (pixel distance / apart)
                center_x, center_y = points[0]
                prev_center_x, prev_center_y = points[1]
                euclidean_distance = calculateDistance(center_x, center_y, prev_center_x, prev_center_y)

                # Width of three tiles = 217 pixel , which is 9 feet
                width_of_3_tiles = 217

                # Mark person bounding box as red is distance is less than 9 feet
                if euclidean_distance < width_of_3_tiles and euclidean_distance > 150:
                    if box_center_x == center_x or box_center_y == center_y:
                        # Draw rectangle for each person
                        cv2.rectangle(img, (x, y), (x + w, y + h), (0, 0, 255), 2)
                        # Draw line between each person
                        cv2.line(img, (center_x, center_y), (prev_center_x, prev_center_y), (0, 255, 0), thickness=2)


    cv2.imshow("Image", img)
    # Close video window by pressing 'x'
    if cv2.waitKey(1) & 0xFF == ord('x'):
        break

In this code at:

  • line 57: We will use class_id == 0, as we only want to detect person. In coco.names file all classes are listed. Among those classes person class appears at first position, so id of person class is 0. Similarly id of bicycle is 1 and car is 2 etc.
  • line 66: In this line we are storing coordinates (x, y) of bounding box for any detected person with center point of that bounding box (for any detected person)
  • line 77-88: Calculate distance between center point (of each detected person on the screen) based on Euclidean distance.
  • line 91: I have mention the pixel difference = 217 (which we got from previous step.), which is equal to 9 feet.
  • line 93-99: If distance is less that 217 pixel difference of less than 9 feet, then make color of those bounding box (of two closer person) to red and also join those center points by a line.
Also Read:  Clip Raster with a Shape file in Python

Conclusion

In this Social distancing detector tutorial you learned:

  • How to save a video frame as image
  • Convert actual distance to pixel apart
  • Find distance between two objects
  • By combining all, make a Social distance detector

If you have any question or suggestion regarding this post, let me know in the comment section below.

35 thoughts on “Social distancing detector using OpenCV and Python”

  1. I am going to use social distancing detector using Opencv and python in my final year project. Your post helped me a lot to start my journey.

    Reply

Leave a comment