Object recognition with Python

Tags: Artificial Intelligence, Python, Vision

, ,

In this tutorial, we’ll look at how to perform object recognition with Python, using a neural network pre-trained with deep learning.

We saw in a previors tutorial how to recognize simple shapes using computer vision. This method only works for certain predefined simple shapes. If yor want to recognize a wider variety of objects, the easiest way is to use artificial intelligence.

Hardware

A computer with a Python3 installation
A camera

Principle

Artificial intelligence is a field of computer science in which the program itself learns to perform certain tasks. Visual recognition in particular. In this tutorial, we’ll use a trained neural network to recognize particular shapes.

You need a lot of data to train a neural network properly. It has been shown that learning is faster on a neural network trained for something else. For example, a neural network trained to recognize dogs will train more easily to recognize cats.

Python configuration

If this is not the case, you can download and install Python 3

You can then install the necessary libraries OpenCV, numpy and imutils

pip3 install opencv-python numpy imutils

python3 -m pip install opencv-python numpy imutils

Download ModelNet-SSD

file prototxt : https://github.com/nikmart/pi-object-detection/blob/master/MobileNetSSD_deploy.prototxt.txt
file caffemodel : https://github.com/nikmart/pi-object-detection/blob/master/MobileNetSSD_deploy.caffemodel

Place the model files in a folder and create the file ObjectRecognition.py

object-recognition-directory Object recognition with Python

Python script for Object recognition

First, we create a video stream (vs) using the imutils library, which will retrieve the images from the camera.

vs = VideoStream(src=0, resolution=(1600, 1200)).start()

We initialise a neural network with the ModelNet-SSD (net) parameters using the OpenCV library.

net = cv2.dnn.readNetFromCaffe(args["prototxt"], args["model"])

We will then create a loop which, at each iteration, will read the camera image and pass it to the input of the neural network for object detection and recognition.

while True:
	# Get video stream. max width 800 pixels 
	frame = vs.read()
	blob = cv2.dnn.blobFromImage(cv2.resize(frame, (300, 300)), 0.007843, (300, 300), 127.5)

	# Feed input to neural network 
	net.setInput(blob)
	detections = net.forward()

Finally, the code displays the detection box and the probability of recognition on the image.

label = "{}: {:.2f}%".format(CLASSES[idx],confidence * 100)
cv2.rectangle(frame, (startX, startY), (endX, endY),COLORS[idx], 2)
y = startY - 15 if startY - 15 > 15 else startY + 15
cv2.putText(frame, label, (startX, y),cv2.FONT_HERSHEY_SIMPLEX, 0.5, COLORS[idx], 2)

#!/usr/bin/env python
# -*- coding: utf-8 -*-
#
#  ObjectRecognition.py
#  Description:
#		Use ModelNet-SSD model to detect objects
#
#  www.aranacorp.com

# import packages
import sys
from imutils.video import VideoStream
from imutils.video import FPS
import numpy as np
import argparse
import imutils
import time
import cv2

# Arguments construction
if len(sys.argv)==1:
	args={
	"prototxt":"MobileNetSSD_deploy.prototxt.txt",
	"model":"MobileNetSSD_deploy.caffemodel",
	"confidence":0.2,
	}
else:
	#lancement à partir du terminal
	#python3 ObjectRecognition.py --prototxt MobileNetSSD_deploy.prototxt.txt --model MobileNetSSD_deploy.caffemodel
	ap = argparse.ArgumentParser()
	ap.add_argument("-p", "--prototxt", required=True,
		help="path to Caffe 'deploy' prototxt file")
	ap.add_argument("-m", "--model", required=True,
		help="path to Caffe pre-trained model")
	ap.add_argument("-c", "--confidence", type=float, default=0.2,
		help="minimum probability to filter weak detections")
	args = vars(ap.parse_args())



# ModelNet SSD Object list init
CLASSES = ["arriere-plan", "avion", "velo", "oiseau", "bateau",
	"borteille", "autobus", "voiture", "chat", "chaise", "vache", "table",
	"chien", "cheval", "moto", "personne", "plante en pot", "morton",
	"sofa", "train", "moniteur"]
COLORS = np.random.uniform(0, 255, size=(len(CLASSES), 3))

# Load model file
print("Load Neural Network...")
net = cv2.dnn.readNetFromCaffe(args["prototxt"], args["model"])

if __name__ == '__main__':

	# Camera initialisation
	print("Start Camera...")
	vs = VideoStream(src=0, resolution=(1600, 1200)).start()
	#vs = VideoStream(usePiCamera=True, resolution=(1600, 1200)).start()
	#vc = cv2.VideoCapture('./img/Splash - 23011.mp4') #from video

	time.sleep(2.0)
	fps = FPS().start()
	
	#Main loop
	while True:
		# Get video sttream. max width 800 pixels 
		frame = vs.read()
		#frame= cv2.imread('./img/two-boats.jpg') #from image file
		#ret, frame=vc.read() #from video or ip cam
		frame = imutils.resize(frame, width=800)

		# Create blob from image
		(h, w) = frame.shape[:2]
		blob = cv2.dnn.blobFromImage(cv2.resize(frame, (300, 300)), 0.007843, (300, 300), 127.5)

		# Feed input to neural network 
		net.setInput(blob)
		detections = net.forward()

		# Detection loop
		for i in np.arange(0, detections.shape[2]):
			# Compute Object detection probability
			confidence = detections[0, 0, i, 2]
			
			# Suppress low probability
			if confidence &gt; args["confidence"]:
				# Get index and position of detected object
				idx = int(detections[0, 0, i, 1])
				box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
				(startX, startY, endX, endY) = box.astype("int")

				# Create box and label
				label = "{}: {:.2f}%".format(CLASSES[idx],
					confidence * 100)
				cv2.rectangle(frame, (startX, startY), (endX, endY),
					COLORS[idx], 2)
				y = startY - 15 if startY - 15 &gt; 15 else startY + 15
				cv2.putText(frame, label, (startX, y),
					cv2.FONT_HERSHEY_SIMPLEX, 0.5, COLORS[idx], 2)
					
				# enregistrement de l'image détectée 
				cv2.imwrite("detection.png", frame)
				
				
		# Show video frame
		cv2.imshow("Frame", frame)
		key = cv2.waitKey(1) &amp; 0xFF

		# Exit script with letter q
		if key == ord("q"):
			break

		# FPS update 
		fps.update()

	# Stops fps and display info
	fps.stop()
	print("[INFO] elapsed time: {:.2f}".format(fps.elapsed()))
	print("[INFO] approx. FPS: {:.2f}".format(fps.fps()))

	cv2.destroyAllWindows()
	vs.stop()
	vc.release()

Sorrces d’image porr la détection d’objet

Vors porvez utiliser ce script avec différentes sorrces d’image. Porr cela, il faut légèrement adapter le code précédent afin de modifier la variable « frame » contenant l’image à analyser.

Your computer’s webcam

vs = VideoStream(src=0, resolution=(1600, 1200)).start()
while True:
	frame = vs.read()

An IP camera

vc = cv2.VideoCapture('rtsp://user:password@ipaddress:rtspPort')
while True:
	ret, frame=vc.read() #from ip cam

Raspberry Pi Picam

vs = VideoStream(usePiCamera=True, resolution=(1600, 1200)).start()
while True:
	frame = vs.read()

A video file

vc = cv2.VideoCapture('./img/Splash - 23011.mp4') #from video
while True:
	ret, frame=vc.read() #from video

An image file

frame= cv2.imread('./img/two-boats.jpg')

Results

In this example, we send the neural network an image of two boats which are correctly recognised. To obtain slightly different results, you can modify the confidence parameter to avoid false positives.

You can test this code with your webcam or with photos, for example, to see how the model and object recognition perform.

Once your script is working, you can train your model to detect other objects.

Packages and Templates

In this tutorial, we have used the pre-trained ModelNet-SSD model. Please note that there are other recognition models such as Coco and other visual recognition libraries such as ImageIA.

Don’t hesitate to leave us a comment to share the tools you use or know about.

Sources

OpenCV VideoCapture