Python Real-Time Facial Recognition/Identification with CUDA Enabled

5 min readJun 23, 2020

Jun 24, 2020

Introduction

In this story I introduce the most basic facial recognition program in Python. This is almost the first time, for me, to write a program on it, and I’m really excited about the potential of this technology, and how easy to make it! So I decided to write this story to share that with you!

Actually, during this period of COVID-19 pandemic, I’m planing to make an app to measure body temperature with facial recognition technology combined with a thermal camera. You may have seen such cool app at the gate of an airport. That’s awesome! I really want to place it on my lab. So I believe this may be the first story for this journey.

NOTE!
I WRITE THIS STORY FOR ONLY UBUNTU USERS

OK then, What is the “face_recognition”?

This is a facial recognition API of Python, which may be the most simple and easy one to use. You just pass training images and order it to remember them. This is a convolutional neural network, but you would never know, while writing a program, that you are training it, since it’s too simple!

After you have trained it (facial recognition CNN model), you can then pass an image to see if there are known faces on the image.

But there is just one thing you need to be clear. The face_recognition Python API is based another API called dlib. This is the heart of face_recognition, which works on CUDA-enabled GPU. Don’t worry, I will show you how to install them while enabling CUDA.

Let Me Make Clear Some Terminologies

face_recognition

As introduced above, this is a Python API of facial recognition/identification. This is the most simple and easy one as long as I know.

dlib

The core of face_recognition API. This is also a Python API. All calculations of extracting features from face is done by dlib. This works either on CPU or on GPU. But I highly recommend to use the GPU-enabled variant to make all calculations done on a GPU. Otherwise, it will be too slow.

OpenCV (Python-cv2)

Most used image manipulation API of Python. Almost anyone who engage in computer vision is using OpenCV in Python or C++.

CUDA

A library to handle nVIDIA GPUs. I assume you had already installed it. If not, please refer to other Medium stories. I cannot show that here :(

Facial Recognition

Facial recognition is just to know if there is a face, and where it is. Just it. You don’t care who it is.

Facial Identification

You also care the identity of that person.

Conbolutional Neural Network (CNN)

A type of neural network specific to image processing.

Install dlib First

What you need before start installation:

Ubuntu 16, or later (I’m on 18)
CUDA 9.0
cuDnn 7.0
gcc 6.4.0
Python 3

prerequisites

Run the following commands to setup installation environment:

$ sudo apt-get update
$ sudo apt-get install python3-dev
$ sudo apt-get install build-dep python3$ sudo apt-get update
$ sudo apt-get install libblas-dev liblapack-dev libatlas-base-dev$ sudo apt-get update
$ sudo apt-get install libblas-dev liblapack-dev libatlas-base-dev

Then download the dlib repository from github:

$ git clone https://github.com/davisking/dlib.git

Compile and Install

Enter the foler dlib and find setup.py file. This contains all the stuff describing how to install it. Open it, and add the following line in it:

os.environ["CC"] = "gcc-6"

so that setup.py will look like this:

...
from setuptools.command.build_ext import build_ext
from distutils.version import LooseVersionos.environ["CC"] = "gcc-6" # HERE!def get_extra_cmake_options():
    """read --clean, --no, --set, --compiler-flags, and -G options from the command line and add them as cmake switches.
    """

Then just run this:

$ cd dlib
$ mkdir build && cd build$ cmake .. -DCUDA_HOST_COMPILER=/usr/bin/gcc-6 
-DCMAKE_PREFIX_PATH=/usr/lib/x86_64-linux-gnu/ -DDLIB_USE_CUDA=1
-DUSE_AVX_INSTRUCTIONS=1 -DUSE_F16C=1$ cmake --build . --config Release
$ sudo ldconfig

After you compiled C programs with the commands above, then install it to Python:

$ cd ..
$ python3 setup.py install

Done! You can check if dlib is successfully installed with CUDA enabled:

>> import dlib
>> print(dlib.DLIB_USE_CUDA)
>> ture

For more details about installing dlib, refer to this site.

Install face_recognition

Just run this command:

$ pip3 install face_recognition

Python Program

Here is the whole code. I’ll explain what’s in this file step by step.

Imports and Environment Settings

You are going to train a CNN model on known faces later. Before that, you need to store them in a folder named as known_faces/{person}/. Here, on my program, the person is trump. The directory tree is:

Just one image for training suffices.

The TOLERANCE determines how sensitive the face detector is. You can make this value lower if it’s detecting faces that are not Trump’s ones.

import dlib
import face_recognition
import os
import cv2KNOWN_FACES_DIR =  "../known_faces" # training data
TOLERANCE = 0.7
FRAME_THICKNESS = 3 # rectangle thickness
FONT_THICKNESS = 2 # font thickness
MODEL = "cnn" # convolutional
video = cv2.VideoCapture(0) # use webcan as input (source of test images)
print("loading known faces")
known_faces = [] # store known faces here
known_names = [] # store name of them hereprint(dlib.DLIB_USE_CUDA) # ture, if cuda is enabled.

Train the CNN

Actually, you don’t need the for loop since we have just one image as training data. The extracted features are stored on known_faces and known_names so that they can be used later to compare faces.

### TRAIN THE CNN MODEL ON KNOWN FACES #############################
for name in os.listdir(KNOWN_FACES_DIR):
    for filename in os.listdir(f"{KNOWN_FACES_DIR}/{name}"):
        image = face_recognition.load_image_file(f"{KNOWN_FACES_DIR}/{name}/{filename}")
        encoding = face_recognition.face_encodings(image)[0]
        known_faces.append(encoding)
        known_names.append(name)
####################################################################

Test the Detector with a Webcam

The while loops over frames taken by the webcam. Facial detection/identification is applied on each frame, and if a known face is fond, it displays that with a rectangle surrounding the face. Note that, if GPU/CUDA is not enabled, this loop wont work since it lacks computational power.

while True:
    ret, image = video.read() # obtain new image from webcan
    locations = face_recognition.face_locations(image, model=MODEL) # find face
    encodings = face_recognition.face_encodings(image, locations) # extrac face features
    for face_encoding, face_location in zip(encodings, locations):
        results = face_recognition.compare_faces(known_faces, face_encoding, TOLERANCE)
        match = None
        if True in results: # if there is a known face
            match = known_names[results.index(True)] # who that person is
            print(f"Match found: {match}")### DRAW RECTANGLE AND #####################################
            top_left = (face_location[3], face_location[0])
            bottom_right = (face_location[1], face_location[2])
            color = [0, 255, 0] # green 
            cv2.rectangle(image, top_left, bottom_right, color, FRAME_THICKNESS)
            top_left = (face_location[3], face_location[2])
            bottom_right = (face_location[1], face_location[2]+22)
            cv2.rectangle(image, top_left, bottom_right, color, cv2.FILLED)
            cv2.putText(image, match, (face_location[3]+10, face_location[2]+15), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (200, 200, 200), FONT_THICKNESS)
            #############################################################cv2.imshow("webcam images", image) # make window
    if cv2.waitKey(1) & 0xFF == ord("q"):
        break

References

“Facial Recognition on Video with Python” by sentdex, YouTube