Hello everyone , I meet you again , I'm your friend, Quan Jun .
Experimental environment ：python 3.6 + opencv-python 3.4.14.51 It is recommended to use anaconda Configure the same environment
chart 1： Face recognition flow chart
There are many ways to collect face images , Data sets can be downloaded directly from the Internet , You can extract pictures from video , You can also collect pictures from the camera in real time .
Face detection is mainly used in the preprocessing of face recognition , That is to accurately calibrate the position and size of the face in the image . Face images contain rich pattern features , Such as histogram features 、 Color features 、 Template features 、 Structural features and Haar Characteristics, etc . Face detection is to pick out the useful information , And use these features to realize face detection .
Face image preprocessing is based on the results of face detection , Image processing and ultimately serve the process of feature extraction . The original image acquired by the system is limited and random by various conditions interfere , Often can't be used directly , It must be corrected in the early stage of image processing 、 Noise filtering and other image preprocessing . For face images , The preprocessing process mainly includes light compensation of face image Compensation 、 Gray scale transformation 、 Histogram equalization 、 normalization 、 Geometric correction 、 Filtering and sharpening .
The features that face recognition system can use are usually divided into visual features 、 Pixel statistics 、 Face image transform coefficient features 、 Face image algebra Characteristics, etc . Face feature extraction is based on some features of face . Face feature extraction , Also known as face representation , It is the process of facial feature modeling . The methods of facial feature extraction can be summarized into two aspects class ： One is knowledge-based representation ; The other is a representation method based on algebraic features or statistical learning .
The feature data of the extracted face image is searched and matched with the feature template stored in the database , By setting a threshold , When the similarity exceeds this threshold , Then input the matching result Out . Face recognition is to compare the face features to be recognized with the obtained face feature template , Judge the identity information of the face according to the similarity degree . This process is divided into two categories ： One is to confirm , It's one-on-one The process of image comparison , The other is identification , It is a process of one to many image matching and comparison .
Opencv Is an open source cross platform computer vision library , Many general algorithms for image processing and computer vision are implemented internally , about python for , In reference to opencv The library needs to be written as import cv2. among ,cv2 yes opencv Of C++ Namespace name , Use it to indicate that the call is C++ Developed opencv The interface of
At present, there are many mature methods for face recognition , This call OpenCv library , and OpenCV It also provides three face recognition methods , Namely LBPH Method 、EigenFishfaces Method 、Fisherfaces Method . This article uses LBPH（Local Binary Patterns Histogram, Local binary pattern histogram ） Method . stay OpenCV in , You can use functions cv2.face.LBPHFaceRecognizer_create() Generate LBPH Recognizer instance model , Then apply cv2.face_FaceRecognizer.train() Function to complete the training , Last use cv2.face_FaceRecognizer.predict() Function to complete face recognition .
CascadeClassifier, yes Opencv A cascade classifier for face detection in . And you can use Haar, You can also use LBP features . among Haar Feature is a feature that reflects the gray change of an image , A feature of pixel module difference . It falls into three categories ： Edge features 、 Linear characteristics 、 Central and diagonal features .
chart 2： Face recognition module diagram
chart 3： Preparation stage
Read first config file , The first line in the file represents the number of names that have been saved , Next each line is a binary （id,name） That is, the label and the corresponding person name The read results are stored in the following two global variables .
id_dict = {
} # In the dictionary is id——name Key value pair
Total_face_num = 999 # The number of faces with user names that have been recognized ,def init(): # take config The information in the file is read into the dictionary
Load face detection classifier Haar, And prepare the identification method LBPH Method
# load OpenCV Face detection classifier Haar
face_cascade = cv2.CascadeClassifier("haarcascade_frontalface_default.xml")
# Prepare the identification method LBPH Method
recognizer = cv2.face.LBPHFaceRecognizer_create()Then open the tab labeled 0 's camera
camera = cv2.VideoCapture(0) # camera
success, img = camera.read() # Take pictures from the camera chart 4： Enter face
Create folder data It is used to store the photos collected from the camera , Clear this directory before each call .
Then there is a cycle , The number of cycles is the number of samples to be collected , The number of samples taken by the camera , The more, the better , But the slower you acquire and train .
Call in loop camera.read() The return value is assigned to the global variable success, and img Used in GUI Real time display in .
And then call cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) It is used to convert the collected picture into gray-scale picture to reduce the amount of calculation .
Then, the loaded face classifier is used to bring the data recorded by the camera into OpenCv in , Give Way Classifier Judge the face .
# among gray For the grayscale image to be detected ,1.3 For each reduction in image size ,5 by minNeighbors
faces = face_cascade.detectMultiScale(gray, 1.3, 5)faces For in img The face detected in the image , And then use it cv2.rectangle Draw a rectangle around the face . And store the area containing the face into data Folder Note that when writing here , The label of each picture Total_face_num That is, how many recognizable users there are currently （ Add one before entry ）, That is, the number of the current user
cv2.rectangle(img, (x, y), (x + w, y + w), (255, 0, 0))
cv2.imwrite("./data/User." + str(T) + '.' + str(sample_num) + '.jpg', gray[y:y + h, x:x + w])Then print a progress bar at the end of the loop , Used to prompt the progress of image acquisition The main principle is that each output does not break a line and moves the cursor to the beginning of the current line , The output content can change continuously according to the progress , At the same time, the progress information is also output in the prompt box of the control
print("\r" + "%{:.1f}".format(sample_num / pictur_num * 100) + "=" * l + "->" + "_" * r, end="")
var.set("%{:.1f}".format(sample_num / pictur_num * 100)) # Control to visualize progress information
window.update() # Refresh controls to show progress in real time Read data Folder , Read the information in the photo , Get two arrays , One faces All face information is stored 、 One ids Deposit is faces The label corresponding to each face in the , Then pass these two arrays to recog.train Used for training
# Training models # Convert all the input pictures into a four-dimensional array
recog.train(faces, np.array(ids))After training, save the trained recognizer to .yml In file , The file name is face number +.yml
recog.save(str(Total_face_num) + ".yml")Modify the configuration file at the end of each training , The first and last lines need to be modified . The first line has an integer representing the total number of faces that have been entered by the current system , Add one for each modification . The way to modify the file here is to read it into memory first , Then modify the data in memory , Finally write back to the file .
f = open('config.txt', 'r+')
flist = f.readlines()
flist[0] = str(int(flist[0]) + 1) + " \n"
f.close()
f = open('config.txt', 'w+')
f.writelines(flist)
f.close()Add a binary to the last line to identify the user . The format is ： label + Space + user name + Space , The user name defaults to Userx（ among x Identify the user number ）
f.write(str(T) + " User" + str(T) + " \n")chart 5： Face brushing flow chart
Since multiple .yml File to store the identifier （ During actual operation, it is stored in a file to identify errors, so this method is adopted ）, So you need to traverse all the .yml file , If you can't identify each one, you will get an unrecognized result , On the contrary, as long as there is one object that can recognize the current object, it will return the recognized result . For each file, face recognition is performed ten times , If it succeeds more than five times, it means that the final result is recognizable , Otherwise, it means that the current file cannot recognize this face .
In the process of identification GUI Display the captured content in real time in the control of , Draw a rectangle around your face , And according to the result returned by the recognizer, it is displayed near the rectangular box in real time .
idnum, confidence = recognizer.predict(gray[y:y + h, x:x + w])
# Load a font to output information identifying the object
font = cv2.FONT_HERSHEY_SIMPLEX
# Output inspection results and user name
cv2.putText(img, str(user_name), (x + 5, y - 5), font, 1, (0, 0, 255), 1)
cv2.putText(img, str(confidence), (x + 5, y + h - 5), font, 1, (0, 0, 0), 1) The two functions of the program can run independently , You need to adopt a multi-threaded method , But when it comes to the use of critical resources , Multiple processes / Threads should be mutually exclusive to avoid errors , The specific design method in this procedure ： This program adopts the method of multithreading to realize parallel . The three buttons of the program correspond to three functions , They are the input faces 、 Face detection 、 Exit procedure . Because the user interface in the program uses python Medium tkinter Ku did it , The response function of the button is command Pointed out that , So here in every command Set multithreading in the function you jump to , Use... For each stroke threading.Thread Create a new thread , Then the handler in the new thread target To realize the function corresponding to the original button .
p = threading.Thread(target=f_scan_face_thread) When it comes to camera access , Threads need mutually exclusive access , So a global variable is set system_state_lock To represent the current system status , To implement the function of mutex with priority . The lock status is 0 Indicates that the camera is not used ,1 It means you are brushing your face ,2 Indicates that a new face is being entered . During the actual execution of the program, if the status is 0, Then both face brushing and entry can be executed smoothly , If the state is 1 It means you are brushing your face , If you click the face brush button at this time , The system will prompt you to brush your face and reject the new request , If you click the enter face button , Because the priority of face input is higher than that of face brushing , So the original face brushing thread will be blocked ,
global system_state_lock
while system_state_lock == 2: # If a new face is being entered, it will be blocked
passThe new input face process starts to execute and the system status is modified to 2, After entry, the status changes to the original status , The blocked face brushing process continues , The face entry thread has just completed the entry phase and is now training , There are two threads running in parallel , So as to ensure that the training data does not affect the use of the system .
For the exit function , Call... Directly within a function exit(), however python By default, the thread of will wait for all child threads to finish before exiting , So use p.setDaemon(True) Set the thread as a guardian thread , In this way, after the main thread exits, other threads also exit, so as to realize the function of exiting the whole program .
The program uses python Medium tkinter Library for visualization , The advantage is that it takes up less resources 、 Lightweight 、 convenient .
window.after(1, video_loop)
# This sentence means to execute in a second video_loop function
# Because this sentence is written in video_loop Function, so the function executes every second .Test environment ：python 3.6 + opencv-python 3.4.14.51 Need to pack ：
chart 6： Need to pack
Enter... From the dataset
Input from the camera
# Experimental environment ：python 3.6 + opencv-python 3.4.14.51
import cv2
import numpy as np
import os
import shutil
import threading
import tkinter as tk
from PIL import Image, ImageTk
# Read first config file , The first line represents the number of names that have been saved , The next line is （id,name） Tag and corresponding person name
id_dict = {
} # In the dictionary is id——name Key value pair
Total_face_num = 999 # The number of faces with user names that have been recognized ,
def init(): # take config The information in the file is read into the dictionary
f = open('config.txt')
global Total_face_num
Total_face_num = int(f.readline())
for i in range(int(Total_face_num)):
line = f.readline()
id_name = line.split(' ')
id_dict[int(id_name[0])] = id_name[1]
f.close()
init()
# load OpenCV Face detection classifier Haar
face_cascade = cv2.CascadeClassifier("haarcascade_frontalface_default.xml")
# Prepare the identification method LBPH Method
recognizer = cv2.face.LBPHFaceRecognizer_create()
# The opening label is 0 's camera
camera = cv2.VideoCapture(0) # camera
success, img = camera.read() # Take pictures from the camera
W_size = 0.1 * camera.get(3)
H_size = 0.1 * camera.get(4)
system_state_lock = 0 # The quantity that marks the state of the system 0 Indicates that no child threads are running 1 It means you are brushing your face 2 Indicates that new faces are being entered .
# amount to mutex lock , For thread synchronization
''' ============================================================================================ This is initialization ============================================================================================ '''
def Get_new_face():
print(" Entering new face information from the camera \n")
# There are directories data Just empty , Create... If it doesn't exist , Make sure there is an empty data Catalog
filepath = "data"
if not os.path.exists(filepath):
os.mkdir(filepath)
else:
shutil.rmtree(filepath)
os.mkdir(filepath)
sample_num = 0 # Number of samples obtained
while True: # Read pictures from the camera
global success
global img # Because it should be displayed in the visual control , So use the global
success, img = camera.read()
# Turn to grayscale picture
if success is True:
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
else:
break
# Face detection , Bring the data recorded by each frame camera into OpenCv in , Give Way Classifier Judge the face
# among gray For the grayscale image to be detected ,1.3 For each reduction in image size ,5 by minNeighbors
face_detector = face_cascade
faces = face_detector.detectMultiScale(gray, 1.3, 5)
# Frame face ,for The loop guarantees a real-time dynamic video stream that can be detected
for (x, y, w, h) in faces:
# xy Is the coordinate of the upper left corner ,w To be wide ,h For the high , use rectangle Frame the face
cv2.rectangle(img, (x, y), (x + w, y + w), (255, 0, 0))
# Number of samples plus 1
sample_num += 1
# Save image , The gray image is regarded as a two-dimensional array to detect the face area , Here is saved in data Buffer folder
T = Total_face_num
cv2.imwrite("./data/User." + str(T) + '.' + str(sample_num) + '.jpg', gray[y:y + h, x:x + w])
pictur_num = 30 # Indicates the number of samples taken by the camera , The more, the better , But the slower you acquire and train
cv2.waitKey(1)
if sample_num > pictur_num:
break
else: # Output progress bar in the console
l = int(sample_num / pictur_num * 50)
r = int((pictur_num - sample_num) / pictur_num * 50)
print("\r" + "%{:.1f}".format(sample_num / pictur_num * 100) + "=" * l + "->" + "_" * r, end="")
var.set("%{:.1f}".format(sample_num / pictur_num * 100)) # Control to visualize progress information
# tk.Tk().update()
window.update() # Refresh controls to show progress in real time
def Train_new_face():
print("\n Training ")
# cv2.destroyAllWindows()
path = 'data'
# Initialize the identification method
recog = cv2.face.LBPHFaceRecognizer_create()
# Call the function and feed the data to the recognizer
faces, ids = get_images_and_labels(path)
print(' The identification code used for this training is :') # Debugging information
print(ids) # Output identification code
# Training models # Convert all the input pictures into a four-dimensional array
recog.train(faces, np.array(ids))
# Save the model
yml = str(Total_face_num) + ".yml"
rec_f = open(yml, "w+")
rec_f.close()
recog.save(yml)
# recog.save('aaa.yml')
# Create a function , Used to get training pictures from dataset folder , And get the id
# Note that the naming format of the picture is User.id.sampleNum
def get_images_and_labels(path):
image_paths = [os.path.join(path, f) for f in os.listdir(path)]
# Create a new connection list To hold
face_samples = []
ids = []
# Traverse the image path , Import pictures and id Add to list in
for image_path in image_paths:
# Convert it to grayscale image through image path
img = Image.open(image_path).convert('L')
# Convert the picture to an array
img_np = np.array(img, 'uint8')
if os.path.split(image_path)[-1].split(".")[-1] != 'jpg':
continue
# In order to obtain id, Split the picture and path and get
id = int(os.path.split(image_path)[-1].split(".")[1])
# Call familiar face classifier
detector = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
faces = detector.detectMultiScale(img_np)
# Will get pictures and id Add to list in
for (x, y, w, h) in faces:
face_samples.append(img_np[y:y + h, x:x + w])
ids.append(id)
return face_samples, ids
def write_config():
print(" The new face training is over ")
f = open('config.txt', "a")
T = Total_face_num
f.write(str(T) + " User" + str(T) + " \n")
f.close()
id_dict[T] = "User" + str(T)
# The way to modify the file here is to read it into memory first , Then modify the data in memory , Finally write back to the file
f = open('config.txt', 'r+')
flist = f.readlines()
flist[0] = str(int(flist[0]) + 1) + " \n"
f.close()
f = open('config.txt', 'w+')
f.writelines(flist)
f.close()
''' ============================================================================================ The above is the realization of the function of inputting new face information ============================================================================================ '''
def scan_face():
# Use the previously trained model
for i in range(Total_face_num): # Every recognizer uses
i += 1
yml = str(i) + ".yml"
print("\n This time :" + yml) # Debugging information
recognizer.read(yml)
ave_poss = 0
for times in range(10): # Each recognizer is scanned ten times
times += 1
cur_poss = 0
global success
global img
global system_state_lock
while system_state_lock == 2: # If a new face is being entered, it will be blocked
print("\r The face brush is blocked by the input face ", end="")
pass
success, img = camera.read()
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Face recognition
faces = face_cascade.detectMultiScale(
gray,
scaleFactor=1.2,
minNeighbors=5,
minSize=(int(W_size), int(H_size))
)
# check
for (x, y, w, h) in faces:
# global system_state_lock
while system_state_lock == 2: # If a new face is being entered, it will be blocked
print("\r The face brush is blocked by the input face ", end="")
pass
# This call Cv2 Medium rectangle function Draw a rectangle around your face
cv2.rectangle(img, (x, y), (x + w, y + h), (0, 255, 0), 2)
# Call the prediction function of the classifier , Receive the return value label and confidence
idnum, confidence = recognizer.predict(gray[y:y + h, x:x + w])
conf = confidence
# Calculate a test result
if confidence < 100: # It can identify the trained objects —— Directly output the name on the screen
if idnum in id_dict:
user_name = id_dict[idnum]
else:
# print(" Unrecognizable ID:{}\t".format(idnum), end="")
user_name = "Untagged user:" + str(idnum)
confidence = "{0}%", format(round(100 - confidence))
else: # This object is not recognized , Then start training
user_name = "unknown"
# print(" Strange face detected \n")
# cv2.destroyAllWindows()
# global Total_face_num
# Total_face_num += 1
# Get_new_face() # Collect new faces
# Train_new_face() # Training the new faces collected
# write_config() # Modify the configuration file
# recognizer.read('aaa.yml') # Read new recognizer
# Load a font to output information identifying the object
font = cv2.FONT_HERSHEY_SIMPLEX
# Output inspection results and user name
cv2.putText(img, str(user_name), (x + 5, y - 5), font, 1, (0, 0, 255), 1)
cv2.putText(img, str(confidence), (x + 5, y + h - 5), font, 1, (0, 0, 0), 1)
# Display the results
# cv2.imshow('camera', img)
print("conf=" + str(conf), end="\t")
if 15 > conf > 0:
cur_poss = 1 # Indicates that... Can be recognized
elif 60 > conf > 35:
cur_poss = 1 # Indicates that... Can be recognized
else:
cur_poss = 0 # It means not recognizable
k = cv2.waitKey(1)
if k == 27:
# cam.release() # Release resources
cv2.destroyAllWindows()
break
ave_poss += cur_poss
if ave_poss >= 5: # If more than half of the identification descriptions are feasible, return to
return i
return 0 # I haven't identified it yet. It means I can't identify it
''' ============================================================================================ The above is about the design of face brushing function ============================================================================================ '''
def f_scan_face_thread():
# Use the previously trained model
# recognizer.read('aaa.yml')
var.set(' face scan ')
ans = scan_face()
if ans == 0:
print(" final result ： Can't recognize ")
var.set(" final result ： Can't recognize ")
else:
ans_name = " final result ：" + str(ans) + id_dict[ans]
print(ans_name)
var.set(ans_name)
global system_state_lock
print(" The lock is released 0")
system_state_lock = 0 # modify system_state_lock, Release resources
def f_scan_face():
global system_state_lock
print("\n The value of the current lock is ：" + str(system_state_lock))
if system_state_lock == 1:
print(" Blocking , Because I am brushing my face ")
return 0
elif system_state_lock == 2: # If a new face is being entered, it will be blocked
print("\n The face brush is blocked by the input face \n"
"")
return 0
system_state_lock = 1
p = threading.Thread(target=f_scan_face_thread)
p.setDaemon(True) # Thread P Set to daemons If main thread exits P Also quit
p.start()
def f_rec_face_thread():
var.set(' entry ')
cv2.destroyAllWindows()
global Total_face_num
Total_face_num += 1
Get_new_face() # Collect new faces
print(" Collection completed , Start training ")
global system_state_lock # Unlock the lock after collection
print(" The lock is released 0")
system_state_lock = 0
Train_new_face() # Training the new faces collected
write_config() # Modify the configuration file
# recognizer.read('aaa.yml') # Read new recognizer
# global system_state_lock
# print(" The lock is released 0")
# system_state_lock = 0 # modify system_state_lock, Release resources
def f_rec_face():
global system_state_lock
print(" The value of the current lock is ：" + str(system_state_lock))
if system_state_lock == 2:
print(" Blocking , Because the face is being entered ")
return 0
else:
system_state_lock = 2 # modify system_state_lock
print(" Change it to 2", end="")
print(" The value of the current lock is ：" + str(system_state_lock))
p = threading.Thread(target=f_rec_face_thread)
p.setDaemon(True) # Thread P Set to daemons If main thread exits P Also quit
p.start()
# tk.Tk().update()
# system_state_lock = 0 # modify system_state_lock, Release resources
def f_exit(): # Exit button
exit()
''' ============================================================================================ The above is about multithreading design ============================================================================================ '''
window = tk.Tk()
window.title('Cheney\' Face_rec 3.0') # Window title
window.geometry('1000x500') # The multiplication here is small x
# Set the label on the graphic interface , Similar to a prompt window
var = tk.StringVar()
l = tk.Label(window, textvariable=var, bg='green', fg='white', font=('Arial', 12), width=50, height=4)
# explain ： bg In the background ,fg For font color ,font For font ,width For the long ,height For the high , The length and height here are the length and height of characters , such as height=2, The label has 2 Characters so high
l.pack() # place l Control
# Set the placement in the window interface Button Press the key and bind the handler
button_a = tk.Button(window, text=' Start brushing your face ', font=('Arial', 12), width=10, height=2, command=f_scan_face)
button_a.place(x=800, y=120)
button_b = tk.Button(window, text=' Enter face ', font=('Arial', 12), width=10, height=2, command=f_rec_face)
button_b.place(x=800, y=220)
button_b = tk.Button(window, text=' sign out ', font=('Arial', 12), width=10, height=2, command=f_exit)
button_b.place(x=800, y=320)
panel = tk.Label(window, width=500, height=350) # Camera module size
panel.place(x=10, y=100) # Location of the camera module
window.config(cursor="arrow")
def video_loop(): # Used in label Dynamic display of camera content in （ Camera embedded control ）
# success, img = camera.read() # Take pictures from the camera
global success
global img
if success:
cv2.waitKey(1)
cv2image = cv2.cvtColor(img, cv2.COLOR_BGR2RGBA) # Convert colors from BGR To RGBA
current_image = Image.fromarray(cv2image) # Convert the image to Image object
imgtk = ImageTk.PhotoImage(image=current_image)
panel.imgtk = imgtk
panel.config(image=imgtk)
window.after(1, video_loop)
video_loop()
# Window loop , Used for display
window.mainloop()
''' ============================================================================================ The above is about interface design ============================================================================================ '''Publisher ： Full stack programmer stack length , Reprint please indicate the source ：https://javaforall.cn/133970.html Link to the original text ：https://javaforall.cn
