Computer vision makes many tasks come true , Two of these tasks are face detection
（ Locate the face in the image ） And face recognition （ Recognize faces as specific people ）.
OpenCV Some face detection and recognition algorithms are implemented . From safety to entertainment , These technologies
It has applications in real environments .
This chapter is an introduction. OpenCV Some face detection and recognition functions of , And define specific types of
Data files that track objects . say concretely , Will study Haar cascade classifier , Pass the score
Analyze the contrast between adjacent image areas , Determine whether a given image or sub image is related to a known class
Type matching . Let's consider how to combine multiple in a hierarchy Haar cascade classifier , With
Then a classifier is used to identify the parent region （ As far as our goal is concerned , It's a face ）, use
Other classifiers recognize sub areas （ Like the eyes ）.
We also introduced “ rectangular ” This humble but important subject . By drawing
system 、 Copy and resize the rectangular image area , We can track the image area
The domain performs simple operations .
This chapter will introduce the following topics ：
· understand Haar cascade .
· find OpenCV Built in Pre Workout Haar cascade , Including some face detectors .
· utilize Haar Cascade detection of faces in still images and videos .
· Collect images, train and test face detectors .
· Use a variety of different face recognition algorithms ：Eigenface、Fisherface And local
Binary mode histogram （Local Binary Pattern Histogram,LBPH）.
· Copy a rectangular area from one image to another , It can be included or not included
Including mask .
· Use depth camera to distinguish face and background based on depth .
· Exchange the faces of two people in an interactive application .
At the end of this chapter , We integrated face tracking and rectangle manipulation into our previous chapters
Developed interactive applications Cameo in . Last , We will have some face to face
Interaction ！
5.1 Technical requirements
It's used in this chapter Python、OpenCV as well as NumPy. As OpenCV Part of , send
Optional opencv_contrib modular , This includes face recognition . A in this chapter
Some parts use OpenCV Yes OpenNI 2 Optional support for capturing images from depth cameras
image . For installation instructions, refer to page 1 Chapter .
The complete code of this chapter can be found in GitHub library
（https://github.com/PacktPublishing/Learning-OpenCV-4-
Computer-Vision-with-Python-Third-Edition） Of chapter05 Folder
Find . Example images are in this book GitHub Library images In the folder .
5.2 Haar Conceptualization of cascading
When it comes to classifying objects and tracking their positions , What exactly do we want to explore ？ structure
What is the recognizable part of an object ？
Even photographic images from webcams , It may also contain many pleasing details
section . however , Because the light 、 visual angle 、 Visual distance 、 Camera jitter and digital noise change
turn , Image details are often unstable . Besides , Even the real differences in physical details cannot
It can arouse our interest in classification . Joseph · House, （ One of the authors of this book ） At school
Did you learn , Under the microscope , No two snowflakes look the same . Fortunately, , As
A Canadian child , He has learned to recognize snowflakes without a microscope , Because of the snow
The similarities of flowers on the whole are more obvious .
therefore , Some methods of abstracting image details help to produce stable classification and tracking results
fruit . These abstractions are called features , It is said to be abstracted from image data . Although any pixel
May affect multiple features , But features should be less than pixels . Express a set of features as a
Vector , The distance between two images can be measured according to the distance between the corresponding feature vectors of the image
The degree of similarity between .
class Haar Feature is one of the common features used in real-time face detection . In the paper
“Robust Real-Time Face Detection”（International Journal of
Computer Vision 57(2),137–154,Kluwer Academic
Publishers,2001） in , author Paul Viola and Michael Jones First class
Haar Features are used for face detection . Can be in
http://www.vision.caltech.edu/html-files/EE148-2005-
Spring/pprs/viola04ijcv.pdf Find the electronic version of this paper at . Each class Haar
Features describe the contrast pattern between adjacent image regions . for example , edge 、 Vertices and thin lines
All generate a feature . Some features are unique , Because these characteristics usually appear in a
A class of objects （ Like face ） On , It will not appear on other objects . You can put these characteristics
Organize into a hierarchy , It's called cascading , The highest level contains the most prominent features , send
The classifier can quickly reject subjects lacking these features .
For any given subject , Features may be compared based on image size and being evaluated
The neighborhood size of degree varies . The size of the neighborhood where the contrast is being evaluated is called window size
Small . To make Haar The scale of cascade classifier remains unchanged , Or be robust to scale changes , window
The mouth size should remain the same , But rescale the image many times , Therefore, it shrinks to some extent
Time object （ Like face ） The size may match the size of the window . Original image and zoom image
Together, they are called image pyramids , Each successive layer in the image pyramid is a smaller
Zoom out the image .OpenCV A scale invariant classifier is provided , Can be in a specific
Format from XML File loading cascade classifier . This classifier internally converts any given image
Convert to image pyramid .
use OpenCV Realized Haar Cascade classifiers do not change the rotation angle or perspective
Not robust . for example , I think the inverted face is different from the upright face , Think the side view
The face is different from the face seen from the front . By considering various transformations of the image and the size of multiple windows
Small , More complicated 、 More resource intensive implementations can improve Haar Cascade robustness to rotation angle
sex . however , We will only introduce OpenCV The implementation of the .
5.3 obtain Haar Cascading data
OpenCV 4 Source code or installed OpenCV 4 Pre packaged build , It should contain the name
data/haarcascades Subfolders of . If this folder cannot be found , Please go back to the
1 Chapter acquisition OpenCV 4 Source code description .
data/haarcascades The folder can be named
cv2.CascadeClassifier Of OpenCV Class loaded XML file . Instances of this class give
Set the XML The document is interpreted as Haar cascade , For some kind of object （ Like face ） Provide a
Test model .cv2.CascadeClassifier This type of... In any image can be detected
object . Usually , We can get static images from files , Or from a video file or video
Frequency camera obtains a series of frames .
find data/haarcascades after , Create a directory for the project elsewhere . stay
Create a folder named cascades Subfolders of , Take the following file from
data/haarcascades Copied to the cascades：
seeing the name of a thing one thinks of its function , These cascades are used to track faces and eyes . They need to be observations
Object front 、 Upright view . later , These levels will be used when building a face detector
United .
If you want to know how to generate these cascading files , See Joseph · House,
Of OpenCV 4 for Secret Agents [1] （ The original book was written in 2019 Year by year Packt Published by the publishing house ） in
The first 3 Chapter . As long as you have enough patience and a powerful computer , You can create your own level
United , And train the created cascades for various types of objects .
[1] This book 2 Chinese version of 《OpenCV Project development practice （ The book first 2 edition ）（ISBN
978-7-111-65234-2） Has been in 2020 Published by mechanical industry press in .—— Editorial note
5.4 Use OpenCV Face detection
Whether face detection is carried out on static images or video return signals ,
cv2.CascadeClassifier There is hardly any difference . Video is just a continuous static picture
image ： Face detection in video only applies face detection to every frame . natural , With more
Advanced technology , You can continuously track the detected face in multiple frames , And determine each frame
Are the faces in the same . however , It's best to know that the basic sequential method is also effective .
Let's test some faces .
5.4.1 Face detection on still images
The first and most basic method of face detection is to load an image and detect
One of the faces . In order to make the result visually meaningful , We will put people in the original image
Draw a rectangle around the face . please remember , Face detector is designed for upright face
Of , We will use multiple people （ Lumberjack ） Standing in a row , They stood side by side
state 、 Face the camera .
take Haar cascade XML After the file is copied to the cascading folder , We create the following basic script
To perform face detection ：
Let's take a step-by-step look at the previous code . First , Use the necessary cv2 Import , This book
Every script of has this import . then , Make a statement face_cascade Variable , This is a
One CascadeClassifier object , Used to load face detection cascade ：
then , use cv2.imread Load the image file , Convert it to a grayscale image , because
Cascade Classifier Need grayscale image . next step , use
face_cascade.detectMultiScale Real face detection ：
detectMultiScale The parameters of include scaleFactor and minNeighbors.
scaleFactor The parameter should be greater than 1.0, Determine the image at each iteration in the face detection process
Downscaling ratio . just as 5.2 Section of the , The purpose of this downscaling is to put different
Face and window size matching to achieve scale invariance .minNeighbors The argument is to preserve
The minimum number of overlapping detection required for the detection result . Usually , We expect to be able to work in multiple heavy
Someone's face is detected in the folding window , More overlap detection makes us more confident about the people we detect
Face is a real face .
The value returned from the detection operation is a tuple list representing the face rectangle .OpenCV Of
cv2.rectangle Function allows us to draw a rectangle at the specified coordinates .x and y respectively
Left and top coordinates ,w and h Respectively represent the width and height of the face rectangle . Through circulation
Loop traversal faces Variable , Draw a blue rectangle around all faces , Please make sure to use the original
Start image , Instead of grayscale images ：
Last , call cv2.imshow Display the processed image . Usually , To prevent images
The window closes automatically , Insert a pair waitKey Call to , When the user presses any key, it returns
return ：
Okay , We detected a group of loggers in the image , Pictured 5-1 Shown .
The photo in this example is Sergei · Procukin – Gorsky （Sergey
Prokudin-Gorsky）（1863—1944） The works of , Procukin – Gorsky is color
Pioneer of photography . Czar Nicholas II let procukin – Gorsky filmed the Russian Empire
People and places , Regard it as a huge documentary project .1909 year , Procukin –
Gorsky photographed these loggers near the svil River in Northwest Russia .
5.4.2 Face detection on video
Now? , We learned how to detect faces on still images . As mentioned earlier ,
We can watch videos （ The camera sends back signals or pre recorded video files ） Every one of them
Repeat the process of face detection on the frame .
The next script will turn on a camera to send back signals , Read a frame , Check the frame
Face , And scan eyes in the detected face . Last , Draw blue around the face
The rectangular , Draw a green rectangle around the eyes . The following is the complete script ：
We decompose the above example into smaller 、 Easier to understand ：
（1） As usual , Import cv2 modular . after , Initialize two
CascadeClassifier object , One for face , The other is for the eyes ：
（2） Like most interactive scripts , Turn on a camera and send back a signal , open
Start iteration frame . continue , Until the user presses a key . When a frame is successfully captured , Put it
Convert to grayscale as the first step of processing ：
（3） Using face detector detectMultiScale Methods face detection .
As mentioned before , We used scaleFactor and minNeighbors Parameters . We also
Use minSize The parameter specifies the minimum size of the face , Specific for 120×120, So it won't
Try to detect faces smaller than this size .（ Suppose the user is sitting near the camera , There can be
Say with certainty , The user's face in the image will be larger than 120×120 Pixels .） The following is right
detectMultiScale Call to ：
（4） Iterate the detected face . Draw a rectangle around each rectangle of the original color image
A blue border . then , Carry out eye detection in the same rectangular area of the gray image ：
The accuracy of eye detector is lower than that of face detector . You may see
shadow 、 Some frames or other facial areas mistaken for eyes . To improve the results ,
You can try to roi_gray Defined as a small area of the face , Because we can easily guess
To the position of the eyes in the upright face . You can also try to use maxSize Parameters to avoid those
Too big can't be a false alarm of the eyes . Besides , You can adjust minSize and maxSize, Make the size
And w and h（ That is, the detected face size ） Is proportional to the . As an exercise , You can do whatever you want
Change these and other parameters .
（5） Loop through the generated eye rectangle , And draw a green outline around it ：
（6） Last , Display the generated frame in the window ：
Run script . If the detector produces accurate results , And if there is any face
In the field of view of the camera , You should see a blue rectangle around your face , Around the eyes
Surrounded by a green rectangle , Pictured 5-2 Shown .
chart 5-2 Script run result
Experiment with this script , Study the table of face and eye detectors under different conditions
present . Try to do it in a brighter or darker room . If you wear glasses , Try taking off your glasses and
Do it once. . Try to do it under different faces and expressions . Adjust the detection in the script
Parameters , Look at the effect of these parameters on the results . When you are satisfied , Let's think about
stay OpenCV What else can we do in .
5.4.3 Face recognition
Face detection is OpenCV A great feature of , It also constitutes a higher-level operation
—— Face recognition —— The basis of . What is face recognition ？ Face recognition is given by the program
The ability to recognize a person in an image or video containing a face . One of the ways to achieve this goal
（ It's also OpenCV The method adopted ） By providing a group of classified pictures for the program （ Face
database ） To train the program , Then recognize according to the characteristics of these pictures .
OpenCV Another important feature of the face recognition module is , Each recognition has a setting
reliability , This allows us to set thresholds in real applications to limit the occurrence of error recognition
rate .
Let's start from scratch . For face recognition , We need faces to be recognized .
Face can be obtained in two ways ： Provide your own images or get free face count
Database . stay http://www.face-rec.org/databases/ There is a large number of faces on
The database is available online . Here are some famous examples ：
· Yale face database （Yalefaces）, The website is
http://vision.ucsd.edu/content/yale-face-database.
· Extended Yale University face database B, The website is
http://vision.ucsd.edu/content/extended-yale-face-database-b-b.
· Face database （ From Cambridge AT&T laboratory ）, The website is
http://www.cl.cam.ac.uk/research/dtg/attarchive/facedatabase.html.
For face recognition of these samples , We have to deal with the faces that contain the sampled people
Image for face recognition . This process may be instructive , But maybe it's better to mention
For our own images as satisfactory . Many computer vision learners have the same idea
Law ： Can you write a program to recognize your face and have a certain degree of confidence .
1. Generate face recognition data
Let's write a script to generate these images . Images with different expressions are just us
The required , But the training image should be square , And the same size . Our show
The example script requires the size of the image to be 200×200, But most of the freely available datasets
Images are smaller than this size .
Here's the script ：
ad locum , We use our new knowledge of how to detect faces in video to generate
Sample image . We detect faces , Crop the face area of the gray conversion frame , Make it major and minor
Whole for 200×200 Pixels , And save it as PGM file , And specify a name in a specific folder
（ In this case jm, The initials of one of the authors , You can use your own name
The first letter of ）. Like many window applications , The program will run all the time , Until the user
Press a key .
The reason why count The variable is because we need the cumulative name of the image . Run script
Seconds , Change your facial expression several times , And check the target folder specified in the script . function
Script for a few seconds , Change your facial expression several times , Check the target file you specified in the script
clip . You will find many images of your face , They become grayscale versions , Adjusted the big
Small , And use <count>.pgm Format name .
modify output_folder Variable , Match it with your name . for example , You can choose
Choose '../data/at/my_name'. Run script , Wait for it in many frames （ such as 20 The frame or
more ） Your face is detected in , Then press any key to exit . Now? , Revise again
output_folder Variable , Match it with the name of a friend you want to recognize . example
Such as , You can choose '../data/at/name_of_my_friend'. Do not change the folder
The basic part （ In this case '../data/at'）, Because in the back “ Load face recognition
Training data ” part , We will write code from all subfolders of this basic folder
Loading training images . Let your friend sit in front of the camera , Run the script again , Let it in
Detect your friend's face in many frames , And then quit . Repeat for anyone else you want to identify
This process .
Now? , We continue to try to recognize the user's face in the video return signal . It would be good
play ！
2. Face recognition
OpenCV 4 Realized 3 Different face recognition algorithms ： Characteristic face
（Eigenface）、Fisherface And local binary pattern histogram （Local Binary
Pattern Histogram,LBPH）. Characteristic face and Fisherface Mainly from one name
Composition analysis （Principal Component Analysis,PCA） The general algorithm of . of
Detailed description of the algorithm , See the following link ：
·PCA：Jonathon Shlens stay http://arxiv.org/pdf/1404.1100v1.pdf Ascending
Provides an intuitive introduction . The algorithm is developed by Karl · Pearson （Karl Pearson） stay
1901 Invented in , And original papers “On Lines and Planes of Closest Fit to
Systems of Points in Space” The website is http://pca.narod.ru/pearson1901.pdf.
·Eigenface： See paper “Eigenfaces for Recognition”（1991）（ author
yes Matthew Turk and Alex Pentland）, The website is
http://www.cs.ucsb.edu/~mturk/Papers/jcn.pdf.
·Fisherface： Groundbreaking paper “The Use of Multiple Measurements in
Taxonomic Problems”（1936）（ The author is R.A.Fisher）, The website is
http://onlinelibrary.wiley.com/doi/10.1111/j.1469-
1809.1936.tb02137.x/pdf.
· Local binary mode ： The first paper to introduce the algorithm is “Performance
evaluation of texture measures with classification based on Kullback
discrimination of distributions”（1994）（ The author is T.Ojala、M.Pietikainen and
D.Harwood）, The website is https://ieeexplore.ieee.org/document/576366.
As far as this book is concerned , We give an overview of these algorithms . First , These algorithms follow
Follow a similar process ： Make a group of classified observations （ Our face database , everyone
Both contain a large number of samples ）, On this basis, train a model , Face image （ This may be possible.
It is the face area we detected in the image or video ） Analyze and determine two things
—— The identity of the subject and the measure of confidence that this identity is correct . The latter is often called
For confidence .
The eigenface algorithm performs principal component analysis （PCA）, Identify a set of observations （ Again
Face database ） The principal component of , Calculate the current observation （ People detected in the image or frame
face ） Divergence relative to data set , And produce a value . The smaller the value , Face database and
The smaller the difference between the detected faces , So it's worth 0 Express exact match .
Fisherface Also from the principal component analysis （PCA） Derivative , And more complex
Logic . Although the calculation is more intensive , But the results are often more accurate than the eigenface algorithm .
contrary ,LBPH Divide the detected face into small cells , And create for each cell
A histogram , Describe whether the brightness of the image is increasing when comparing neighborhood pixels in a given direction
Add . The histogram of this cell can be compared with the corresponding cell in the model , thus
Generate similarity measures . stay OpenCV In the face recognizer of ,LBPH The implementation of is the only allowed
The model sample face and the detected face have different shapes 、 Face recognition of different sizes
device . therefore , It's very convenient. , The author of this book found that the accuracy of this algorithm is better than the other two
Algorithm .
3. Load the training data of face recognition
No matter what face recognition algorithm you choose , We can all load in the same way
Training images . In front of “ Generate face recognition data ” In the part , We generated training
Images , And save them in folders organized according to people's names or initials .
for example , The following folder structure can contain the author of this book Joseph Howse（J.H.） and
Joe Minichino（J.M.） Face image samples ：
We write a script to load these images , And a kind of OpenCV Face recognizer
Label them in an understandable way . To deal with file systems and data , We will make
use Python Standard library os Modules and cv2 and numpy modular . We create the following
import Script at the beginning of the statement ：
Let's add the following read_images function , This function can traverse the subdirectories of the directory ,
Load image , Adjust these images to the specified size , And put the adjusted image into a
In a list . meanwhile , This function also builds two other lists ： The first is the name or person
Initials list （ Based on subfolder name ）, The second is associated with the loaded image
A list of labels or numbers ID list . for example ,jh It's a name ,0 It can be from jh Sub file
Clip the labels of all images loaded . Last , This function converts the list of images and labels into
NumPy Array , And back to 3 A variable ： Name list 、 Graphic NumPy Arrays and labels
NumPy Array . The following is the implementation of this function ：
Call by adding the following code read_images function ：
Edit the path_to_training_images Variable , To ensure that the change
The quantity is the same as before “ Generate face recognition data ” Part of the code defined output_folder Variable
Match the basic folder of .
up to now , We already have training data in a useful format , But not yet
Build a face recognizer , No training . We will finish it in the following content
These tasks , At the same time, we will continue to implement the same script .
4. Face recognition based on feature face
Now that you have the training image array and its label array , It takes only two lines of code to create
And train a face recognizer ：
What we've done here ？ We use OpenCV Of
cv2.EigenFaceRecognizer_create Function to create a feature face recognizer , adopt
Pass image arrays and labels （ Numbers ID） Array training recognizer . We can also choose to
Two parameters are passed to cv2.EigenFaceRecognizer_create：
·num_components： This is a PCA The number of principal components to be retained .
·threshold： This is a floating point value , Specify the confidence threshold . The rejection confidence is lower than
Threshold face . By default , This threshold is the maximum floating point value , Therefore, no
Face .
To test the recognizer , We use a face detector and a camera to send back
video signal . As you did in the previous script , We can use the following code
To initialize the face detector ：
The following code initializes the signal returned by the camera , Traversal frame （ Until the user presses any
key ） And face detection and recognition for each frame ：
Let's take a look at the most important functions in the previous code block . For everyone detected
face , We convert it and resize it , In order to get gray that matches the expected size
Degree version （ In this case , The expected size is “ Load the training data of face recognition ” Subsection
training_image_size Variable defined 200×200 Pixels ）. then , The adjusted
Gray face is passed to face recognizer predict function . This function will return a tag and
Degree of confidence . We look for the name of the person corresponding to the digital tag of this face .（ please remember , We
stay “ Load the training data of face recognition ” Created in names Array .） We are identifying
The name and confidence are given in blue text above the face . After traversing all detected faces
after , Display the annotated image .
We use a simple method for face detection and recognition , Its purpose is to
Enables you to run a basic application , And understand OpenCV 4 Face recognition in
cheng . To improve the method and make it more robust , Further steps can be taken , for example ,
Correctly align and rotate the detected face to maximize the recognition accuracy .
Run script , You should see something like this 5-3 The effect of .
Next , We will consider how to adjust these scripts , Use other face recognition algorithms to
Replace feature face .
5. be based on Fisherface Face recognition
How to base on Fisherface Face recognition ？ This process is not much like the above process
Big change , Just instantiate a different algorithm . Using default parameters ,model Variable
The statement is as follows ：
cv2.face.FisherFaceRecognizer_create Acceptance and
cv2.createEigenFace-Recognizer_create Same two optional parameters ： To ensure
The number of principal components left and the confidence threshold .
6. be based on LBPH Face recognition
Last , Let's have a quick look LBPH Algorithm . Again , The process is similar to the above
cheng . But , The algorithm factory accepts the following optional parameters （ According to the order ）：
·radius： It is used to calculate the pixel distance between the neighborhoods of the cell histogram （ The default is
1）.
·neighbors： Used to calculate the number of neighborhoods of the cell histogram （ The default is 8）.
·grid_x： The number of cells that divide the face horizontally （ The default is 8）.
·grid_y： The number of cells that vertically segment the face （ The default is 8）.
·confidence： Confidence threshold （ By default , Is the highest possible floating-point value ,
This will not discard any results ）.
Using default parameters ,model The statement is as follows ：
Please note that , Use LBPH, We don't need to resize the image , Because the division
For the grid, it is allowed to compare the recognized patterns in each cell .
7. Discard results based on confidence
predict Method returns a tuple , The first element is the target of the identified individual
sign , The second element is confidence . All algorithms have the option of setting the confidence threshold
term , This threshold measures the matching degree between the recognized face and the original model , therefore , The score is 0
To indicate a perfect match .
In some cases , You may prefer to keep all the recognition results , Further application
To deal with , In this way, you can propose your own algorithm to estimate the confidence of the recognition results . example
Such as , If you try to identify people in the video , You may want to analyze the confidence in subsequent frames
degree , To determine whether the recognition is successful . under these circumstances , You can check the settings obtained by the algorithm
reliability , And draw your own conclusion .
The typical value range of confidence depends on the algorithm . Characteristic face and Fishface Produced
The range of values （ roughly ） stay 0～20 000, lower than 4000 All the scores of indicate a phase
When you identify the result with confidence . about LBPH, The reference value of good recognition result is lower than 50, the
There are more than 80 The values of are considered to be bad confidence .
The conventional custom method is , Until there are enough arbitrary confidence with satisfactory
Frame number of degrees , Just draw a rectangle around the recognized face , But you can use
OpenCV Face recognition module to customize the application according to needs .
5.5 Change your face under infrared
Face detection and recognition is not limited to the visible spectrum . In the near infrared （Near-
Infrared,NIR） Camera and near-infrared light source , Even if it seems to the human eye to be a completely dark field
In the view , Face detection and recognition is also possible . This function is used in security and monitoring applications
It's very useful .
In the 4 In the chapter , We studied ASUS Xtion PRO Wait for the base of near infrared depth camera
This usage . We have expanded interactive applications Cameo Object oriented code . We use deep
The camera took some pictures . According to the depth , We divide each frame into a main layer
（ Such as the user's face ） And other layers . Paint other layers black , In this way, hiding is achieved
The effect of the background , Only the main layer in the interactive video signal （ User's face ） Appear on the screen
On .
Now? , We modify Cameo, Practice your previous deep segmentation skills and face detection
New skills . Let's detect faces , When at least two faces are detected in the same frame , In exchange for
These two faces , Make one person's head appear on another person's body . We won't copy it
All pixels in the measured face rectangle , Instead, only a part of the main depth layer of the rectangle is copied
Pixels . This should achieve the effect of face changing , Without swapping the background pixels around the face .
Once the modification is completed ,Cameo Will be able to produce similar graphs 5-4 Output .
We saw Joseph · House's face and his mother Janet · House, （Janet
Howse） Their faces exchanged . Even though Cameo Is to copy pixels from a rectangular area （ In the foreground ,
The bottom of the exchange area is clearly visible ）, Some background pixels are not swapped , So we won't
See that there are rectangular edges everywhere .
You can https://github.com/PacktPublishing/Learning-
OpenCV-4-Computer-Vision-with-Python-Third-Edition Find a book at
Database alignment Cameo All relevant changes to the source code , Especially in chapter05/cameo file
Modifications in the folder . For the sake of simplicity , We won't discuss all the changes in this book , however
Some key points will be discussed in the next two sections .
5.5.1 Modify the loop of the application
In order to support the face changing function ,Cameo The project has two new modules , be known as rects and
trackers.rects The module contains functions for copying and exchanging rectangles , With an optional
Mask of , Copy or swap operations can be limited to specific pixels .trackers The module contains a
One is called FaceTracker Class , It makes OpenCV The face detection function of is suitable for object-oriented
Programming style .
Because we have already introduced OpenCV Face detection function , And already
After demonstrating the object-oriented programming style in the previous chapter , So I won't discuss it here
FaceTracker The implementation of the . You can check it in the library of this book FaceTracker The implementation of the .
open cameo.py, In this way, you can browse all changes to the application ：
（1） At the top of the file , You need to import a new module , Such as bold in the following code block
Shown ：
（2） Now? , We turn our attention to CameoDepth Class __init__ Methodical
modify . The updated application uses a FaceTracker example . As its function
Part of the ,FaceTracker You can draw a rectangle around the detected face . We give
Cameo The user has an option to enable or disable the drawing of face rectangles . We will pass a
Boolean variables track the currently selected option . The following code block shows initialization
FaceTracker Object and Boolean variables （ Show in bold ）：
We are CameoDepth Of run method FaceTracker object , The method
Contains the application main loop that captures and processes frames . Every frame successfully captured , Just call
FaceTracker To update the face detection results , And get the latest detected face .
then , For every face , Create a mask from the depth camera parallax map .（ In the 4 In the chapter ,
We created such a mask for the whole image , Not for each face rectangle
Create a mask .） then , Call function rects.swapRects To perform the mask of the face rectangle
In exchange for .（ later , We will be in 5.5.2 Section discusses swapRects The implementation of the .）
（3） According to the currently selected option , We might let FaceTracker Around the face
Draw a rectangle around . All relevant changes are shown in bold in the following code block ：
（4） Last , modify onKeypress Method so that the user can press X Key to start or stop
Display the rectangle around the detected face . Again , The relevant changes are in the following code block
Show in bold ：
Next , Let's look at the introduction earlier in this section rects Module implementation .
5.5.2 Mask copy operation
rects The module is in rects.py Implemented in . stay 5.5.1 In the festival , We've seen
Yes rects.swapRects A call to function . But , In considering swapRects The implementation of the
Before , We first need a more basic copyRect function .
Back to the first 2 Chapter , We learned how to use NumPy The slicing syntax of converts data from a
The rectangular region of interest is copied to another rectangular region of interest . In the rectangle of interest
Outside the area , The source and target images are not affected . Now? , We want to further restrict this reply
Control operation . We want to use a given mask of the same size as the source rectangle .
We will only copy those pixels in the source rectangle whose mask value is not zero . Other pixels should be guaranteed
Keep the original value of the target image . This logic contains an array of conditions and two possible outputs
An array of values , It can be used numpy.where Functions express simply .
Remember this method , Let's consider copyRect function . It accepts a source image and
Target image 、 A source rectangle and a destination rectangle , And a mask as a parameter . The latter can
Can be None, under these circumstances , Just adjust the content size of the source rectangle to match the target moment
shape , Then assign the adjusted content to the target rectangle . otherwise , The next step is to be sure
The mask and image have the same number of channels . Suppose the mask has a channel , But the image may
There are three channels （BGR）. We can use numpy.array Of repeat and reshape Fang
Method to add repeated channels to the mask . Last , Use numpy.where Perform a copy operation .
The complete implementation is as follows ：
