Reference books ：《Python Face recognition from introduction to engineering practice 》
The whole book consists of 8 Chapter ：
The first 1 Chapter ： The basic knowledge and necessary common sense of face recognition are introduced ;
The first 2~4 Chapter ： The mathematics related to face recognition is explained in detail 、 machine learning 、 Computer vision 、OpenCV Related foundations and algorithms ;
The first 5 Chapter ： Explain the principle and use of deep learning Keras How to implement the deep learning model ;
The first 6 Chapter ： The common face recognition algorithms are introduced ;
The first 7~8 Chapter ： The implementation method of face recognition engine is explained in detail , And how to make the model engineering .
How to read this book
This book is divided into 3 part ：
The first part ： The basic chapter （1~4 Chapter ）,（ Basic knowledge and concepts are introduced , Including machine learning and mathematical knowledge closely related to face recognition 、 Computer vision and OpenCV Kuo et al .）
The second part ： Application articles （5~6 Chapter ）,（ The framework of in-depth learning is explained in detail Keras, And how to design and implement a human face recognition engine through this framework .）
The third part ： Expansion （7~8 Chapter ）,（ After the face recognition engine is completed , No matter how good the local validation is , Whether it can be used is the most critical . In this part , The deployment of face recognition system and image retrieval knowledge are explained .）
The first part ： The basic chapter 【 Theory Chapter 】
What is face recognition ？
Narrow expression ： By analyzing and comparing the facial visual feature information , A computer vision technique that performs authentication or lookup .
（ namely An authentication technology .（ And fingerprint recognition 、 Voiceprint recognition 、 Finger vein recognition 、 Iris recognition belongs to the same field , That is, the field of biometrics .）（ therefore , In the narrow sense, face recognition generally refers to the scene of identity confirmation or search through face images .））
In a broad sense ： Recognize faces in pictures or video streams , And a series of related operations on the face image .
（ for example , In face authentication , Inevitably, there will be experiences such as image acquisition 、 Face detection 、 Face location 、 Face extraction 、 Face preprocessing 、 Face feature extraction 、 Face features are better than equivalent steps , These can be considered as the category of face recognition .）
The application of face recognition ：
1. Authentication scenario ：（ Access control system 、 Mobile phone unlocking, etc ）
2. Certificate verification scenario ：（ Face verification, etc ）
3. Face retrieval scene ：（ Face pictures “ One to many ” Ground comparison ）
4. Face classification scene ：（ Judge the face in the face picture ： Gender 、 Age range 、 race 、 Expression, etc ）
5. Interactive application scenarios ：（ Beauty App）
…………
The goal of face recognition ：
In different application scenarios , The targets of face recognition may be different . however , For the vast majority of face recognition applications , The goal of face recognition is similar .
The general process of face recognition can be described as ：
1. The face recognition model is used to judge whether there is a face in the picture
2. If there are faces , Then locate the location of the face / Or extract the advanced features of the face image （ As the feature vector of the face image , And used in the subsequent image processing .）
Because the application scenarios of face recognition are different , The choice and focus of the above steps are also different . for example ：
Locate the position of the face in the picture , Can be used for things like AR And so on
（ By locating the key points of the face , It can perform geometric transformation on face images ）
（ The correction of face in image can be realized by geometric transformation ）（ meanwhile , The obtained face key points can also be used to realize such as “ Thin face ” Wait for the operation .
If the function you want to achieve is not the geometric transformation of the face image ,
Instead, it makes a specific judgment on the face in the picture , For example, judge the gender of the face in the picture 、 Age etc. ,
（ Then the goal at this time is ： Extract the advanced features of the face in the picture , Then according to the extracted advanced features , Use classifiers for classification , Such as gender recognition 、 Age determination and other functions ;)
（ For face comparison , A feasible idea is still to extract the advanced features of human face in the image , Then compare the two features , So we can get a similarity value , By comparing the value with the preset threshold value , So as to judge whether the faces in the two pictures belong to the same person .）
From the introduction above, we can see that , Many steps of different face recognition applications overlap , The difference lies only in the depth of the operation level .
Through reasonable choice 、 Combine the operation levels of face pictures , We can achieve our desired goal .
This implementation process can be said to be “ All change is the same ”, The core technology is to extract advanced features of face images .
General methods of face recognition ：
（ Take the face contrast scene as an example , Introduce a feasible idea of face comparison ）
1. Image preprocessing ：
（ In many computer vision projects , Image preprocessing is often required .）
（ The main reason is that the input pictures are often out of specification , It may even interfere with the subsequent work of the system .）
（ Such as , The picture is noisy , Or the picture size does not meet the system requirements , These are the things that need to be done in the pretreatment step .）
（ The corresponding processing method can be to filter the picture , So as to make the picture more in line with the system requirements .）
2. Face detection ： The operation of judging whether there is a face in a picture .
（ For face recognition applications , Face detection can be said to be an essential part .）
（ The quality of face detection , It will directly affect the performance of the whole system .）
3. Face correction ：
（ Also known as face correction 、 Face righting 、 Face alignment, etc ）
（ Because the face image in the picture is often not “ Face up ”, Some have side faces , There are faces with oblique angles . This kind of facial image that seems not very regular in geometric form , It may adversely affect the subsequent face related operations . therefore , Someone proposed face correction .）
（ We can think roughly of , Face correction is a geometric transformation of face images in images , The purpose is to reduce the influence of geometric factors such as inclination angle on the system .）
（ therefore , Face correction is also generally considered as geometric normalization of face images . Face correction is generally used in face comparison and other application scenarios with subsequent face feature extraction .）
（ however , With the wide application of deep learning technology , Face correction is not absolutely required to exist in the system .）
（ The predictive ability of deep learning model is much stronger than that of traditional face recognition methods , Because it wins with big data sample training . So some face recognition systems have face correction , Some models do not .）
4. Facial feature point location ： After detecting the position of the face in the picture , Locate the point in the picture that can represent the key position of the face in the picture .
（ Common facial feature points are composed of left and right eyes 、 Left and right corners of the mouth 、 Nose here 5 It's made up of dots 5 Point face feature points , And the contour of face and lips 68 Point face feature points, etc .）
（ By locating the facial feature points in the picture , Face correction is possible , It can also be applied to some mapping applications .）
5. Face feature extraction ：
（ For many face recognition applications , Face feature extraction is a key step .（ For example, in gender judgment 、 Age recognition 、 The face is more beautiful than in the equivalent scene , Take the extracted facial features as the main judgment basis .） The quality of the extracted facial features will directly affect the output results .）
（ We can argue that RGB The form of the color picture is a red 、 green 、 Matrix of blue three channels , The binary image and gray image itself are a matrix in storage , There are many pixels in these pictures .）
（ The extracted features are often expressed in the form of feature vectors , There are usually not many elements of a vector （ Generally in ‘ thousand ’ This order of magnitude ））
（ therefore , From a macro point of view , The process of feature extraction can be regarded as a process of data extraction and compression .）
（ mathematically , In fact, it is a process of dimension reduction .）
6. classifier ： Refers to a classification algorithm .
（ for example , We need to judge the gender of the face in the picture , After extracting the advanced features of the face image , We should judge the gender according to the extracted feature .）
（ This process is actually a process of two categories , That's what everyone knows ： Without regard to special circumstances , Human beings can be divided into two categories , Man or A woman ）
（ The judgment is based on the advanced features of the face image mentioned above , The algorithm used for judgment is the so-called classifier .）
（ Of course , The design and implementation of this classifier is not so easy , We will describe it in detail in the following content .）
Through the above introduction , A feasible way for us to understand face comparison is ：
First, preprocess the picture , Then face detection and judgment , Finally, the features are extracted and compared .
Face comparison is a typical application scenario in face recognition , We can summarize the commonness of face recognition applications from this example ：
1. Image preprocessing ：（ The purpose is to reduce the interference caused by the picture itself to the system judgment , Or make the picture format more suitable for the system . Common processing methods include image denoising 、 Size conversion 、 Grayscale processing, etc .）
2. Face detection ：（ For face recognition application scenarios , If there is no face in the picture , Then all subsequent operations will become meaningless , It may even lead to wrong results . If you can't recognize the face in the picture , It will also lead to the early termination of the entire system . therefore , Face detection plays a very important role in the application of face recognition , It can even be regarded as an indispensable part .）
3. Feature point location and feature extraction ：（ If the face recognition system wants to realize some advanced functions , Acquiring features will be an essential part . For different face recognition application scenarios , Their definitions of features are also different . For example, you want to automatically add an eye as a decoration to the face in the picture , So the feature we need to get is the position of the eyes in the picture , So it is very necessary to locate the feature points based on human eyes ; And for face contrast 、 Gender recognition and other scenarios , It is necessary to obtain a feature vector that can represent the face in the picture .）
4. The use of features ：（ We have made it clear that , Getting the features we need is an important basis for subsequent operations . Several ways of using features have been mentioned above , Such as using classifiers for classification 、 Use comparator to compare , Or use the located facial feature points to map the image . There is no doubt that the purpose of feature utilization is very clear , Because this is often the direct output of the system we ultimately want to obtain .）
The above process may be very complex in implementation , But for most face recognition applications , The general idea is the same .
The above contents can be summarized as the general method of constructing face recognition system , We will design and implement the face recognition system with this idea in the following content .
A new starting point of computer vision
（ Face recognition is an important application of computer vision , therefore , When it comes to face recognition, we have to mention computer vision ）
We divide the research fields related to visual perception and image processing with computer as a tool into an independent research space , This research space is what we call computer vision , Also known as machine vision .
Since the concept of artificial intelligence was put forward , Has been associated with computer vision .
20 century 50 years ： A typical application scenario of the proposed perceptron algorithm , It is used to acquire the image from the image sensor 20X20 Pixel letters for recognition .
20 century 90 years ： Machine learning algorithms have ushered in a “ blowout ” Type development period .
（ With the development of more machine learning algorithms , Machine learning has become an important tool in the field of computer vision , It is mainly used in image detection 、 Identification and classification .）
（ It is worth mentioning that , Face recognition also ushered in a climax of research at this time . however , The time point that can really be regarded as a new starting point of computer vision is 2012 year .）
here we are 21 century , Computer vision has become a big research category of computer science .
（ International Conference on computer vision and pattern recognition （CVPR）、 International Conference on computer vision （ICCV） And other top-level conferences in the field of computer vision have also become an annual event in the field of artificial intelligence , It plays an important role in the field of computer science .
………………
The birth of deep learning has opened a new research field for machine learning .
After that , Deep learning has also become a powerful means to study computer vision , In areas such as face recognition 、 Object detection and other fields shine .
therefore , The birth of deep learning , especially AlexNet The implementation of the , It is also considered as a new starting point for the development of computer vision .
The application of computer vision ：
1. Image classification ：（Image Classification, It refers to the identification of the overall content of the picture , Output category label （ contain Coarse grained classification And Fine grained classification ）
· Digital classification
· Scene classification
· Face recognition classification
· Classification of animals and plants
· Vehicle type classification
· Food classification
· Art classification
· edge detection
……
It can be used to develop photo recognition applications （ Such as ： Plant recognition APP etc. ）, Search the image search engine （ Such as ：‘ Baidu knowledge map ’ etc. ）
2. Image segmentation ：（Object Segmentation, It refers to the pixel level classification of semantic objects of pictures , And output the outline, etc （ Including semantic segmentation 、 Instance segmentation 、 Panoramic segmentation ）
· Scene segmentation
· People are like cutouts
· Organizational segmentation
· Remote sensing image
……
It can be used for matting products （ Such as ： Certificate photo generation ）, Medical images （ Organ segmentation software ） Other fields
3. object detection ：（Object Detection, It refers to locating the target of the picture , And output specific categories ）
· Face detection
· Pedestrian detection
· Vehicle detection
· Text detection
· Sign detection
· defect detection
· Target count
……
Can be used for security monitoring （ Geling deep pupil security monitoring system ）, Industrial manufacturing （ Product appearance defect detection system ） Other fields
4. target recognition （ Text features ）：（ It refers to the recognition of words and signs in various scenes ）
· Document recognition
· ID card identification
· Ticket identification
· Bank card identification
· License plate recognition
· Invoice identification
· Express order identification
· Instrument panel reading recognition
It can be used for character recognition （ Such as ：OCR System ）, Bill entry （ Such as ： Invoice scanner ） Other fields
5. target recognition （ biosignature ）：（ It refers to the face 、 Human body related biometrics ）
· Face recognition
· Body recognition
· Gesture recognition
· fingerprint identification
· Iris recognition
· Gait recognition
……
It can be used for attendance payment （ Such as ： Face attendance machine ）, Social games （ Such as ：AR game ） Other fields
6. Target tracking ：（Object Tracking, It refers to tracking semantic objects in video ）
· Vehicle tracking
· Pedestrian tracking
· Animal tracking
· Gesture tracking
· Face tracking
……
Can be used for security monitoring （ Such as ： Key crowd monitoring ）, Autopilot （ Such as ： Front car tracking ） Other fields
7. Image quality analysis and aesthetic evaluation ：（Image quality assessment and Photo aesthetics assessment, It refers to scoring the quality of the picture and the aesthetics of the picture ）
（ Image quality analysis ： Exposure 、 clarity 、 Color 、 Texture 、 The noise 、 Anti shaking 、 The flash 、 focusing 、 Artifact ）
· Quality score
· Aesthetic score
· Composition optimization
……
It can be used for picture display （ Such as ： Search engine image sorting ）, Automatic composition recommendation （ Such as ： Recommendations for camera composition optimization ） Other fields
8. Image noise reduction and restoration ：（Image Denoising, To remove noise from a picture 、 Defects, etc , Improve the quality of photos ）
· Medical image denoising
· Photographic image noise reduction
· Image to rain
· Watermark removal
· Image shadow removal
· Defect repair
· Beautiful face
……
It can be used for photographic image processing （ Such as ： Noise reduction of night scene image ）, Medical image denoising （ Such as ：MRI Image denoising ） Other fields
9. Image enhancement ：（Image Enhancement, Refers to the contrast of the picture 、 Tone and other styles are enhanced , Make it have higher aesthetic value or benefit other visual tasks ）
· Night scene image enhancement
· Defog algorithm
· Underwater image enhancement
· Aesthetic enhancement
……
It can be used as a post photography tool （ Such as ：snapseed） Other fields
10. Image deblurring and super resolution ：（Image deblur and image super-resolution, It refers to removing image blur and improving image resolution respectively ）
· Image de blur
· Image super-resolution
……
It can be used to improve the resolution of old photos （ Such as ：‘ You and I were ’）, Video jitter repair （ Such as ：‘After effects’） Other fields
11. Image translation and stylization ：（Neural style transfer and image to image translation, It refers to image stylization based on neural network and image translation tasks ）
· Oil painting style
· Animation style
· Image coloring
· Line color
· Font migration
……
It can be used to make artistic style drawings （ Such as ：Prisma）, Animation animation creation （ Such as ：‘PaintsChainer’） Other fields
12. Image generation ：（Image synthesis, Refers to the generation of new non-existent images ）
· Image generation
· Data to enhance
· Data simulation
……
It can be used to make data sets （ Such as ：NVIDIA HD face dataset ） Other fields
13. Three dimensional reconstruction （ Human face ）：（3D Face/Body reconstruction, It refers to 3D face reconstruction based on 2D image / human body ）
· Expression driven
· Human body drive
· Attitude editing
· Virtual anchor
· Key positioning
· Virtual make-up test
……
It can be used for medical treatment and Cosmetology （ Such as ：3D Cosmetic surgery ）, Game character customization （ Such as ： Netease game ） Other fields
14. Three dimensional reconstruction （ Common scenarios ）：（Scene reconstruction, It refers to the reconstruction of 3D scene based on 2D image ）
· Depth estimation
· Stereo vision
·SFS
·SFM
· Point cloud post processing
· Vision SLAM
……
It can be used for automatic driving （ Such as ： Sweeping robot ）, Virtual reality （ Such as ：AR game ） Other fields
15. image editing （ Human face ）：（Face/Person Attribute Edit, It refers to the face / Edit and modify the relevant attributes of the human body
· Face Age editor
· Face Expression editor
· Face Attitude editing
· Face Make up editor
· Face Gender editors
· Face Identity editor
· human body Attitude editing
……
It can be used for face editing （ Such as ： Video face change ）, Virtual fitting （ Such as ： Mobile fitting room ） Other fields
16. image editing （ Universal ）：（Image Edit, It refers to editing and modifying the contents of the image ）
· Deep editing
· Background fusion
· Texture repair
· Interactive Editing
……
Can be used for picture refocus （ Such as ：Focus） Other fields
17. Video processing ：（ A video is a picture of many frames , Video processing can be regarded as processing of a single picture of many frames ）
· Video classification
· behavior analysis
· Video segmentation
· Video generation
· Video prediction
· Video Retrieval
· Optical flow estimation
· Key frame extraction
· Video description
· Video face change
……
Available security monitoring （ Such as ： Video abstract retrieval system ）, Post content creation （ Such as ： Intelligent video clip ） Other fields
（18. Picture splicing ）
（ Continuous updating 、 Improving ...）
