Base class for all facemark models

Facial landmark detection is a useful algorithm with many possible applications including expression transfer, virtual make-up, and facial puppetry. This class implements an API for facial landmark detector. Two kinds of algorithms are implemented including active appearance model [AAM] and regressed local binary features [LBF] able to work in real-time.

All facemark models in OpenCV are derived from the abstract base class Facemark, which provides a unified access to all facemark algorithms in OpenCV.

To utilize the Facemark API in your program, please take a look at the opencv tutorials, as well as mexopencv samples.

Description

Facemark is a base class which provides universal access to any specific facemark algorithm. Therefore, the users should declare a desired algorithm before they can use it in their application.

The typical pipeline for facemark detection is listed as follows:

• (Non-mandatory) Set a user defined face detection using cv.Facemark.setFaceDetector. The facemark algorithms are desgined to fit the facial points into a face. Therefore, the face information should be provided to the facemark algorithm. Some algorithms might provides a default face recognition function. However, the users might prefer to use their own face detector to obtains the best possible detection result.
• (Non-mandatory) Training the model for a specific algorithm using cv.Facemark.training. In this case, the model should be automatically saved by the algorithm. If the user already have a trained model, then this part can be omitted.
• Load the trained model using cv.Facemark.loadModel.
• Perform the fitting via the cv.Facemark.fit.

Example

Here is an example of loading a pretrained model (you should provide full paths to files specified below):

obj = cv.Facemark('LBF', 'CascadeFace','lbpcascade_frontalface.xml');
obj.loadModel('lbfmodel.yaml');
img = imread('lena.jpg');
faces = obj.getFaces(img);
landmarks = obj.fit(img, faces);
for i=1:numel(faces)
    img = cv.rectangle(img, faces{i}, 'Color','g');
    img = cv.Facemark.drawFacemarks(img, landmarks{i});
end
imshow(img)

Example

Here is an example of training a model:

% filename to save the trained model
obj = cv.Facemark('LBF', 'ModelFilename','ibug68.model');
obj.setFaceDetector('myFaceDetector');

% load the list of dataset: image paths and landmark file paths
[imgFiles, ptsFiles] = cv.Facemark.loadDatasetList(...
    'data/images_train.txt', 'data/points_train.txt');

% add training samples
for i=1:numel(imgFiles)
    img = imread(imgFiles{i});
    pts = cv.Facemark.loadFacePoints(ptsFiles{i});
    obj.addTrainingSample(img, pts);
end
obj.training();

References

[AAM]:

G. Tzimiropoulos and M. Pantic, "Optimization problems for fast AAM fitting in-the-wild," ICCV 2013.

[LBF]:

S. Ren, et al. , "Face alignment at 3000 fps via regressing local binary features", CVPR 2014.