Multimedia Laboratory

Projects / Face Sketch Synthesis and Recognition

* Artists have a fascinating ability to capture the most distinctive characteristics of human faces and depict them on sketches. Although sketches are very different from photos in style and appearance, human beings often can easily recognize a person from his sketch. In the topic of face sketch synthesis and recognition, we design computer programs to also have such amazing abilities.

Face Photo‐Sketch Synthesis and Recognition
X. Wang and X. Tang, IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), Vol. 31, pp. 1955-1967, 2009.

In this paper, we propose a novel face photo-sketch synthesis and recognition method using a multiscale Markov Random Fields (MRF) model. Our system has three components: 1) given a face photo, synthesizing a sketch drawing; 2) given a face sketch drawing, synthesizing a photo; and 3) searching for face photos in the database based on a query sketch drawn by an artist. It has useful applications for both digital entertainment and law enforcement.

Coupled Information-Theoretic Encoding for Face Photo-Sketch Recognition
W. Zhang, X. Wang, and X. Tang, in Proceedings of IEEE Computer Society Conference on Computer Vision and Patter Recognition (CVPR) 2011

Automatic face photo-sketch recognition has important applications for law enforcement. Recent research has focused on transforming photos and sketches into the same modality for matching or developing advanced classification algorithms to reduce the modality gap between features extracted from photos and sketches. In this paper, we propose a new inter-modality face recognition approach by reducing the modality gap at the feature extraction stage. A new face descriptor based on coupled information-theoretic encoding is used to capture discriminative local face structures and to effectively match photos and sketches.

Face Sketch Recognition
X. Wang and X. Tang, IEEE Transactions on Circuits and Systems for Video Technology (CSVT), Special Issue on Image- and Video- Based Biometrics, Vol. 14, No. 1, pp. 50-57, January, 2004.

Automatic retrieval of face images from police mug-shot databases is critically important for law enforcement agencies. It can effectively help investigators to locate or narrow down potential suspects. However, in many cases, the photo image of a suspect is not available and the best substitute is often a sketch drawing based on the recollection of an eyewitness. In this paper, we present a novel photo retrieval system using face sketches. By transforming a photo image into a sketch, we reduce the difference between photo and sketch significantly, thus allowing effective matching between the two. Experiments over a data set containing 188 people clearly demonstrate the efficacy of the algorithm.

Back to Top

Share This Page

© 2013 Multimedia Laboratory

The Chinese University of Hong Kong 香港中文大学

Site Credits