Multimedia Lab >> Research >> Projects
IDFace: Identification and Detection of Face
Automatic face identification and detection has great potential in a large array of application areas, including banking and security system access authentication, video surveillance, mugshot matching for law enforcement, duplicate ID card identification, face-based video compression for video conferencing, and multimedia information retrieval. In this project, we will develop a core set of essential tools for face recognition system development. In addition, several prototype application systems utilizing the developed software packages will be implemented to demonstrate the superiority of the technology.
3D Object Reconstruction from Single 2D Line Drawings
Multimedia applications extensively use 3D models. A 2D line drawing is the simplest and most straightforward way of illustrating a 3D object. It is very helpful if such a drawing can be used for generating a 3D model directly. Unfortunately, current CAD tools cannot do it, denying designers a convenient means of input. Therefore, it is highly desirable to develop algorithms that can convert a design sketch into a 3D model. Given a line drawing representing a 3D object, our approach to the conversion problem is finding face topology first and then doing 3D geometry reconstruction. Now we have developed a tool that can reconstruct complex 3D objects with planar faces and common 3D objects with curved faces from single 2D line drawings.
 
Image Segmentation by Iterative Optimization

Image segmentation plays an important role in computer vision and image analysis. We formulate image segmentation as a probability maximization problem and solve it using an iterative optimization scheme. Our algorithm can automatically segment an image into regions with relevant textures or colors without the need to know the number of regions in advance. Its results match image edges very well and are consistent with human perception. (The Program)

 
Chinese News Video Retrieval Engine

As digital video libraries and archives of immense size are becoming available over data networks, efficient video retrieval and browsing have become crucially important. In this project, we construct a Chinese news video retrieval engine using a range of newly developed technology, including automatic news video story parsing, video caption recognition, and speech recognition.

 
Video-based Chinese Character Recognition
We develop a handwritten Chinese character recognition system using a video camera. The system combines the advantages of both online and offline approaches. It allows users to write on any regular paper just like using an off-line system. At the same time, using a video camera attached on the computer, the system can capture the stroke temporal information similar to an on-line system.
 
Real Time Feature Based 3-D Deformable Face Tracking
We develop a novel framework for 3D tracking of the non-rigid face deformation from a single camera. The proposed algorithm integrates three types of features which discriminate face deformation across different views: 1) the semantic features which provide constant correspondences between 3D model points and major facial features; 2) the silhouette features which provide dynamic correspondences between 3D model points and facial silhouette under varying views; 3) the online tracking features that provide redundant correspondences between 3D model points and salient image features. In order to estimate the high dimensional 3D deformation parameters, we develop a hierarchical parameter estimation algorithm to robustly estimate both rigid and non-rigid 3D parameters. (Demo)
Performance Driven Face Animation via Non-rigid 3D Tracking
In this demo, a performance driven 3D face animation system is proposed. The proposed system consists of two key components: a robust non-rigid 3D tracking module and a MPEG4 compliant facial animation module. Firstly, the facial motion is tracked from source videos which contain both the rigid 3D head motion (6 DOF) and the non-rigid expression variations. Afterward, the tracked facial motion is parameterized via estimating a set of MPEG4 facial animation parameters(FAP). As the final step, these FAP values are transferred to the MPEG4-compliant face model for the animation purpose. The proposed tracking and animation system has a strong generalization ability and can be used in the indoor environment with no additional assumptions. (Paper and Demos --- FFmpeg codec libary is needed for video rendering. )