Facial motion capture

Facial motion capture is the process of electronically converting the movements of a person's face into a digital database using cameras or laser scanners.

Uses your system voice; voice and rate in settings.

Snapshot

≈ 5 min read

Names in other languages

No names in other languages are held for this entry yet.

Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.

Facial motion capture is the process of electronically converting the movements of a person's face into a digital database using cameras or laser scanners. This database may then be used to produce computer-generated imagery (CGI), computer animation for movies, games, or real-time avatars.

Because the motion of CGI characters is derived from the movements of real people, it results in a more realistic and nuanced computer character animation than if the animation were created manually. A facial motion capture database describes the coordinates or relative positions of reference points on the actor's face.

The capture may be in two dimensions, in which case the capture process is sometimes called "expression tracking", or in three dimensions. Two-dimensional capture can be achieved using a single camera and capture software. This produces less sophisticated tracking, and is unable to fully capture three-dimensional motions such as head rotation.

Three-dimensional capture is accomplished using multi-camera rigs or laser marker system. Such systems are typically far more expensive, complicated, and time-consuming to use. Two predominant technologies exist: marker and markerless tracking systems.…

Facial motion capture is the process of electronically converting the movements of a person's face into a digital database using cameras or laser scanners. This database may then be used to produce computer-generated imagery (CGI), computer animation for movies, games, or real-time avatars.

Because the motion of CGI characters is derived from the movements of real people, it results in a more realistic and nuanced computer character animation than if the animation were created manually. A facial motion capture database describes the coordinates or relative positions of reference points on the actor's face.

The capture may be in two dimensions, in which case the capture process is sometimes called "expression tracking", or in three dimensions. Two-dimensional capture can be achieved using a single camera and capture software. This produces less sophisticated tracking, and is unable to fully capture three-dimensional motions such as head rotation.

Three-dimensional capture is accomplished using multi-camera rigs or laser marker system. Such systems are typically far more expensive, complicated, and time-consuming to use. Two predominant technologies exist: marker and markerless tracking systems.

Facial motion capture is related to body motion capture, but is more challenging due to the higher resolution requirements to detect and track subtle expressions possible from small movements of the eyes and lips. These movements are often less than a few millimeters, requiring even greater resolution and fidelity and different filtering techniques than usually used in full body capture.

The additional constraints of the face also allow more opportunities for using models and rules. Facial expression capture is similar to facial motion capture. It is a process of using visual or mechanical means to manipulate computer-generated characters with input from human faces, or to recognize emotions from a user.

Technology

Digital video-based methods are becoming increasingly preferred, as mechanical systems tend to be cumbersome and difficult to use. Using digital cameras, the input user's expressions are processed to provide the head pose, which allows the software to then find the eyes, nose and mouth. The face is initially calibrated using a neutral expression.

Then depending on the architecture, the eyebrows, eyelids, cheeks, and mouth can be processed as differences from the neutral expression. This is done by looking for the edges of the lips for instance and recognizing it as a unique object. Often contrast enhancing makeup or markers are worn, or some other method to make the processing faster.

Like voice recognition, the best techniques are only good 90 percent of the time, requiring a great deal of tweaking by hand, or tolerance for errors. Since computer-generated characters don't actually have muscles, different techniques are used to achieve the same results.

Some animators create bones or objects that are controlled by the capture software, and move them accordingly, which when the character is rigged correctly gives a good approximation. Since faces are very elastic this technique is often mixed with others, adjusting the weights differently for the skin elasticity and other factors depending on the desired expressions.

Usage

Several commercial companies are developing products that have been used, but are rather expensive. It is expected that this will become a major input device for computer games once the software is available in an affordable format, but the hardware and software do not yet exist, despite the research for the last 15 years producing results that are almost usable.

See also

Eye tracking Computer facial animation Deepfake Facial recognition system Facial Action Coding System Uncanny valley

Carnegie Mellon University Delft University of Technology Intel Sheffield and Otago

Text derived from https://en.wikipedia.org/w/index.php?oldid=1368591123 under CC BY-SA 4.0; modifications have been made.

Sources cited in this revision

5 named source(s) extracted from the cited revision. A source with no author is shown by its publication; nothing is invented.