Motion capture
|
|
MotionCapture.jpg
Motion capture, or mocap, is a technique of digitally recording the movements of real things — usually humans — it originally developed as an analysis tool in biomechanics research, but has grown increasingly important as a source of motion data for computer animation. In this application, it has been widely used for both cinema and video games.
A person or animal wears a black costume with white dots or ping pong balls sewn to it at the joints of the body (elbows, knees, shoulders, etc.). Animals are sometimes naturally black color, so they don't have to wear a costume. The person/animal moves/acts in front of one or more cameras. The movie tranfers from the camera into a computer containing a motion capture program. Motion capture programs are usually included on computer animation CDs. The motion capture program only sees the white dots or white balls. It excludes everything else in the movie scene. The program connects the dots with lines. The dots serve as the joints, and the lines serve as the bones. Now you have an animated stick man (or stick animal) moving the same way as the actor. This stick figure becomes the skeleton of a computer animated character. The character copies precisely the exact movements of the human or animal actor. This is a big shortcut in animation, which normally requires the animator to draw each frame by hand (in traditional animation) or manually animate (in non-mocap computer animation). It saves a lot of animation time, thus making the animation cheaper, and creates more natural movement than manual animation.
Although there are many different systems for capturing motion data, one technique contains optical systems, which employ photogrammetry to establish the position of an object in 3D space based on its observed location within the 2D fields of a number of cameras. These systems produce data with 3 degrees of freedom for each marker, and rotational information must be inferred from the relative orientation of several markers. A related technique Match Moving can derive 3D camera movement from a single 2D image sequence without the use of photogrammetry.
Another technique is the use of a magnetic systems, which directly indicate the position and orientation of the sensors with respect to a transmitter. Since the sensor output is 6DOF, useful results can be obtained with a much smaller number of sensors than you would require markers in an optical systems. The major restrictions are that the response is quite nonlinear - epecially near the edges of the capture area, and that the wiring from the sensors tends to preclude extreme movements on the part of the performers.
For the optical systems, the commonest approach is to use passive reflective markers, and to identify each marker from its relative location, possibly with the aid of kinematic constraints. These sort of systems are the most popular for entertainment applications - mostly because they can track a large number of markers and expanding the capture area just involves the addition of more cameras. Two of the largest manufacturers of this type of system are Oxford Metrics [1] (http://www.vicon.com/) and Motion Analysis [2] (http://www.motionanalysis.com)
The other type of optical system uses active markers, this has the advantage that there is no doubt about which marker is which, but are most useful with small numbers of markers - in general, the overall update rate drops as the marker count increases. These systems are at their best when used for capturing a subject such as a single arm or leg - since few markers are needed, the update rate can be kept high. As a result, these systems are popular in the biomechanics market. One such is the Northern Digital Optotrak [3] (http://www.ndigital.com).
With the magnetic systems, there is a distinction between "DC" and "AC" systems - one uses square pulses, and the other uses sine wave pulses. In practice, both systems seem to work about as well as each other - it seems likely that the biggest factor behind the different choice of techniques is based on who holds which patent, rather than any technical considerations. Two of the better known makers of these kind of systems are Ascension technology [4] (http://www.ascension-tech.com/) and Polhemus [5] (http://www.polhemus.com/)
A motion capture session only records the movements of the actor, not his visual appearance. These movements are recorded as animation data which is then "mapped" onto a 3D model (which can be of a normal human, giant robot, or anything else) that was created by a computer artist, and the model can then be made to perform the same movements that were recorded. This can be compared to the earlier technique of rotoscope, where the visual appearance of the motion of an actor was filmed; that film was then used, frame by frame, as a guide or pattern for the motion of a hand-drawn animated character.
| Contents |
The procedure
In the motion capture session itself, an actor, often a martial artist, dancer, or mime, wears a leotard with a number of reflective markers taped or glued to specific points all over his body. At least two cameras, and preferably an array of cameras, film the actor as he acts, or performs specific motions. The cameras report to a computer the exact position of each reflective marker, many times per second.
In particularly complex scenes that are shot with particularly expensive equipment, a motion control camera can pan, tilt, or dolly around the stage while the actor is performing. These camera motions are also tracked meticulously and fed to the computer; or, a computer controlling the camera motion has already been programmed with the motion control data, and the camera meticulously follows the directions of this computer.
Instead of such an optical system, a magnetic system can be used, in which the actor wears a number of sensors which detect a nearby magnetic field and transmit data on each sensor's inferred 3D position to the computer.
After the recording session is complete, a computer post-process this mass of data and determines the exact movement of the actor, as inferred from the 3D position of each marker at each moment. Mocap data is notorious for requiring a human to spend a great deal of time to "clean up" the data. A single sensor mis-reading might cause the computer to believe that the actor's arm was pointed straight up into the air for a fraction of a second, for example, when it was not.
After post-processing, the computer exports animation data, which computer animators can associate with a 3D model and then manipulate using normal computer animation software such as Maya or 3D Studio Max. If the actor's performance was good and the software post-processing was accurate, this manipulation is limited to placing the actor in the scene that the animator has created and controlling the 3D model's interaction with objects. The animator does not have to move that particular model's arms and legs around manually -- the movement is already present in the animation data.
Advantages
Mocap offers several advantages over traditional computer animation of a 3D model:
- Mocap can take far fewer man-hours of work to animate a character. One actor working for a day (and then technical staff working for many days afterwards to clean up the mocap data) can create a great deal of animation that would have taken months for traditional animators.
- Mocap can capture secondary animation that traditional animators might not have had the skill, vision, or time to create. For example, a slight movement of the hip by the actor might cause his head to twist slightly. This nuance might not be imagined by a traditional animator, but it would be captured accurately in a mocap session, and this is the reason that mocap animation often seems shockingly realistic compared with traditionally animated models. Incidentally, one of the hallmarks of rotoscope in traditional animation is just such secondary "business."
- Mocap can accurately capture difficult-to-model physical movement. For example, if the mocap actor does a backflip while holding nunchucks by the chain, both sticks of the nunchucks will be captured by the cameras moving in a perfectly realistic fashion. A traditional animator might not be able to physically simulate the movement of the sticks adequately due to other motions by the actor.
On the negative side, mocap data is very difficult to manipulate once captured and processed, and if the data is wrong, it is often easier to throw it away and reshoot the scene rather than trying to manipulate the data as could be done easily with traditionally animated computer models.
Another important point is that while is common and comparatively easy to mocap a human actor in order to animate a biped model, applying it motion capture to animals like horses or even things like cars or trucks is much harder. Humans constitute the vast majority of mocap actors, because humans are patient, will take direction, and will not attempt to lick markers off their bodies, unlike dogs, cats, tarantulas, and so forth. Motion capture may not even be suitable for animation of bipeds with superhuman powers, due to the mocap actor's inability to fly through the air, morph his fists into hammers, and so forth.
Motion capture equipment is expensive. It can cost many tens of thousands of dollars for the digital video cameras, lights, software, and staff to run a mocap studio, and this technology investment can become obsolete every few years as better software and techniques are invented. Some large movie studios and video game publishers have established their own dedicated mocap studios, but most mocap work is contracted to individual companies that specialize in mocap.
Applications
Video games are increasingly using motion capture animation for such animation as the movement of a football or basketball player or the combat moves of a martial artist.
Movies have increasingly used motion capture animation as computer-generated animation has replaced traditional cell animation, and as it has become exotically fashionable to utilize completely computer-generated creatures, such as Gollum and Jar-Jar Binks, in live-action movies.
Related techniques
Facial motion capture is also sometimes utilized to digitally capture the complex movements in a human face, especially while speaking. This is generally performed with an optical setup using a single camera at close range, with small reflective markers glued or taped to the actor's face.
Performance capture is a further development of these techniques, where both body motions and facial movements are recorded. This technique was used in making of The Polar Express, where all actors were animated this way.
Due to current technology limitations, a motion capture session only records the movement of a few key points on the actor's body, where the sensors or reflective markers are placed. One might extrapolate that future technology might include full-frame imaging from many camera angles that would record the exact position of every inch of the actor's body, clothing, and hair for the entire duration of the session, resulting in a higher resolution of detail than is possible today.
An early example is the Universal Capture technique developed for the Matrix movies. It allows to process a multiple-angle high-definition footage of an actor's performance and algorithmically generate a 3D recording without requiring use of special markers. The resulting recording can be displayed from any angle with any lighting conditions. The images from the original recording are used as textures.
An alternative approach was developed by a Russian company VirtuSphere, where the suit contains internal sensors recording the angular movements of the joins, removing the need for external cameras and other equipment. Even though this technology could potentially lead to much lower costs for mocap, the company failed to commercialise it.
External links
- [6] (http://www.virtualcinematography.org/) - several papers on Universal Capture use in Matrix films
de:Motion Capturing
fr:Motion capture
ja:モーションキャプチャ
it:Cattura del movimento