Planar 3D Object Tracking
Date: 08-30-12 10:06AM | Category: Tutorials - 3D

3D Object tracking is a fairly common effect in film and television today. It's really any instance where a CG model is inserted onto a moving object within a shot. A classic example is anytime you see a talking animal, a CG head has been composited on top of the live shot. Another (and much more advanced) example is the Iron Man suit from the films. Infact, here's a photo I found the other day showing the live footage, and the suits that were digitally inserted:

As I stated the effect is pretty common in the visual effects industry, yet it seems to be elusive for the amateur effects artist. To my knowledge (and please feel free to email me if I'm wrong) there's only 1 piece of affordable software that can handle the task: Syntheyes.

Syntheyes is similar to all of the matchmoving software packages available today, except that it can "reverse" the 3D tracking from the movement of the camera in a shot, to the movement of an object within that shot.

I have a small project I'm working on right now involving some object tracking, and I'll post a quick preview at the end of the post. Basically I'm attempting a few shots that involve inserting a small device into the palm of my hand, very similar to the respulsor cannon from Iron Man (which is certainly where I got the inspiration.) The problem is that the palm of your hand is a very small surface and not nearly large enough to obtain the tracking data necessary for solid 3D track...but worry not, I've come up with a pretty unique solution that can solve this issue.

My solution was that I need to "double-track" the footage...meaning I want to track a small set of markers on my palm in order to digitally insert a second, larger set of markers that encompass my entire hand, then track that in Syntheyes. It may sound complicated, but its really a pretty simple notion and luckily I've found the perfect method to accomplish just that.

My first task was obviously the initial track. I need a small set of markers on my palm and more importantly a piece of software that can handle a tracking area that small. So I did some searching for some alternate tracking solutions and came up with Mocha. Mocha is different from other tracking software in that it uses planar tracking, not markers or reference points. So instead of creating a set of dots all over my hand, I just need a solid bright piece of paper in the middle of my palm, and Mocha can track that quite well. Infact one of Mocha's main uses is screen replacement shots. For example, Mocha can track the rectangular area of a cell phone screen and allow the artist to digitally replace that screen with any digital layer.

So using Mocha, I'm able to track a small planar surface, then digitally insert an expanded set of markers that Syntheyes can now use to produce an accurate 3D track. Another bonus with this technique is that I can disable any motion blur on my larger Mocha tracking layer which makes life much easier in Syntheyes.

As promised, here's a quick clip showing the original footage, the Mocha track, and finally the CG element derived from Syntheyes:


Loading the player ...

That's just a "teaser"/proof of concept shot, I've found a decent location and plan to shoot a few things in the next few weeks. Check back to see the final sequence, which should be pretty cool :).