3D Space and Alignment in After Effects

I mentioned earlier that I’ve been working on matching camera movements from After Effects in the physical world. Since that earlier post, my experiments ran into a bit of trouble: I found that it was extraordinarily difficult to get a physical model to line up proportionally to one constructed in 3D within After Effects.

In this post, I’ll detail the failed experiments that lead me to realize that I didn’t understand the relationship between the real physical world in front of the camera and the virtual 3D world inside of After Effects. Then, I’ll proceed to explain the controlled experiment I conducted to nail down this relationship and my current, much improved, understanding of it.

Building on my earlier success with the servo, I set about reproducing a two-axis move. I borrowed an X-Y table from Shahar Zaks and set about figuring out how to control it:

Motion control rig with exacti and janky tripod

Once I was up-and-running with Shahar’s electronics, I put together a simple shot in After Effects: a slow push-in on the Stanford Research Institute through the window of which you would see Doug Englebart and the rest of the Augment Research Center’s staff working on early computers. Here’s a draft (i.e. low-res) render of the animation:

333 Ravenswood test comp (augment around a table) from Greg Borenstein on Vimeo.

My goal was to replace the architectural drawing with a miniature 3D-printed version that I would place in front a green screen, shoot with the moving camera, and live-composite into this animation, giving it something of the texture or early special effects movies from the 60s and 70s. And allowing the viewer to see how the image was constructed from physical objects in real time.

333 Ravenswood 3D print

While I was waiting for the miniature to come back from the 3D printer, I did some experiments with a scale-model I constructed out of blue foam. I exported the camera position data from my After Effects Composition in the manner I described in my previous post and played it back through a Processing sketch which communicated with an Arduino to move the real video camera in a manner that would reproduce the virtual camera move from my animation:

Motion-controlled test shot for tracking from Greg Borenstein on Vimeo.

You can see some obvious flaws in this footage: some camera jitter from the X-Y table’s stepper motors and some back-and-forth jerking at the end of the shot when the table runs into the side of its moveable area. However, when I tried to composite this shot in with my After Effects animation, I discovered much more dramatic problems: the camera seemed to be moving faster in the physical version than the virtual one and it was impossible to arrange it in the virtual space so that it fit the composition.

I discovered two problems with the footage. The first one was simple to solve: I was recording video at 30 frames/sec and the animation had been rendered at 30 frames/sec, but in between I was playing back the camera position information at 24 frames/sec (because that was the frame rate I happened to have set for my Processing sketch). Once I tried the experiment again with that problem corrected, I discovered a deeper one, the parallax was incorrect.

Parallax is the optical effect that causes objects that are further away from a given point of view to appear to move more slowly as the point of view shifts compared to objects that are closer. The distance between the physical camera and the miniature building was not proportional to the distance between the virtual camera and the digital building. And further these ratios didn’t reflect the size relationship between the physical building and its digital stand-in.

In order to get the physical shot to match the digital one, all of these proportions would need to come into alignment. But how to achieve that without an enormous, time consuming, and un-reproducible guess-and-check process?

After talking to Chris Kairalla extensively about the problem, I began to understand that what I really needed was to learn the proportion between the pixels that measure After Effects’ 3D world and the inches of the real one. Chris and I devised an experiment to determine exactly this proportion.

The experiment was as follows: print out some reference image, for example, a 1 inch grid. Place the grid in front of a camera on a tripod at, say, 3 feet and take a photograph. Then, move the camera back one foot and take a second photograph. Now, bring both of these photographs as well as the un-scaled grid into an After Effects composition. Make the un-scaled grid a 3D layer so that it is affected by virtual cameras within the comp. Leave the two photographs as 2D layers so they are not affected by the virtual cameras. Now introduce a virtual camera. Move it back away from the origin while viewing only the un-scaled grid and the photograph taken at 4 feet. When the un-scaled grid appears to match the picture perfectly, stop. Now, introduce a second camera. Hide the 4 foot picture and reveal the 3 foot picture. Move this camera back from the origin until the un-scaled grid appears to match the 3 foot picture. You will now have two measurements each of which corresponds to a known physical distance. If the measurements are consistent, you now have an answer to the relationship between pixels and inches: simply divide the pixel distance of the closer camera by 36 and the later one by 48. These should come out to about the same number.

Chris predicted the number would be either 72 because that is “a magic number for pixels” or 100 which would have been the sensible choice.

After some fiddling and false starts arising from trying to be too anal retentive about getting the match between layers (I forgot to take into account that my photo setup was imperfectly aligned and so I shouldn’t expect a pixel perfect match), I came up with evidence in favor of a 100px/inch correspondence. Here’s the output of rendering the comp arranged in the manner described above:

After Effects 3D alignment test 2 from Greg Borenstein on Vimeo.

Note here that I setup the un-scaled grid as a “difference layer” so that the places where it perfectly matched the photo’s grid would be black and the imperfect places would be white. The background field is gray because the exposure on the stills didn’t render the raw paper perfectly white.

With this result, we now know for sure: in After Effects 100 pixels equals 1 inch in the real world.

So now, with this new knowledge, I can go back and redesign my comp and my motion control setup in order to ensure that they are in scale with each other.

The one other possible wild card would be if my camera (a Canon T2i, by the way) somehow did something different when shooting video that affected the variables in this result. To test this possibility I made another comp in which I used similar techniques as I’ve already described to to match up the video I took at 3 and 4 feet with the stills taken from the same position. Thankfully, things came out as expected:

Test of matching still and video footage in After Effects from Greg Borenstein on Vimeo.

The video was smaller and had a wider aspect ratio (
actually making it a much better match for my comp, the way I had it set up), but once I overcame these differences via scaling, the “difference blend” test yielded these tight results confirming no other strange problems.

The next experiment will be to design a still shot that reproduces the proportional positions of a few objects in a comp and make sure that it composites correctly. After that, I’ll move on to a one-axis camera move and, eventually, build back up to a complex two-axis move like I attempted here.

This entry was posted in Art, Science. Bookmark the permalink.

One Response to 3D Space and Alignment in After Effects

  1. Shadrack says:

    Is there a software that can actually do all of this for you? What if u have a virtual or live environment and need to shoot a live subject that can actually fit into that environment seamlessly?

Leave a Reply

Your email address will not be published. Required fields are marked *