The objects in the bottom row of images have been placed in the original photos after the fact. What’s so impressive though, is that lighting and color characteristics of the objects and scenes interact with each other. That is, the objects appear to be lit by the existing lighting of the scene and, at the same time, cast new shadows in the scene.
Check out the video below for a more in-depth look.
With a single image and a small amount of annotation, our method creates a physical model of the scene that is suitable for realistically rendering synthetic objects with diffuse, specular, and even glowing materials while accounting for lighting interactions between the objects and the scene.
This technology is incredible. It’s also a bit scary for image manipulation concerns. What blows my mind, however, is the simplicity and ease of creating a 3D space from a 2D image whereby you can insert a moving object and ultimately turn a still image into a live action scene for your synthetic creation to interact inside.
Who else thinks Adobe needs to snatch this tech up ASAP?