Kevin Karsch pulls up two photos on his Macbook. The first shows an unremarkable living room with light spilling through partially shaded windows.
A plush Pokemon doll stares down from a bookshelf. In the foreground, an empty beer bottle stands on a coffee table.
With a few short keystroke, Karsch switches to the other photo. It’s a nearly identical shot of the room. There’s just one added detail: a 7-foot-tall marble statue of an angel.
The angel isn’t real. It’s a 3-D model, and the statue’s digital insertion took all of 10 minutes.
Karsch, a University Ph.D. student in computer science, has been working on an image-rendering program that can make picture-perfect manipulations that, when revealed, can be as unnerving as they are breathtaking. He’s been working on this rendering tech since 2009, and it has garnered some serious investor attention. And already he’s turned to the next step: video.
Get The Daily Illini in your inbox!
“The idea of inserting objects into digital content has been around for some time. But for our technology, the goal of it is to make things much more efficient and much easier for the user,” Karsch said.
The underlying novelties of the system are its speed and how it handles light, Karsch explained. In the pictures above, the angel isn’t gaining its photorealism from anything inherent in the 3-D model of the statute itself.
Rather, it’s the way light coming through the windows flows over the uneven surfaces of the statue’s robe, the way the shadow’s gradient lightens and darkens the angel’s face, and the way all the lighting conforms to the angle of the light coming through the window.
The rendering program works by first processing the 2-D picture into something that can be represented as three-dimensional space. To accomplish this, the user draws lines over vanishing points, parallel lines and light sources within a 2-D image.
The payoff for all this tagging and line-dragging is that the algorithm can recognize the depth and three-dimensional layout of the room.
So when Karsch places the angel statue in the picture, the program can model the effect of the light hitting the statue and even the effect of the light reflected by the statue’s color.
The statue looks impressive, to be sure, but for Karsch, the real leap forward is how relatively simple it was to produce. With a little instruction, even a rank amateur could produce the image in little more than 10 minutes.
“Photoshopping costs a lot of money to have the program, and it could take years to learn,” Karcsh said, comparing the results of his rendering software to that of the popular Adobe software.
The next step for Karsch and his colleagues is to use their algorithm to make the same kind of manipulations for video, and he explained that the program would allow artists to easily insert objects and effects into a scene without ever needing them to be physically present. Even legendary scenes like the historical footage mash-ups found in “Forrest Gump” could be easily replicated.
“That scene probably took one or more artists several hours, if not days, to produce,” Karsch said. “Hopefully, with our technology, that can be done in just a few minutes or maybe an hour.”
But with great ease and availability comes the question of its popular use, and Karsch thinks there may come a point where literally anybody can use this kind of technology.
Karsch is unsure of what this could mean for the general idea of validity or realism in photography, but film critic Roger Ebert doesn’t think the spread of accessible image manipulation software means much at this point.
In fact, it’s old news:
“Years ago, backgrounds were matte paintings. Now they’re CGI. Film has always employed trickery,” Ebert wrote in an email, adding: “I think today’s audiences are so savvy and cynical they assume they’re looking at CGI — even if they’re not.”
Rather, Karsch’s software is just one more link in a chain stretching back more than a hundred years to the days of the Lumière brothers and George Méliès (who recently experienced a revival as a supporting character in Martin Scorsese’s “Hugo”).
The irony is that the quest for this realism is accomplished by increasingly sophisticated tools of illusion, explained James Hay, a professor of media and cinema studies at the University.
“You could say that the history of media productivity, of film and TV, has been toward greater and greater realism,” Hay said. “Technologies of visualization are increasingly about overcoming a perception that the earlier technology is unrealistic in some ways.”
Karsch thinks the balance between technology and human agency is decreasing.
Most animation needs some kind of motion-capture to ground the animation in believable movement. But the with the technological tools of illusion growing more sophisticated, the balance is moving toward a day when no image we see could be construed as “real.”
Whether we’ve already reached that day is an interesting question, but Hay suggested that we all keep in mind René Magritte’s famous painting “The Treachery of Images,” which shows a detailed picture of a pipe with the words “This is not a pipe” written underneath.
Magritte’s point with the iconic pipe was that an image can never be the real thing. But for innovators like Karsch, perhaps letting people believe the pipe is the real accomplishment.