Smile like she means it

Smile like she means it

By Tracy Culumber

For years, people have tried to decipher the emotions hidden within Leonardo da Vinci’s masterpiece, the Mona Lisa. Some have speculated the painting’s subject is a man, or self-portrait of da Vinci. However, emotion and gender recognition software recently developed at the University has dispelled some of these theories with results that may give Mona Lisa a reason to smile.

This software, developed by Electrical and Computer Engineering Professor Thomas Huang and his students, determined the Mona Lisa is neither a man, nor a depiction of her creator. The software, developed at the Beckman Institute, 405 N. Mathews Ave., broke down her emotions into percentages and determined she is 83 percent happy in the painting.

Using databases containing hundreds of sample male and female facial expressions, Huang and his students have applied gender and emotion recognition algorithms to determine the results. They also compared Mona Lisa’s portrait to a sketch of da Vinci to analyze the differences in shape and proportion. Using standard and mean deviations between hundreds of faces in the database, they determined that her features were very dissimilar to those of her creator.

Nemanja Petrovic, a University alumnus and researcher for Google, Inc., worked with Huang on several journals, conferences and projects involving biometrics and image restoration. He called the Mona Lisa “an interesting example, but also a difficult example,” of this technology because the image is somewhat blurred and not totally realistic.

Huang and Dennis Lin, graduate student, echoed this sentiment, explaining that accurate results are difficult to find in paintings. Huang said two kinds of information could be implemented into the algorithms: the texture of the face and the shape of the facial features.

Get The Daily Illini in your inbox!

  • Catch the latest on University of Illinois news, sports, and more. Delivered every weekday.
  • Stay up to date on all things Illini sports. Delivered every Monday.
Thank you for subscribing!

Texture details, such as the tiny cracks in the paint itself, skew the results, so they were not applied to the various algorithms. For example, when texture was applied to the age algorithm, data suggested Mona Lisa was 60-years-old at the time of the painting, he said.

Huang and his students determined her age, gender and emotional state by documenting the shape of her facial features, such as the curve of her mouth and the crinkles around her eyes.

Huang worked mainly with four graduate students: Lin, Jilin Tu, Zhenqiu Zhang, and Shyamsundar Rajaram.

In 2005, Nicu Sebe, a researcher at the University of Amsterdam and Huang’s former colleague, developed emotion algorithms using Huang’s software to determine Mona Lisa’s emotions. Huang and his students took Sebe’s research further by determining her gender and applying different algorithms.

“We use techniques from image processing and machine learning in order to decide if the face is male or female,” Rajaram said.

Because the likelihood probabilities of Mona Lisa being female and male are respectively 60 percent and 40 percent, the world may never know the absolute truth, Huang said.

“The best we can do is build models and fit them to the face the best we can,” Lin said.

Huang, who received the “2006 Electronic Imaging Scientist of the Year” Award, explained that the Mona Lisa project is “just for fun” and is a unique way of displaying the power of human-computer interaction. Huang’s research on the painting falls under the category of human computer interface, or biometrics.

“The public’s interest indicates that it is worthwhile for people in technology and the arts to work together to answer these questions,” Huang said. “The reactions of artists and psychologists have been very positive – only one person said that it’s 100 percent crap.”

Although Huang said there were “no definitive results,” in analyzing art work, several museums have approached him to decipher the emotions of subjects in other paintings such as self-portraits of Van Gogh and Frida Kahlo respectively, as well as Edvard Munch’s, The Scream, for a Japanese television station.

Petrovic said this kind of human recognition technology might also be very valuable for surveillance and customer relationship management because it allows retailers to use a computer to easily identify the age, gender and emotional state of a customer.