Call it a perfect storm for facial recognition. The combination of the lightning-fast emergence of GPUs and the continuing growth of social networking has created a virtual play land for researchers who have been trying to perfect the automatic recognition of people in photos.
Researchers from Harvard and MIT were on hand at INVIDIA's GPU Technology Conference this week to share how they're trying to take advantage of this opportunity. Put simply, they're working on applying the same approach the human brain takes in processing visual clues to facial recognition computing schemes. The potential applications are endless, but can be divided into two main categories: adding visual intelligence to social networks like Facebook, and creating virtual human eyes that can handle any job requiring identification of people or objects.
The effort came about when David Cox, a PhD student at Harvard, who's also the principal investigator for the visual neuroscience group at Harvard University's Rowland Institute, had a happenstance meeting with MIT PhD candidates Nicolas Pinto and James DiCarlo in 2006. The three decided to embark on a project in which they would apply neuroscience to facial recognition and analyze photos posted on Facebook. The team secured a base of 500 willing participants, all of whom agreed to have their photos analyzed so long as their privacy was protected, and started filtering photos based on a long list of paramterers.
While the work has been done without Facebook's involvement''They won't give us the time of day,' said Cox'the team has achieved some remarkable results thanks to advances in GPU computing. For instance, in an experiment analyzing a network of 100 Facebook Friends, the team's program was able to successfully auto-tag photos 86 percent of the time, a number Pinto called 'impressive' in the world of facial networking.
Such results would not have been possible prior to the existence of GPUs. In fact, since the project started, Cox said the team has seen a 1000x improvement in processing speeds, enabling them to sort 5,000 images in 2 minutes using a GPU cluster, down from the 2 hours it took a few short years ago. 'We kind of lucked out that GPUs came onto the scene just as we were working on this,' said Cox.
Despite the fast-growing computing power, the effort still has limitations. Several audience members who attended the presentation from Cox and Pinto asked about things such as what they're doing about mis-labeled photos, how they're handling photos of the same person taken many years apart, or why they're not scaling the effort to handle 100,000 faces rather than 500. The answers were the same each time: They're not able to do so yet, but they're excited to expand their efforts as GPU technology advances.
As it stands, they're emulating the brain's visual system by running each face through a set of three filters, each one drilling down into progressively more detailed parameters. Adding any more layers, said Pinto, would greatly increase the odds of making a mistake or otherwise corrupting the data.
With a bit more refinement, the technology will certainly find a market waiting for it. Cox foresees the team spinning its work off into a startup company that's likely to target multiple markets. Not only does facial recognition offer social networks the potential to intelligently do things like suggest new friends to based on making connections between faces in various photos, it also could be used for things like categorizing apples based on appearance or identifying people caught in security surveillance videos.
The team's Facebook effort also holds implications for a related project called PubFig, in which nearly 60,000 photos of public figures pulled from the Internet have been used to test facial verification and recognition capabilities. Because that data set contained some duplicate images'for instance, the same image of Angelina Jolie appears about a dozen times, which throws off efforts to do parallel analysis of subsets of PubFig photos'the results have been a bit more willy-nilly than desired, admitted Cox. He's hoping that the success of the Facebook effort will inspire other research teams to refine the PubFig project.
Any way you look at it, Cox and Pinto and their team are on the cusp of addressing a challenge that computer scientists have been trying to tackle for decades.
'Face recognition is really a fascinating problem,' Pinto said, adding that as GPU technology progresses, he expects the team's research efforts to follow suit. 'It's just the beginning.'