Researchers have success in detecting if images of faces were manipulated

Make some noise for Adobe in its effort to detect fakery. They unleashed the powers of machine learning to automatically detect when images of faces have been manipulated.

Adobe AI is addressing fakery with the help of research efforts along with scientists at UC Berkeley.

Ryan Whitwam, ExtremeTech, said the team used a Convolutional Neural Network.

The Adobe Communications Team blogged about their training exercise.

"The researchers created an extensive training set of images by scripting Photoshop to use Face Aware Liquify on thousands of pictures scraped from the Internet. A subset of those photos was randomly chosen for training. In addition, an artist was hired to alter images that were mixed in to the data set. This element of human creativity broadened the range of alterations and techniques used for the test set beyond those synthetically generated images."

The blog said, "The tool also identified specific areas and methods of facial warping. In the experiment, the tool reverted altered images to calculation of their original state, with results that impressed the researchers."

Professor Alexei A. Efros, UC Berkeley, said that although it may sound impossible because there are so many variations of facial geometry possible, "in this case, because deep learning can look at a combination of low-level image data, such as warping artifacts, as well as higher level cues such as layout, it seems to work."

Whitwam: "The detection tool highlights areas of the image that appear to be modified and it can go a step further. Since the training is based on an established Adobe tool, the AI can reverse the edits to produce a very good approximation of the original photo."

How it works: Adobe has a Face Aware Liquify tool, to make tweaks to someone's face in a picture (James Vincent commented that Photoshop's Liquify tool was commonly used to adjust the shape of faces and alter facial expressions). The new tool can identify if Face Aware Liquify was used.

This is one attempt but no magic bullet. It is just a research project for the moment, said The Verge. James Vincent said Adobe was sharing the new research in collaboration with scientists from UC Berkeley.

Whitwam: "This tool won't be able to detect anything other than the Photoshop face liquify filter. It's going to take much more work to create an AI that can account for all or even most image manipulation techniques."

Nonetheless, it is a research effort worth watching.

"While still in its early stages," blogged the Adobe Communications Team, "this collaboration between Adobe Research and UC Berkeley is a step towards democratizing image forensics, the science of uncovering and analyzing changes to digital images."

The paper describing their work is up on arXiv. The paper, submitted this month, is titled "Detecting Photoshopped Faces by Scripting Photoshop," by Sheng-Yu Wang, Oliver Wang, Andrew Owens, Richard Zhang and Alexei Efros.

Adobe's researchers are well aware that, in their own words, "The journey of democratizing image forensics is just beginning."

Vincent in The Verge reported that there were no immediate plans to turn this work into a product. Vincent quoted a spokesperson, who said that it was just one of many efforts across Adobe to better detect manipulations—-ranging from manipulations in image, video, audio and documents.

What were the results of their experiment? They tested the accuracy of the neural network in picking up manipulations. "The AI trained on manipulated photos was able to spot the fake with 99 percent accuracy," said Whitwam. Interesting: When asked to spot a sample of edited faces, human volunteers got the right answer only 53 percent of the time, said Vincent.

Adobe Photoshop, after all, is a standard image editing tool and is used often in malicious photo manipulations. The paper discussed the team's method for detecting one very popular Photoshop manipulation—image warping applied to human faces, "using a model trained entirely using fake images that were automatically generated by scripting Photoshop itself."

Why their research matters: They showed that (1) their model outperformed humans in recognizing manipulated images (2) can predict the specific location of edits and (3) in some cases can be used to "undo" a manipulation to reconstruct the unedited image.

The head of Adobe Research, Gavin Miller, meanwhile, said that "Beyond technologies like this, the best defense will be a sophisticated public who know that content can be manipulated—often to delight them, but sometimes to mislead them."

© 2019 Science X Network