Facebook shows off its experimental augmented reality app

LAGUNA BEACH, California — Facebook just gave an early look into an experimental camera app that uses artificial intelligence to make live video look like art.

Speaking Tuesday at the Wall Street Journal’s WSJD Live conference, Facebook Chief Product Officer Chris Cox demoed the app, which he said offered a look at how the social network is investing in augmented reality.

The application aims to recreate the look of famous artwork in real-time within the camera using a technology called “style transfer.” It’s a bit like the app Prisma, except that the effects are created within the camera live, not after you shoot a photo or video.

“Style transfer lets you take an artist like a Monet or a Rembrandt and transfer the representation of that style onto any image,” he said. Cox primarily demoed a Starry Night-style effect but he said they also had styles for Georgia O’Keeffe and other artists.

Behind the scenes, Cox said the app uses a new type of artificial intelligence technology called convolutional neural nets, as well as “some cool computer vision stuff.”

While apps like Prisma have used similar AI technology to create fine art-inspired images, they aren’t able to process images or video in real time.

“This is very interesting because it’s taking something that was a known technology but it was getting it to be really, really fast on the phone and able to be done at a low enough latency that there’s no issues with dropping frames or with stuttering or blurring or anything like that,” Cox said.

The application also offers an interesting look into the social network’s efforts in augmented reality and how it views the smartphone camera as an entry point to AR.

Facebook has been investing heavily in virtual reality, but the company’s ambitions in augmented reality have been less clear, though Zuckerberg has previously confirmed the company is working on AR applications. The CEO also showed off a concept for augmented reality glasses at Facebook’s F8 conference earlier this year.

“This is something that’s a prototype right now, it’s not connected to any of our services,” Cox said. “But it’s an area of work we’re really invested in which is making it easy for the camera to be an early application of AR.”