A person using Instagram.
Lorenzo Di Cola | NurPhoto via Getty Images
Pugs, Ferraris, mountains, brunches, beaches and babies – Instagram is full of it. In fact, over the past decade, it has become one of the largest image databases on the planet, and the owner of the company, Facebook, uses this treasure chest to learn machines that are in a photo.
Facebook announced Thursday that they have built an artificial intelligence program that can “see” what they are looking at. It did so by releasing more than 1 billion public images from Instagram.
The “computer vision” program, nicknamed SEER, outperformed existing AI models in an object recognition test, Facebook said.
It achieved a “classification accuracy score” of 84.2% when it attempted to deliver a test through ImageNet, a large visual database designed for use in software research for the recognition of visual objects . It basically tests whether an AI program can identify what is in a photo.
New approach
While many AI models are trained on carefully labeled datasets, Facebook said SEER has learned how to identify objects on photos by analyzing random, unmarked and uncurated Instagram images. This AI technique is known as self-supervised learning (SEER is a play on SElf-supERvised).
“The future of AI is to create systems that can learn directly from the information they receive – whether it’s text, images or some other type of data – without relying on carefully compiled and labeled datasets to teach them how. to objects in a photo, interpret a block of text or perform one of the countless other tasks we ask, ‘Facebook researchers wrote in a blog post.
“The performance of SEER shows that learning under supervision can perform in computer vision tasks in the real world,” they add. “It’s a breakthrough that will eventually pave the way for more flexible, accurate and adaptable computer vision models in the future.”
Although it is only a research project, a Facebook spokesperson said the potential uses are relatively broad. It includes improved auto-generated text for describing images to people with visual impairments, better automated categorization of items sold on Facebook Marketplace, and better systems to keep malicious images away from the Facebook platform.
Privacy issue?
But many Instagram users will be surprised to hear that their images are used to train Facebook AI systems.
“We inform Instagram account holders in our data policies that we are using the information we have to support research and innovation, including technological advances like this,” Priya Goyal, a software engineer at Facebook AI Research, told CNBC .
Facebook said it would open up some of the source’s software so other researchers could experiment with it.
“While we share the details of our research and create an open library that will enable other researchers to use self-study to train models on crude images, we are not sharing the images or the SEER mode,” Goyal said said.
Other big technology companies, including Google and Microsoft, are also trying to push the boundaries of computer vision. Last summer, Google released the SimCLRv2 computer vision model, while OpenAI released iGPT 2.