How Pinterest Uses AI to Capture Our Imaginations

Posted by E.L.E.X on April 20 2019 under Tech/Cybersecurity

THE TERM “ARTIFICIAL intelligence” conjures up images of both the distant future (humanoid robots, self-driving cars) and the hard work it will take to get there. But the technology is already pervasive, and in places, you might not realize.

Take Pinterest, the visual search platform where people discover and save ideas. The company invests heavily in artificial intelligence, often in ways that might not be obvious to a casual user.

“We’re constantly experimenting with applications that people can imagine using today,” says Vijay Narayanan, the company’s Head of Discovery and Content. “There are AI projects focused on things like self-driving cars, but there’s also the everyday, accessible AI that helps people live better lives now.”

In order to develop the “everyday, accessible AI,” Pinterest follows a simple but often-ignored rule: Before anything else, define your problem. “Don’t develop technology just for the sake of developing technology,” says Narayanan. “Instead, develop technology with a purpose, and one that supports your core mission.”

Features like surfacing recipe suggestions based on diet or suggesting home decor for a user’s specific taste are available because of deep learning (a subset of AI and machine learning that uses neural networks to mimic the brain to more quickly analyze data and train computer models). By understanding the intention behind a simple search, Pinterest’s deep learning models deliver highly personalized results.

And Internet users appreciate personalization—80 percent of them are more likely to make a purchase if the experience is personalized. By using AI to analyze mountains of data, Pinterest tailors search results for each of its hundreds of millions of users. That’s a big reason why Pinterest considers itself “at its core, a data, and AI company,” according to Vanja Josifovski, it's Chief Technology Officer.

Discovering a User’s Intention
In September, Pinterest reached 250 million monthly active users. Pinterest co-founder Evan Sharp recently said that while users spend time in other popular apps, they invest time in Pinterest. People use what they discover on Pinterest. Whether for a costume party or on a road trip, they make use of what they find.

For this to be true, Pinterest needs to solve the “discovery problem.” When a user enters a search term, nearly 75 percent of which are three words or fewer, what is he or she actually looking for? There may not be a clear answer to start, but the deep learning-powered search experience helps draft one.

Say the user searches “BBQ,” and selects an image that depicts an intimate backyard scene. The app will recommend other images and searches. Maybe they include table settings or lawn games. Strung together, they represent a path of discovery, starting with A, the vague desire to grill something, and ending with Z, a fastidiously planned weekend barbeque with friends.

One way Pinterest makes recommendations relevant is through a neural network called PinSage, developed in part using the TensorFlow and PyTorch deep-learning frameworks on Amazon Web Services (AWS). The deep-learning model places each image, according to theme, within one giant “graph” of other images.

Three billion images, or “nodes,” form the graph; 18 billion lines connect them. The result is a detailed context for each image, which allows Pinterest to recommend thematically similar images for users, such as charcoal briquettes and grilled meats. Rather than a linear list of results, the user receives a comprehensive guide to what he or she could plan for the weekend.

Pinterest’s deep-learning personalization efforts use more than just users’ search terms to improve. They also learn from what users capture with their phones’ cameras.

Pinterest Lens camera search allows users to search by taking a photo of an object offline and receive results for online recommendations. An apple will return results for related recipes for dishes, such as pie or cider. A photo of a pair of running shoes will result in related shoes, and even athletic clothing to style with it, available for purchase.

When a user takes a photo through Lens, the deep-learning model breaks it into objects, colors, and visual patterns. It then uses that information to make suggestions, which can be either visually similar, thematically similar, or both. Every month, the model processes, and therefore learns from, hundreds of millions of visual searches (in 2018, a year after launching Lens, Pinterest announced that users were making 600 million visual searches a month). The model improves with every search because it receives feedback on which suggestions users pursued, and which they ignored.

Pinterest is also using Amazon Rekognition, an AI service for image recognition and analysis, as part of its multipronged approach to provide a trustworthy platform for all Pinners. Rekognition automatically filters out content that’s in violation of policies, allowing Pinterest to focus on areas core to its mission of improving discovery.

Investing in the Everyday (with Lots of Data)
Pinterest has no trouble attracting new users. And more users means more opportunity (and demand) for personalization and diversification in search.

This growth, coupled with the AI efforts that capitalize on the breadth of data it collects from its users, has meant that Pinterest has tripled its data storage capacity over the last two years.

Servicing the thousands of elements that make up the user experience, and storing the more than 175 billion pins users have saved, would be extremely time-consuming to do on-site, says Josifovski. So Pinterest uses cloud-based storage from AWS. Storing all the images that strengthen the Lens model, for example, is one case for which Pinterest relies on Amazon S3, and, to train and deploy deep-learning models, it uses Amazon EC2 P3 instances leveraging NVIDIA Tesla V100 Tensor Core GPUs.

Using cloud computing rather than on-premises servers also allows Pinterest to scale its AI efforts quickly with high-performance compute capacity, freeing engineers to experiment with new features like Shop the Look, which singles out a product in an image, allows the user to select it, and shows the user where to find it online.

This focus on everyday, accessible AI means more new features to inspire users and more opportunities for discovery.

How Pinterest Uses AI to Capture Our Imaginations

YOU MAY ALSO LIKE

Comments

Add Comment