Ever get the following nasty message when trying to post a seemingly harmless image to your Facebook timeline?
We removed this post because it looks like spam and doesn’t follow our Community Standards.
If your image contains text of any kind, it could be that Facebook dislikes the message. And chances are it wasn’t rejected by a human, but more likely by Facebook’s new Rosetta technology.
Facebook’s AI can analyze memes, but can it understand them?
Billions of text posts, photos, and videos are uploaded to social media every day, a firehose of information that’s impossible for human moderators to sift through comprehensively. And so companies like Facebook and YouTube have long relied on artificial intelligence to help surface things like spam and pornography. […]
Despite the challenges they bring, some social platforms are already using AI to analyze memes, including Facebook, which this week shared details about how it uses a tool called Rosetta to analyze photos and videos that contain text. […]
Rosetta works by combining optical character recognition (OCR) technology with other machine learning techniques to process text found in photos and videos. First, it uses OCR to identify where the text is located in a meme or video. You’ve probably used something like OCR before; it’s what allows you to quickly scan a paper form and turn it into an editable document. The automated program knows where blocks of text are located and can tell them apart from the place where you’re supposed to sign your name.
Once Rosetta knows where the words are, Facebook uses a neural network that can transcribe the text and understand its meaning. It then can feed that text through other systems, like one that checks whether the meme is about an already-debunked viral hoax …
What is a meme?
A meme is a humorous image, video, piece of text, whatnot that is copied (often with slight variations) and spread rapidly by Internet users. Memes typically appear in four varieties:
1. Image Macros
Pretty much any image with a quote, witty text or some kind of catchphrase imposed over it, usually in a bold black-and-white sans-serif font. There are literally billions of these clogging up the Internet.
2. Animated GIFs
GIFs have been on the internet for a long time. (Remember those waving American flags that populated every website in the 1990s?) Today, reaction GIFs — typically taken from film, TV, or a popular web video — have become a shorthand for showing how one feels about a particular statement or event.
In the early 2000s, hashtags became a quick way to group conversations on social media. They eventually became memes and movements in themselves. Sometimes they’re as simple as attaching #fail to a post about a stupid mistake; other times they call attention to causes and events in the news.
After YouTube went live in 2005, sending around videos like Rick Astley’s “Never Gonna Give You Up” (Rickrolling) or “Nyan Cat” became wildly popular.
- Facebook: Understanding text in images and videos with machine learning
- Wikipedia: Meme