What happened when Meta used AI to ban Facebook and Instagram accounts
The announcement startled Camille Hanson awake one night this March. Her Facebook and Instagram accounts, where she had nearly a million followers and runs a business with her husband, who teaches English to non-native speakers, were marked for deletion. The meta said it violated community guidelines for scams and fraud.
“I’m just an English teacher,” Ms. Hanson, 40, recalled thinking. She was baffled by claims of fraud and deceit. “What am I doing wrong?” she wondered.
The notice said she could appeal, so she clicked a button to start the process. A week later, she received another notice: Her appeal had been rejected.
“All your information will be permanently deleted,” read Meta’s response. “You cannot request further review of this decision.
It was like having “an actual brick-and-mortar store and someone just closes your office or storefront,” Ms. Hanson’s husband, Calvin, 42, said in an interview from their home in Portugal.
What happened to the Hansons was a case of artificial intelligence gone wrong. The technology is increasingly responsible for deciding which accounts have violated Meta’s content rules, as well as managing appeals of those decisions. In addition to the hassle, users like the Hansons often find it difficult to reach out to anyone in the company to find out what to do.
In March, Meta announced plans to hand more power to AI bots to decide which accounts to ban or remove harmful images. Months later, the company laid off thousands of employees, including those who performed those jobs.
Users have organized online, with more than 60,000 recently signing a petition asking Meta to provide more information about account bans and let people review appeals. Several communities on Reddit and elsewhere have been bombarded with complaints from Meta users about the company’s AI
Daniel Roberts, a spokesman for Meta, disputed that AI was worse at manipulating accounts than humans, saying the company’s new AI moderation tools make 13 percent fewer mistakes than human employees and find 10 percent more violations. Of the five accounts The New York Times flagged as Meta, all were banned by the company’s old AI moderation tools, Mr. Roberts said.
“We are committed to making fewer enforcement mistakes and helping individuals protect their accounts – and AI delivers on both,” he said.
The meta has experienced a few instances where the AI has gone wrong. In May, hackers misused its AI customer service chatbot to attack 34,000 Instagram accounts, including Barack Obama’s former White House account. That same month, employees revolted over a program that tracked their keyboard movements to train artificial intelligence, leading Meta to suspend the effort.
Last week, 26 former Meta employees alleged in a lawsuit that the company fired them using an AI algorithm that targeted them for their disabilities. A spokesperson for Meta said that “workforce management and organizational decisions were and are made by humans, not artificial intelligence.”
People Over Platforms, a user advocacy group that sells account recovery services, created an online petition this year to get Meta to change its account moderation policies. The meta accused many signatories of sharing child abuse material — a charge that users said added insult to injury — and then banned them.
That was the accusation that Meta made against Athenia Rodney, the owner JuneteenthNYgroup commemorating the end of slavery in the United States when the company banned all of her personal and professional Facebook and Instagram accounts last year.
“Our technology has found your account or activity on it does not meet our rules,” read the notice from Meta. “As a result, our technology took a hit.
Ms Rodney said she did not understand Meta’s reasoning because her group’s content was “family friendly”. After her accounts were banned, she searched JuneteenthNY’s photo archive for potential posts that could trigger the AI bots, but found nothing.
Then Ms. Rodney remembered that shortly before Meta canceled her accounts, the company alerted her to suspicious logins from Seattle and Singapore — even though she lives in Atlanta. It was a sign that her accounts had been hacked.
Ms Rodney quickly changed her passwords, but six of her Instagram and Facebook accounts for various businesses and personal profiles were banned.
To get those accounts back, Ms. Rodney said she submitted identification documents, filed complaints with the Better Business Bureau and the Federal Trade Commission, contacted state consumer protection agencies and messaged Meta employees on LinkedIn. Nothing worked.
Since then, JuneteenthNY has had to convince sponsors that it is a legitimate organization that has been around for nearly two decades.
“Our sponsors go to our social media page to confirm that we are a valid organisation, that we have been doing this for 17 years,” Ms Rodney said. “So now when they go to our account and see 500 people, they say, ‘You didn’t do it – you’re all frauds’.”
After The Times flagged five of Meta’s banned accounts, the company said it investigated and found that two were the result of hacks and one was for a legitimate reason. The hacked accounts were legitimately banned because the hackers violated child safety policies, Meta said.
Meta then restored two hacked accounts, including Ms. Rodney’s JuneteenthNY account. Two of the five accounts flagged were mistakenly banned, including the Hansons’, the company said.
Roberto Suarez, an Instagram influencer who had cultivated more than 500,000 users before this year’s ban, was among those who received an email from Meta that his account had been reinstated — along with his photos, followers, messages and more. He started posting content again to fans who hadn’t heard from him in months.
A few weeks later, Meta sent him another email. “Your account has been suspended,” it said, claiming he had infringed copyright on his content. “If you do not take any action, your account will be permanently disabled.”
Mr. Suarez denied infringing any copyright. Shortly after The Times inquired about the new ban, his account was reinstated.
Ms Hanson also received an email from Met saying her account was back. “We have reviewed your account and found that the activity on it meets our community standards for fraud and scams, so you can resume using Instagram,” the message read. “We’re sorry we got it wrong.