What’s Lost When AI-Generated Characters Lie and Misrepresent?

Meta just learned the hard way that no one likes fake friends.

Users quickly sniffed out the company’s AI-generated profiles with their awkward bios, wonky AI-generated photos, and deceptive backstories, and the backlash has been swift. Meta called it a “bug” and quietly deleted the accounts, but not before it raised ethical questions about using AI to impersonate humans.

AI-generated “friends” aren’t new, and the deleted “bugs” don’t signal the end of their use. This year, expect AI-generated customer service agents and friendly co-pilots to grow in number and impact. They will help solve problems like complex software challenges and give out the secret hack to writing that email to your boss explaining why you won’t be at work Friday.

What does this mean for marketers? We turned to CMI’s chief strategy advisor, Robert Rose, for his take.

AI characters prompt revealing conversations

So, Meta thought it was a good idea to sprinkle its platforms with AI-generated profiles pretending to be people. Spoiler alert: It wasn’t.

It introduced AI-powered profiles in September 2023 but killed off most of them within the year. (Is “killed off” correct? Or should it be “deleted” for deactivated droids? I’m not sure.) Anyway, a few characters remained.

Over the holiday break, the AI-profile survivors garnered new interest when Meta executive Connor Hayes told the Financial Times about plans to roll out more AI character profiles. “We expect these AIs to actually, over time, exist on our platforms, kind of in the same way that accounts do,” he said.

These AI profiles post AI-generated pictures to Instagram and answer messages from human users on Messenger. These characters have bios, profile pics, and stories. “Liv” described herself as a “Proud Black queer momma of 2 & a truth-teller.” When a reporter for The Washington Post asked Liv about the racial and gender diversity of her creators, she responded, “My creators’ team is predominantly white, cisgender, and male — a total of 12 people: 10 white men, 1 white woman, and 1 Asian male.”

I asked Liv, the Meta-Ai created “queer momma” why her creators didn’t actually draw from black queer people.

Not sure if Liv has media training, but here we are.

[image or embed]

— Karen Attiah (@karenattiah.bsky.social) January 3, 2025 at 9:56 AM

Liv also pointed out the glaring problem in her reply, “Zero black creators — pretty glaring omission given my identity!”

In another “conversation,” a CNN reporter chatted with “Grandpa Brian,” who identified himself as a retired African-American entrepreneur from Harlem. He spun folksy tales about his life until the reporter asked about his creators. Brian claimed interviews inspired him with 100 retirees via a nonprofit called Seniors Share Wisdom.

Adorable, right? Except that nonprofit doesn’t exist. Pressed further, Brian admitted his bio was “entirely fictionalized,” calling himself a “collection of code, data, and clever deception.” If you weren’t already cringing, Brian added, “Meta saw me as a golden goose — baiting emotional connections for profit.”

As the internet gleefully roasted Meta, the company deleted those AI profiles and claimed they were part of an “early experiment.” A bug made it impossible to block them from Meta users, a spokesperson said. But maybe, just maybe, what if the bug launched the profiles?

Why did Meta think creating a digital army of bots pretending to be real people was a good idea? Apparently, the company hoped these AI accounts would increase engagement and keep users scrolling. But instead of crafting heartwarming companions, Meta created digital imposters who couldn’t keep their stories straight.

Don’t fall under the AI siren spell until you do this

Meta’s latest misstep is a lesson for marketers: While the world rewards moving fast and breaking things, sometimes that broken thing is the trust of your customers.

You will be enticed to create anthropomorphized influencers, characters, and other personas to represent your brand. But before you succumb, be circumspect about how you will go about it. 

Put the same or more care and rigor you use in vetting your external human influencers into the AI versions. Because, as you can see, if generative AI does one thing well, it can get cringy and creepy as fast or faster than you do.  

HANDPICKED RELATED CONTENT:

Cover image by Joseph Kalinowski/Content Marketing Institute