I don’t want to admit it, but I did spend a lot of money online this holiday shopping season. And unsurprisingly, some of those purchases didn’t meet my expectations. A photobook I bought was damaged in transit, so I snapped a few pictures, emailed them to the merchant, and got a refund. Online shopping platforms have long depended on photos submitted by customers to confirm that refund requests are legitimate. But generative AI is now starting to break that system.
A Pinch Too Suspicious
On the Chinese social media app RedNote, WIRED found at least a dozen posts from ecommerce sellers and customer service representatives complaining about allegedly AI-generated refund claims they’ve received. In one case, a customer complained that the bed sheet they purchased was torn to pieces, but the Chinese characters on the shipping label looked like gibberish. In another, the buyer sent a picture of a coffee mug with cracks that looked like paper tears. “This is a ceramic cup, not a cardboard cup. Who could tear apart a ceramic cup into layers like this?” the seller wrote.
The merchants reported that there are a few product categories where AI-generated damage photos are being abused the most: fresh groceries, low-cost beauty products, and fragile items like ceramic cups. Sellers often don’t ask customers to return these goods before issuing a refund, making them more prone to return scams.
In November, a merchant who sells live crabs on Douyin, the Chinese version of TikTok, received a photo from a customer that made it look like most of the crabs she bought arrived already dead, while two others had escaped. The buyer even sent videos showing the dead crabs being poked by a human finger. But something was off.
“My family has farmed crabs for over 30 years. We’ve never seen a dead crab whose legs are pointing up,” Gao Jing, the seller, said in a video she later posted on Douyin. But what ultimately gave away the con was the sexes of the crabs. There were two males and four females in the first video, while the second clip had three males and three females. One of them also had nine instead of eight legs.
Gao later reported the fraud to the police, who determined the videos were indeed fabricated and detained the buyer for eight days, according to a police notice Gao shared online. The case drew widespread attention on Chinese social media, in part because it was the first known AI refund scam of its kind to trigger a regulatory response.
Lowering Barriers
This problem isn’t unique to China. Forter, a New York-based fraud detection company, estimates that AI-doctored images used in refund claims have increased by more than 15 percent since the start of the year, and are continuing to rise globally.
“This trend started in mid-2024, but has accelerated over the past year as image-generation tools have become widely accessible and incredibly easy to use.” says Michael Reitblat, CEO and cofounder of Forter. He adds that the AI doesn’t have to get everything right, as frontline retail workers and refund review teams may not have the time to closely scrutinize each picture.