Deepfake porn crisis batters South Korea schools

Activists wearing eye masks, hold posters reading ‘Repeated deepfake sex crimes, the state is an accomplice too’ during a protest against deepfake porn in Seoul on August 30, 2024 – Copyright AFP Anthony WALLACE

Claire Lee and Hieun Shin

After South Korean authorities uncovered a sprawling network of AI deepfake porn Telegram chatrooms targeting schools and universities, teenage activist Bang Seo-yoon began collecting testimony of abuse from victims.

Many of the cases she documented followed the same pattern: schoolboys steal innocuous selfies from private Instagram accounts and create explicit images to share in the chat rooms, specifically to humiliate female classmates — or even teachers.

Super-wired South Korea, with the world’s fastest average internet speeds, has long battled sexual cyber violence, but experts say a toxic combination of Telegram, AI tech, and lax laws has supercharged the issue — and it is tearing through the country’s schools.

“It’s not just the harm caused by the deepfake itself, but the spread of those videos among acquaintances that is even more humiliating and painful,” Bang, 18, told AFP.

She has received thousands of reports from devastated victims since authorities in August found the first such Telegram chatrooms, typically set up within a school or university to prey on female students and staff.

Most perpetrators are teens, police say.

Deepfake prevalence is increasing exponentially globally, industry data shows, up 500 percent on year in 2023, cybersecurity startup Security Hero estimates, with 99 percent of victims women — typically famous singers and actresses.

But while celebrities have powerful backers to protect them — the K-pop agency behind girlband NewJeans recently took legal action against deepfake porn — many ordinary victims are struggling to get justice, activists say.

– ‘Live in fear’ –

Prosecution rates are woeful: between 2021 and July this year, 793 deepfake crimes were reported but only 16 people were arrested and prosecuted, according to police data obtained by a lawmaker.

After news of the chat rooms spread, complaints surged, with 118 cases reported in just five days in late August, and seven people arrested amid a police crackdown.

But six out of seven alleged perpetrators were teenagers, police say, which complicates prosecutions as South Korean courts rarely issue arrest warrants for minors.

The chatrooms, multiple of which AFP attempted to join before being removed by moderators, have lewd names such as “the lonely masturbator” and rules requiring members to post photos of women they wish to see “punished”.

Victims find themselves “sexually insulted and mocked by their classmates in online spaces”, Kang Myeong-suk, head of victim support at the Women’s Human Rights Institute of Korea told AFP.

“But the perpetrators often face no consequences,” she said, adding that victims now “live in fear of where their manipulated images might be distributed by those around them”.

“Some online comments say the victims should ‘get over it’ as these deepfake images are not even real,” Kang said.

“But just because manipulated images aren’t real doesn’t mean the pain the victims endure is any less genuine.”

– Victim blaming –

While overall crime rates in South Korea are generally low, the country has long suffered from an epidemic of spy-cam crimes, which led to major protests in 2018 inspired by the global #MeToo movement, eventually forcing lawmakers to strengthen laws.

Even so “the penalties issued are often trivial, like fines or probation, which are disproportionate to the gravity of the offenses”, professor Yoon Kim Ji-young told AFP.

There have also been Telegram porn scandals before, most notably in 2020 when a group blackmailing women and girls to make sexual content for paid chatrooms was uncovered. The ringleader was jailed.

But things have not improved.

President Yoon Suk Yeol’s dismissive views on feminism — which he has blamed for the country’s low birthrate — have signalled to men it is “okay to be hostile or discriminatory towards women”, Yoon Kim said.

South Korean police blame low prosecution rates on Telegram, which is famed for its reluctance to cooperate with authorities. Its founder was recently arrested in France for failing to curb illegal content on the app.

But one victim of a 2021 deepfake porn incident told AFP that this was no excuse — many victims manage to identify their attackers themselves simply by determined sleuthing.

The victim, who requested anonymity, said it had been a “huge trauma” to bring her assailant to justice after she was attacked in 2021 with a barrage of Telegram messages containing deepfake images showing her being sexually assaulted.

Her attacker was a fellow student at the prestigious Seoul National University, who she had rarely interacted with but always thought was “gentle”.

“It was hard to accept,” she said, adding police required her to collect all the evidence herself, then she had to lobby hard for a trial, which is now ongoing.

“The world I thought I knew completely collapsed,” she said in a letter she plans to submit to the court on September 26.

“No one should be treated as an object or used as a means to compensate for the inferiority complexes of individuals like the defendant, simply because they are women.”