AI-generated personas becoming increasingly common online

ABC NEWS (Australia)

ABC investigated the rise of AI personas where people are forging bonds with influencers generated by artificial intelligence.

Transcript

00:00Well, it actually began when I met a woman at a party who had created one of her own.

00:06This character was called Margot Monet.

00:09She was entirely fictitious, generated using AI image generation software.

00:15And the goal was, she said, to monetize Margot, which was primarily through brand partnerships

00:21but also selling subscriptions to people who would then get access to exclusive not-safe-for-work

00:27content.

00:29She said she'd been actually inspired by this character, Aitana, whose creators I spoke

00:35to and who said she could earn sometimes $80,000 a month.

00:40And once I started looking at these two, my Instagram algorithm obviously caught on and

00:46suddenly I was inundated with these AI influences.

00:50So it really kind of piqued my interest from there.

00:53I have to admit, I've been following Lil Miquela for a number of years now and watching her

00:58progress with interest, she's become more and more realistic as time goes on.

01:03Just how common are these AI influences?

01:06Well, as I say, once you start looking, they really come out of the woodwork.

01:13It's hard to establish.

01:14There are definitely hundreds, probably thousands, but what really kind of struck me as I researched

01:19more and more was that there was kind of these two different types of AI influences.

01:24There was the quite basic image generation with a human behind the keyboard kind of orchestrating,

01:29but then there were these AI chatbot companions, which were chatbots like ChatGPT, but where

01:35ChatGPT responds with knowledge and to search requests, these AI chatbots offer companionship

01:42and conversation support.

01:46When you factor this other group in, the largest provider of these chatbot companions is a

01:52company called Replica, and they say that they've got 10 million customers, all of whom

01:56are paying a subscription.

01:58They pay the subscription to design their friend, but usually a girlfriend or a boyfriend,

02:03virtual of course, which then support them.

02:07Wow, that's incredible.

02:09Some of these AI influencers having millions of followers, clearly it may have a good financial

02:15impact on the people operating those accounts, but what about on the flip side?

02:18What is the impact on the people who are interacting with these influencers as though

02:23they are real people?

02:26The first one that probably will occur to most people is body image issues.

02:33For decades, we've kind of spoken about, particularly young women, but increasingly young men who

02:36kind of get continually confronted with images of models and fitness influencers whose full-time

02:43jobs are to look good and stay fit, and it kind of creates unrealistic expectations and

02:47a lot of pressure on young people.

02:49Now, they are obviously being confronted with these entirely fake creations who can be designed

02:56in any way the designer wants in pursuit of whatever they consider to be the most desirable

03:01features.

03:03Less obvious is this risk of a thing called limerence, which is, I spoke to a psychiatrist

03:08who has already seen this between his clients, or his patients rather, and real influencers,

03:14which is the projection of a real connection when none exists.

03:19This psychiatrist, Rahul Kahana, actually had a patient who had done this with an older

03:26female influencer who provided the likes of cooking tutorials, and in the patient's mind,

03:31this connection had taken on a real-world aspect, which was obviously highly distressing

03:36when he kind of came to realize in time that there was no real connection there.

03:40There's also this risk that comes with these kind of parasocial relationships, which is

03:44a complete imbalance of power between the user and the creator.

03:48In a normal human relationship, a healthy one, there's an equal distribution of power.

03:54In these cases, all of the power rests with the creator.

03:58They can delete an account, they can block a user, or they can even change a bit of code,

04:02so one of these chatbot companions can suddenly start acting completely differently.

04:06We actually saw this with Replica.

04:07They were threatened with shutdown by Italian regulators, and they removed an erotic functionality

04:16from these characters, and the community that used them went on to Reddit with an outpouring

04:22of grief, talking about the lobotomy, they called it, which was essentially thousands

04:25and thousands of people seemingly to have lost their boyfriends or girlfriends.

04:30So when you start kind of getting that real-world connection kind of forged, it can be very

04:36damaging to the user.

04:38And the final thing, and possibly the darkest thing, is the implications for AI pornography.

04:43Margot Monet, like many of these other influencers, sell essentially not-safe-for-work content.

04:51The potential there for what one psychiatrist described to me as opening Pandora's box and

04:57creating a really dangerous spiral for people who haven't had normal sexual relationships

05:02and engage with AI pornography, they get unrealistic expectations, it can kind of

05:07feed the darkest kind of desires, I suppose, and that can have a kind of a really damaging

05:12effect on what that person's perception of what a healthy, happy sexual relationship

05:17could be.

05:18God, just an absolutely fascinating space this is, and will only become more and more,

05:25I guess, complex as we move forward into the brave new world of AI.

05:29Gerard and Kan, thank you so much for talking us through it.

Category

Transcript

Recommended