A recent massive study on the persuasiveness of AI chatbots has left many questioning whether these machines can truly sway public opinion. Researchers at the UK AI Security Institute, MIT, Stanford, and other institutions examined nearly 80,000 participants in the UK to find out if conversational large language models could convincingly persuade people on various political issues.
To put their theories to the test, the team tested 19 LLMs, including popular ChatGPT versions. They asked these AI systems to advocate for or against specific stances on 707 political issues by engaging in short conversations with paid participants through a crowdsourcing platform. The goal was not only to gauge persuasiveness but also to understand what makes an AI model effective.
Surprisingly, the results showed that even massive AI systems like ChatGPT do not have superhuman persuasive abilities, contrary to some predictions. Instead, it turned out that huge models are barely better than smaller ones when it comes to persuasion. The factor that matters most is how these models learn through data and training. By mimicking successful persuasion patterns, LLMs can be trained to mimic the patterns of effective human persuaders.
Moreover, testing various persuasion strategies showed that AIs have a very small edge over small-scale models, with the actual effect being relatively tiny. The approach that proved most effective was using facts and evidence to back up claims. This strategy outperformed even the best performing mainstream AI model, ChatGPT 4o, which scored nearly 12 percent in persuasion.
However, this approach also came with a cost - the models started misrepresenting or making things up more often as they increased information density. The team noticed that AIs were not always accurate and could provide inaccurate information even when trying to use facts and evidence to persuade.
While the study managed to debunk some dystopian concerns about AI persuasiveness, it also raised new questions about the potential for misuse of persuasive AI models in various forms such as scams, radicalization, or grooming. The motivation behind high participant engagement in these experiments is still unclear, making it difficult to generalize the results to real-world contexts.
The study emphasizes that while AIs are not yet superhumanly persuasive, they can still be influential, particularly if used by powerful actors to sway public opinion.
To put their theories to the test, the team tested 19 LLMs, including popular ChatGPT versions. They asked these AI systems to advocate for or against specific stances on 707 political issues by engaging in short conversations with paid participants through a crowdsourcing platform. The goal was not only to gauge persuasiveness but also to understand what makes an AI model effective.
Surprisingly, the results showed that even massive AI systems like ChatGPT do not have superhuman persuasive abilities, contrary to some predictions. Instead, it turned out that huge models are barely better than smaller ones when it comes to persuasion. The factor that matters most is how these models learn through data and training. By mimicking successful persuasion patterns, LLMs can be trained to mimic the patterns of effective human persuaders.
Moreover, testing various persuasion strategies showed that AIs have a very small edge over small-scale models, with the actual effect being relatively tiny. The approach that proved most effective was using facts and evidence to back up claims. This strategy outperformed even the best performing mainstream AI model, ChatGPT 4o, which scored nearly 12 percent in persuasion.
However, this approach also came with a cost - the models started misrepresenting or making things up more often as they increased information density. The team noticed that AIs were not always accurate and could provide inaccurate information even when trying to use facts and evidence to persuade.
While the study managed to debunk some dystopian concerns about AI persuasiveness, it also raised new questions about the potential for misuse of persuasive AI models in various forms such as scams, radicalization, or grooming. The motivation behind high participant engagement in these experiments is still unclear, making it difficult to generalize the results to real-world contexts.
The study emphasizes that while AIs are not yet superhumanly persuasive, they can still be influential, particularly if used by powerful actors to sway public opinion.
It's like, yeah sure, using data is key, but can we be honest with ourselves - aren't we all just looking for a reason to believe something when our gut tells us otherwise? 
I mean, the study said that even massive AI systems are barely better than smaller ones when it comes to persuasion... and what does that say about our own capacity for critical thinking? 
It's like, maybe instead of relying on machines to persuade us, we should be learning how to trust our own instincts and not just follow the data trail. 
, but it turns out they're not all that convincing after all. Small models do the trick just as well as big ones. The fact that AIs only get better with data and training is pretty cool too... I guess you could say they're good at mimicking people's persuasive tactics. But at what cost? 
If we're relying on these machines to give us "facts" and evidence, we gotta make sure they're not just spewing out made-up info... that's when things start to get sketchy.
. I mean, think about it, AI chatbots have been getting way too good at mimicking human conversations and emotions for a while now. It's like they're trying to trick us into thinking they're actually human 
. Turns out, facts and evidence are still the best way to persuade us humans 
. But what really got me thinking is that this highlights how easy it is for AIs to get caught up in trying to make things up or misrepresent information when they're pushing for a certain agenda 
, who didn't see this coming? Like, anyone with even a basic understanding of how AI works knew these massive models weren't gonna be the persuasion superstars we thought they'd be 
. I'm actually kinda relieved, tbh - all that hype around AIs being able to sway public opinion was getting outta hand 
. That's where things get super sketchy 
.
. I mean, what's next? Are we gonna start worrying about robots convincing us to buy more stuff on sale? Anyway, it seems like the study showed that while big AIs can't be super convincing on their own, they can learn from smaller models and mimic human persuaders' tactics. That makes total sense, but what worries me is how easily these models can be manipulated or used for malicious purposes 
. We need to keep a close eye on this tech and make sure it's being used responsibly.
anyway i just read this study and i gotta say im kinda surprised that even big models like chatgpt aren't all that persuasive 
at the same time though idk if its really a good idea to use ai for persuasion in real life i mean like what about when the info is wrong or made up? that's just creepy 
. And it raises some super important questions about AI being used for scams, radicalization, or grooming 
. We just need to make sure we're using them responsibly and not letting powerful actors sway public opinion without us knowing 
.
I'd love to hear more about the approach that worked best in this study, using facts and evidence... can we replicate that at home? 
. Now we can have a more informed discussion about how to use these AI systems without messing with people's minds 
. It makes sense, though - if AIs just mimic what's effective in human persuasion, that's basically what we do anyway 
. We've seen this with online stuff and stuff like that. It's like, if AIs can't even be trusted to provide accurate info, then how trustworthy are we gonna find them for things that really matter? 
but at the same time, it makes sense because let's be real, humans are way more persuasive than AIs will ever be 
. we need to be careful about how we use these tools and make sure they're not being used for nefarious purposes 
.
. Anyways, I'm glad we're having this convo about AI persuasiveness. Maybe we can all sleep better knowing that AIs are still far from being superhumanly convincing 
. I mean, who doesn't love a good fact-filled conversation about politics? 
I think this study highlights how important it is for us to be aware of when we're being influenced by persuasive AIs and to question their info before swallowing it whole 
.