AI chatbots ‘unable to accurately summarise news’
A study by the BBC found that four major chatbots – including ChatGPT and Google’s Gemini – struggled to accurately summarise news stories.
Some of the most popular AI chatbots are unable to accurately summarise the news, with the chief executive of BBC News warning that tech firms were “playing with fire” on the issue of misinformation.
Deborah Turness said the BBC was hoping to “open up a new conversation with AI tech providers” on the issue, with the aim of finding solutions to so-called distortions in AI chatbot responses.
A BBC study tested OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini and Perplexity AI by giving them access to the BBC News website and then asking them basic questions about the news before rating the quality of answers from the assistants.
The researchers found “significant issues” with just over half of the answers generated by the chatbots, and clear factual errors were introduced into around a fifth of answers.
The study said that each assistant tested served up inaccuracies, but in general Microsoft’s Copilot and Google’s Gemini had the more significant issues.
The BBC News chief executive said AI’s “extraordinary benefits” must not be outweighed by the creation of a “world where people searching for answers are served distorted, defective content that presents itself as fact”.
“In what can feel like a chaotic world, it surely cannot be right that consumers seeking clarity are met with yet more confusion,” she said.
“It’s not hard to see how quickly AI’s distortion could undermine people’s already fragile faith in facts and verified information.”
“We live in troubled times, and how long will it be before an AI-distorted headline causes significant real-world harm?”
In January, Apple announced it had “paused” the use of its Apple Intelligence AI tools to create news summaries after a number of incidents where it created inaccurate headlines.
Ms Turness said Apple’s “bold and responsible decision” shows the iPhone maker did “recognise the high stakes of distorted news and information”.
“We’d like other tech companies to hear our concerns, just as Apple did. It’s time for us to work together – the news industry, tech companies – and of course government too has a big role to play here,” she said.
“Earning trust has never been more critical. As the CEO of BBC News, that is my number one priority.
“I’ll end with a question: how can we work urgently together to ensure that this nascent technology is designed to help people find trusted information, rather than add to the chaos and confusion?
“We at the BBC are ready to host the conversation.”
