Study of ChatGPT citations finds ‘spectrum of inaccuracy’
Image by Franz Bachinger from Pixabay
A study put out this week by the Tow Center for Digital Journalism looking at how ChatGPT produces citations (i.e. sources) for publishers’ content found “a spectrum of inaccuracy in the responses.”
The researchers examined citations produced by ChatGPT after the chatbot was asked to identify the source of sample quotations plucked from a mix of publishers. They found “numerous” instances where publishers’ content was inaccurately cited by ChatGPT. While they found “some” entirely correct citations (i.e. meaning ChatGPT accurately returned the publisher, date, and URL of the block quote shared with it), there were “many” citations that were entirely wrong; and “some” that fell somewhere in between.