A Canadian government-commissioned Deloitte healthcare report that cost one province nearly $1.6 million contains potentially AI-generated errors, marking the second country this year to fall victim to the consulting firm’s fact-checking shortcomings.
The errors—found in an investigation published Saturday by The Independent, a progressive Canadian news outlet covering the country’s easternmost province Newfoundland and Labrador—appear in a 526-page report that was disseminated by its government in May.
The report advised the then Liberal-led government’s Department of Health and Community Services on topics including virtual care, retention incentives, and the impacts of the COVID-19 pandemic on healthcare workers during a time when the province’s healthcare sector is facing nurse and doctor staffing shortages.
The Deloitte report contained false citations, pulled from made up academic papers to draw conclusions for cost-effectiveness analyses, and cited real researchers on papers they hadn’t worked on, The Independent found. It even included fictional papers co-authored by researchers who said they had never worked together.
“Deloitte Canada firmly stands behind the recommendations put forward in our report,” a Deloitte Canada spokesperson told Fortune in a statement. “We are revising the report to make a small number of citation corrections, which do not impact the report findings. AI was not used to write the report; it was selectively used to support a small number of research citations.”
The lengthy report also cited an academic paper from the Canadian Journal of Respiratory Therapy, which is yet to be found when searching its database.
“It sounds like if you’re coming up with things like this, they may be pretty heavily using AI to generate work,” Gail Tomblin Murphy, an adjunct professor in the School of Nursing at Dalhousie University in Nova Scotia, told The Independent. Tomblin Murphy was cited by Deloitte in an academic paper that “does not exist.” She added that she had only worked with three of the six other authors named in the false citation.
“And I definitely think that there’s many challenges with that. We have to be very careful to make sure that the evidence that’s informing reports [is] the best evidence, that it’s validated evidence. And that, at the end of the day, these reports—not just because they cost governments and they cost the public—[are] accurate and evidence-informed and helpful to move things forward.”
As of Monday, the report remains on the Canadian government’s website.
The Canadian government spent just under $1.6 million on the report, paying in eight installments, according to an access to information request published in a blogpost last Wednesday.
Tony Wakeham, leader of the Progressive Conservative Party in the province and the province’s new premier, was sworn into office in late October. Newfoundland and Labrador’s premier office and the province’s Department of Health and Community Services did not immediately respond to Fortune’s request for comment on the May report and has not publicly addressed the issue.
The revelation comes on the heels of news last month that Deloitte leveraged AI in a $290,000 report published in July to help the Australian government crack down on welfare. But a researcher flagged hallucinations in the 237-paged study, which included references to non-existent academic research papers and a fabricated quote from a federal court judgment.
In the revised study, which was quietly uploaded to the Australian government’s website, the consulting firm admitted it had used the generative AI language system, Azure OpenAI, to help create the report.
“The updates made in no way impact or affect the substantive content, findings and recommendations in the report,” Deloitte wrote in a section in the updated study.
Deloitte’s member firm in Australia was required to pay the government a partial refund for the report. No information has been made public yet in regards to a potential refund for Canada’s report.
Source link
