I came across this story in the Financial Review. One of the world’s leading consulting firms submitted an AI-generated report filled with fake references and fabricated quotes to the Australian government, and the whole thing was entirely preventable.

What Happened

Here is the short version:

• The project: A $440,000 report for the Department of Employment and Workplace Relations
• The problem: The report was littered with errors including three nonexistent academic references and a made-up quote from a Federal Court judgement
• The confession: Buried in a quietly updated methodology section, Deloitte admitted using Azure OpenAI GPT-4o for “core analytical tasks”
• The outcome: Partial refund, damaged reputation, and a report whose recommendations can no longer be trusted

Deloitte earns significant revenue advising clients on AI implementation while emphasising “the need to always have humans review any output of AI.” They did not follow their own advice.

What Went Wrong (And How to Avoid It)

1. No Disclosure, No Accountability

Deloitte did not disclose AI use in the original report. Every AI-generated or AI-assisted output should be clearly labelled. This is not about legal protection, it is about setting appropriate expectations and making sure the right review processes kick in.

A simple disclosure statement would have changed everything: “This analysis was conducted with the assistance of generative AI tools (Azure OpenAI GPT-4o). All outputs have been verified by qualified analysts against primary sources.”

2. Missing Human Oversight

AI was used for “core analytical tasks” without adequate human verification of references and quotes. AI outputs should be treated as drafts, never final deliverables. Subject matter experts need to verify all factual claims (especially citations), and a checklist should ensure every AI-generated element is validated before it goes anywhere near a client.

One practical approach is a verification workflow where one team member generates with AI, a second validates all references against primary sources, and a third reviews for logical consistency.

3. Wrong Tool for the Job

They used generative AI for “traceability and documentation gaps,” a task requiring factual accuracy and verifiable sources, when LLMs are known to hallucinate references. This is like asking someone who is brilliant at creative writing to do your tax return (I mean, they will give it a red hot go, but you probably will not love the result).

AI is good for drafting structure, summarising verified content, and identifying patterns in existing documentation. It is not good for creating citations, generating factual claims without source material, or filling gaps in documentation with “educated guesses.”

The better approach would have been to use AI to analyse and summarise existing documentation, then have analysts fill genuine gaps through proper research.

4. No Governance Framework

There were no clear policies on when, where, and how AI tools should be used in client deliverables. Before you deploy AI in any business-critical process, you need to define approved use cases, set quality standards and review requirements, create approval workflows for AI-assisted client work, and document everything.

A simple policy matrix goes a long way: “AI may be used for initial drafts and analysis support. All AI outputs require Subject Matter Expert (SME) verification. Client-facing deliverables must undergo additional review if AI-assisted.”

5. Speed Over Quality

The pressure to deliver may have led to shortcuts, using AI to fill gaps quickly rather than doing proper research. I think this is probably the easiest trap to fall into when you are starting out with AI. The temptation to save time overrides the need to verify, and before you know it, you have published something that is built on sand.

Build a culture where quality comes before speed, where team members feel safe raising concerns about AI outputs, and where “I need more time to verify this properly” is an acceptable response.

The Real Cost

Beyond the immediate financial hit and reputational damage, this incident erodes trust in professional services firms at a critical moment when organisations need guidance on AI adoption.

Dr Christopher Rudge, who discovered the errors, put it bluntly: “You cannot trust the recommendations when the very foundation of the report is built on a flawed, originally undisclosed, and non-expert methodology.”

The Path Forward

The Deloitte case study is a clear blueprint of what not to do, and for me it reinforces everything I believe about AI adoption. You cannot bolt AI onto chaos and expect good results. You need foundations first.

What that looks like in practice:

• Transparency about AI use – be upfront, always
• Robust human oversight – AI drafts, humans verify
• Appropriate tool selection – match the tool to the task
• Clear governance frameworks – rules before deployment
• Quality-first culture – slow down to get it right

These are not complex requirements. They are basic principles of responsible AI implementation, the same principles any organisation should follow before deploying AI in any business-critical process.

The question is not whether AI should be used in professional services. It is whether organisations have the processes, governance, and culture to use it responsibly.

Deloitte’s $440,000 lesson is one the rest of us can learn from without paying the price.

You can read the full article here: https://www.afr.com/companies/professional-services/deloitte-to-refund-government-after-admitting-ai-errors-in-440k-report-20251005-p5n05p