Of late, really late late, AI, and large language models in particular, have started gaining my respect. Two years back, an earlier version of chatgpt produced literally garbage, it returned a set of social science papers that never existed. This to me was without doubt, a blow to the trust quotient in AI projects. I quickly abandoned chatgpt and moved over to Google’s beta project. Bard. Later , Gemini. After the static incompetence of Chatgpt. Bard’s anchoring on google was a refreshing possibility. And yet, Gemini too quickly showed its lack of depth in my research endeavours.
I was back with chatgpt. Which by then reinvented itslf. This was like the Indian second innings comeback in the recent Border Gavaskar trophy. It was, while far from perfect, pleasantly way better in serving my knowledge needs. It was delivering, with obviously a much better deep learning model at the backend. More importantly, it has cut down the tendency to misinform with a straight face. It learned to caution when it was unsure, thus asking the human in the middle to be more cautious while using the AI tool.
This ability to misinform is a threat , a real risk in many fields – particularly in medicine where lives are at stake. This risk is different from AI biases which are equally or more harmful. This risk is about earnestly selecting the wrong information, as if the AI model is in some sort of delusion. This risk today goes by the name ” Hallucination Risk”.
Lets try to dig a little deeper into this risk and our ability to contain its costs.
To begin with generative AI, powered by advanced deep learning techniques, has revolutionized the way we create content, engage with technology, and solve complex problems. However, this transformative capability comes with challenges, the most critical issues is the risk of hallucination—a phenomenon where AI generates information that is incorrect, fabricated, or misleading while presenting it as factual or accurate.
In this blog, we’ll delve into what hallucination in AI is about, why it happens, the implications of hallucinations, and the measures being taken to mitigate this issue.
While hallucinations can be benign—such as a chatbot inventing a fictional scenario—they can also have severe consequences in applications like medical diagnostics, legal documentation, or academic research.
Hallucinations arise due to the way generative AI systems are designed and trained. Here are the primary reasons:
1. Training Data Limitations: AI models are trained on vast datasets scraped from the internet, which may contain inaccuracies, biases, and conflicting information. If the training data is flawed, the model inherits those flaws. Garbage in, garbage out
2. Lack of Context Awareness: Generative models work by predicting the most likely sequence of words based on input. They lack a true understanding of context, which can lead to plausible but incorrect outputs. This is a real challenge, human brain is far superior in this area of contextual judgement.
3. Over-Optimization for Fluency: AI models prioritize generating coherent and natural-sounding text. This fluency can mask errors, making fabricated information appear credible. Here, trying to sound human like often leads to trade offs in production of relevant content. AI is yet to figure out the balancing act. Its a kid in front of human intelligence.
4. Absence of Grounding Mechanisms: Many generative models do not have a built-in mechanism to verify facts against reliable external databases, leading to unchecked outputs.
Implications of Hallucinations
The potential risks of hallucinations vary depending on the application:
Healthcare: An AI system providing incorrect medical advice or misinterpreting symptoms could endanger lives.
Legal: Fabricated legal precedents or inaccurate documentation can undermine trust and lead to significant consequences.
Education and Research: Misinformation in academic work can perpetuate falsehoods and erode the credibility of AI as a tool.
Misinformation Spread: Generative AI could contribute to the dissemination of false information, impacting public opinion and decision-making.
Strategies to Mitigate Hallucinations
The AI research community and developers are actively working on solutions to reduce hallucination risks:
1. Enhanced Training Data Curation: Ensuring that training datasets are diverse, accurate, and free from biases can reduce the likelihood of hallucinations.
2. Reinforcement Learning with Human Feedback (RLHF): By incorporating feedback from human reviewers, models can learn to align better with factual and ethical standards.
3. Fact-Checking Mechanisms: Integrating real-time fact-checking tools or grounding the model in external, verified data sources can enhance accuracy.
4. Explainability and Transparency: Developing models that can justify their outputs or trace the source of their information helps users assess reliability.
5. Prompt Engineering: Crafting precise and unambiguous prompts can guide the model towards generating accurate responses.
6. Domain-Specific Fine-Tuning: Tailoring models for specific fields, like law or medicine, with domain-specific data reduces the risk of errors in critical areas.
The Road Ahead
As AI systems evolve, their ability to distinguish between fact and fiction will improve, but it is equally important for users to approach generative AI with caution. Recognizing its limitations, validating critical outputs, and implementing checks and balances will ensure these tools are used responsibly and effectively.
Generative AI holds incredible potential, but its development must go hand-in-hand with accountability. By addressing the issue of hallucinations head-on, we can build AI systems that are not only powerful but also trustworthy and aligned with human needs.
Suggested Reading
1. NIST AI 600-1, August 2024. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf.
2. U.S. Government Accountability Office. “Science & Tech Spotlight: Generative AI.” GAO-23-106782, June 2023. https://www.gao.gov/products/gao-23-106782.
3. State of California. Benefits and Risks of Generative Artificial Intelligence Report. November 2023. https://www.govops.ca.gov/wp-content/uploads/sites/11/2023/11/GenAI-EO-1-Report_FINAL.pdf.
4. Cloud Security Alliance. “AI Hallucinations: Generative AI’s Costly Blunders.” April 23, 2024. https://cloudsecurityalliance.org/blog/2024/04/23/ai-hallucinations-the-emerging-market-for-insuring-against-generative-ai-s-costly-blunders.
5. North Carolina Department of Information Technology. “Use of Publicly Available Generative AI.” Accessed December 22, 2024. https://it.nc.gov/programs/privacy-data-protection/artificial-intelligence/use-publicy-available-generative-ai.
6. New York State Office of the Attorney General. “Symposium Report on the Next Decade of Artificial Intelligence.” Accessed December 22, 2024. https://ag.ny.gov/symposium-report-next-decade-ai.
7. U.S. Government Accountability Office. Generative AI. GAO-23-106491, June 2023. https://www.gao.gov/assets/830/826491.pdf.
8. Recent interview by Sarah Bird ( December 18, 2024, Financial Times) on this issue is worth a read. Link : https://www.ft.com/content/aac74337-cb3f-43e7-894a-d85afedd36101. National Institute of Standards and Technology. Artificial Intelligence Risk Management Framework: Generative AI.
Leave a Reply