This is a great post Justin - I've put similar guardrails to try and reduce hallucinations. I've also found that AI tends to hallucinate much more when you try to force it to give citations for specific numbers and for URLs. When I asked it to cite the specific $ spend in a 10K or the source of the company's strategy from their website, it almost never was able to find the right page number, dollar value, or exact URL.
What I did find though, was giving it specific context as you mentioned really helps. For example, I have a step that extracts financial data from the SEC API and feed this as a structured input into another step, rather than asking the AI to find and cite the numbers itself. Another example is to use a RAG pipeline that pre-indexes documents and feed pre-validated information instead it trying to search and potentially hallucinate sources or answers.
But I totally agree with your post - the more specific and structured context I provide upfront, the less likely the AI fills in gaps with made up info.
- A structured process for structured outputs - way more reliable
- Giving the AI curated context rather than a data dump
I've also found that pre-processing long documents helps a lot -- although as you noted in your post yesterday, needs to be balanced with providing depth/richness of context when asking the LLM to do something creative
This is a great post Justin - I've put similar guardrails to try and reduce hallucinations. I've also found that AI tends to hallucinate much more when you try to force it to give citations for specific numbers and for URLs. When I asked it to cite the specific $ spend in a 10K or the source of the company's strategy from their website, it almost never was able to find the right page number, dollar value, or exact URL.
What I did find though, was giving it specific context as you mentioned really helps. For example, I have a step that extracts financial data from the SEC API and feed this as a structured input into another step, rather than asking the AI to find and cite the numbers itself. Another example is to use a RAG pipeline that pre-indexes documents and feed pre-validated information instead it trying to search and potentially hallucinate sources or answers.
But I totally agree with your post - the more specific and structured context I provide upfront, the less likely the AI fills in gaps with made up info.
Super valuable examples!
- A structured process for structured outputs - way more reliable
- Giving the AI curated context rather than a data dump
I've also found that pre-processing long documents helps a lot -- although as you noted in your post yesterday, needs to be balanced with providing depth/richness of context when asking the LLM to do something creative