Veracity AI
User Guidelines
Thanks for trying out veri-fact.ai. It uses AI (specifically a 'Large Language Model' (LLM)) to summarize relevant text from reliable sources retrieved from the internet. Some things to consider when using it:
- The app will always give a response. To get a sensible response, ensure what you submit is ‘fact-checkable’. We suggest:
- Making sure there is a verifiable statement expressed somewhere in the text you submit. You are not likely to get sensible results from checking an opinion, a difficult-to-verify reason for something, or a little-known fact.
- Making sure there is only 1 statement.
- You may still not get a sensible response, because LLMs can sometimes give wrong (and often 'sounds right') explanations as an answer.
- It’s up to us, if we want to use these systems, to assess the quality of the LLM’s response and have a sense for when the AI may have trouble giving a reliable response:
- Is the statement you want to fact-check well-known enough that it is likely to appear in multiple places on the internet? Some cases where it may not:
- a breaking news event: there may not be very many websites with relevant content and those search results that are returned may be of low reliability since the sources mentioning the event are likely either random passersbys with little understanding of the context or involved parties that might have hidden motives in how they talk about it.
Suggestion: Double-check the sources to the right of our page. Do they sound familiar? Do they look typical?
- a breaking news event: there may not be very many websites with relevant content and those search results that are returned may be of low reliability since the sources mentioning the event are likely either random passersbys with little understanding of the context or involved parties that might have hidden motives in how they talk about it.
- What if all the sources analyzed have low or absent source credibility? The system will likely identify this in its answer, but it is always good to check the source credibility scores for yourself. Note that the database veri-fact.ai uses only has so many entries. If the returned website comes from a source that isn’t in the database, that doesn’t necessarily mean it is unreliable, but it does indicate it is not a well-known source. The less well-known a source is, the less likely it is to be based on expert knowledge.
- Is the statement you want to fact-check well-known enough that it is likely to appear in multiple places on the internet? Some cases where it may not:
- Using LLMs to help us fact-check is a new area of research where new ways of doing it are proposed frequently and how well they do is also improving. The developers of veri-fact.ai aim to use the most up-to-date research about how to best use LLMs for fact-checking. They also hope to improve it and you can help here! You are invited to share your feedback that will be used to identify and try to correct problems you encounter, as well as to improve the overall user experience. The veri-fact.ai Team thanks you in advance for your participation in this collective experiment to better ground our public discussions in factuality!
- If you would like to know more technical details about how the AI behind the app works click here