Have you ever thought about your place in the world and whether the contributions you’ve made are impactful and memorable? Most of us can think of some good things we’ve done that have made a difference for at least one individual. But have you done enough to be important in the eyes of AI?
I found an interesting website (intheweights.com) that queries over 15 of the most well-known models with the question “who is this person,” clusters the answers, and provides a “strength score.” I was curious to know if what I have done was deemed worthy of being included in training AI models, so I put my name in. This site showed me that the Gemini 3.1 Lite and Kimi K2 0905 models knew who I was and described me as “A senior consultant and expert in data management, business intelligence, and analytics, often associated with enterprise data strategy.” This came with a strength score of 115, which indicates a very high likelihood of being accurate (it is).
Intheweights.com recognizes that AI hallucinations can happen and also provides a list of models that contain your name with possibly fabricated information. For me, this included being listed as a music video and film director, a former NHL hockey player with the Philadelphia Flyers and other teams, a fictional character from the TV show “The Office,” and a politician from Pennsylvania. As entertaining as these careers sound, none of them are true. It also does make me wonder why multiple AI models insist on creating hallucinations about me that are connected to the state of Pennsylvania. Why is this happening?
What’s going on in the background is that the answers are coming from each model’s recall of training data, not a live search of current data. There is no source or attribution. This site’s creators admit that models can hallucinate details and that common names exacerbate the situation. Good thing my name isn’t John Smith!
This is a low-stakes and funny example of a high-stakes issue in enterprise AI: models generating confident and specific wrong answers when they are working from memory instead of data that is grounded and governed. If this occurs with something as simple as “who is Steve Cavolick,” it happens with product specs, customer histories, and financial figures, but in a much quieter way because no one is doing live fact checking of their agents while in the middle of a workflow. In this scenario, the failure mode is not the AI itself. It’s that no one built a governance layer to catch it.
To avoid this issue, organizations that deploy enterprise AI must have controls in place. The first method is using Retrieval Augmented Generation (RAG) to ground LLMs with external data, such as your company files. However, even with RAG, hallucinations can still occur because of poor retrieval or due to a model ignoring context. If RAG pulls the wrong document, you’ll get a wrong answer. This is where your second line of defense comes in, which is controlling the generation of answers.
Once the model retrieves the correct documents to answer the question, the prompt must explicitly force the model to rely on the provided information. You can add system prompt rules like “If the information is not in the provided context, state ‘I don’t know’ instead of guessing.” You can even have the model cite the source to a specific document and page before it begins to answer the question.
An industrial-strength RAG pipeline should include automated checks to measure how factual the answer is against the context of the retrieved document. You could also use a separate model to score the final output for answer relevance against the context before it is presented to the user. AI observability frameworks such as IBM’s watsonx.governance excel at tracking accuracy of output over time and should be a part of your AI architecture.
If you want AI to be at the forefront of your company’s transformation, it must have governance and verification included from the get-go. It can’t be bolted on later after the first bad answer reaches a customer. This is the difference between AI that sounds right and AI that is right. It’s funny when the internet thinks you scored a hat trick. It’s not funny when AI tells your customer something false with the same conviction.
If you don’t yet have an AI data architecture you can trust, you don't have to build it alone. Please Contact LRS to book a meeting with me and our AI specialists, and let’s get your governance strategy right from the start.