AI models are not static; they have some inherent variability and evolve over time. That means your Visibility Audit (formerly known as a report) results can shift even if you run the same prompts multiple times consecutively.
This variability is normal and expected. It happens because AI models like ChatGPT, Claude, Gemini, and Perplexity are influenced by:
- Ongoing training updates (new data added behind the scenes)
- Live internet crawling
- Model tweaks (changes made by the companies that operate the models)
- Contextual randomness (LLMs don't always give identical answers, even to the same prompt)
The last factor might give you pause: yes, there is an element of randomness in how all AI models work. While they're incredibly advanced, at their core, these models generate responses one word at a time based on weighted probabilities; essentially, they choose the next most likely word from a list of possible options. It's not a coin flip, but more like rolling a loaded die where certain outcomes are much more likely than others.
In tightly competitive categories (think Nike vs. Adidas), this can lead to small variations from one run to the next. If the model considers both brands equally valid answers to a prompt, it may prioritize one over the other simply due to minor shifts in phrasing or probability. That's why visibility scores can fluctuate and why keeping your Visibility Audit running on a schedule helps spot true patterns over time.
What does this mean for my Visibility Audit?
Each Visibility Audit is a diagnostic point-in-time check that simulates how a persona might interact with an AI model today, not a historical log or a permanent record. The more frequently it runs, the more accurate your baseline visibility score will be.
Once you understand your base range and begin making adjustments to improve your scoring, you'll notice new trends as your Visibility Audit runs.
How do I use variability to improve my strategy?
Tracking variability over time is a key insight that Gumshoe provides. It helps you:
- Understand how stable or unstable your visibility is across models
- Spot when a model starts responding differently, positively, or negatively
- Detect the impact of external factors
- Adjust your strategy as models evolve
Your Visibility Audit already runs on a schedule set by your subscription, which helps you:
- Set a baseline
- Monitor change over time
- Respond quickly when something shifts