Every revenue operator has lived some version of this story. Usually more than once.
In the first version, a new SVP of Sales comes in. You ask what changes they'd like to see in the dashboards so they get the information they need. They say, "I'll trust your judgment. What we have now isn't working for me, so pull together what you've seen work at other organizations."
You build it.
The new dashboard doesn't work for them either. But now they've accepted that they need to give you more specifics if they want the dashboard they're dreaming of, so you start the process over.
In the second version, marketing asks for a dashboard and gives you a lot of detail during requirements gathering.
You deliver exactly what they described.
They tell you it isn't useful and never log in again.
Here's the good news: I'm watching people who threw up their hands and walked away from reporting get re-engaged with their data. Really re-engaged.
The problem was never the dashboards we delivered. The problem is that when someone looks at a dashboard, they immediately have follow-up questions. Dashboards can't slice and dice the data far enough to reach a detailed conclusion, and follow-up questions rarely make it back to RevOps. So the stakeholder declares it useless and moves on.
An LLM layer changes that. It can recognize and isolate patterns, then keep analyzing as questions keep coming in.
However.
LLMs don't follow the same rules and definitions your reports use unless they're explicitly told to. Even then, they occasionally go off the rails and create unnecessary QA work for you.
In my experience, they're still worth the hiccups. I've used this technology to deliver analysis executives loved, at a standard I would have called impossible with the tools we had a few years ago.
Below, I'll walk through the four flavors of AI-driven analytics I've seen, their gotchas, and how to make the most of each scenario. But first, let's talk about the elephant in the room.
No.
Some AI-powered analytics tools are marketing themselves as analyst replacements. That's absolute insanity.
For AI-powered analytics to even begin to function, you need two things:
A quick note on "clean." You don't need pristine contact and company data to get useful information out of your CRM. It's great to have, but it's a moving target that will never, ever be hit. Tomorrow, some of today's data will be wrong because people changed jobs, companies were acquired, and so on.
What you do need:
I went deep on what to fix first (and what can wait) in our article on RevOps data management.
To do both, you need a good analyst. And you need to keep that analyst around, because someone has to gather business requirements and apply the context that only comes from working with stakeholders and end users. That context is what turns a pile of correct numbers into something useful.
Without an analyst, you'll have many meetings derailed by arguments over how many leads marketing sourced, whether sales followed up, and what counts as marketing-sourced pipeline in the first place. You'll also have an irritated, skeptical board wondering why you keep investing in marketing when you have zero idea how it's helping you reach a revenue goal.
(To be clear, marketing is a great investment. But it's hard to justify significant spend when your reporting isn't wired for your actual sales cycle. If you're running a six- to nine-month B2B sales cycle, last quarter's campaigns won't show up in this quarter's closed-won numbers, and your analytics need to account for that.)
The hardest part of all this is getting whatever LLM your end users interact with to ingest, and consistently remember, how you define and calculate your metrics.
It is no longer optional for revenue operations to define and manage how those metrics work. (It never really was.) If your leadership team hasn't agreed on what counts as a marketing qualified lead, an opportunity, or pipeline coverage, start there. Our guide to aligning revenue teams using KPI metrics walks through how to get that sign-off, and it's the first thing I'd tackle before connecting any LLM to anything.
The other problem is that your end users will inevitably interact with multiple LLMs. That means you need to define your source of truth, communicate it until you're blue in the face (and then keep going), and still support whatever systems people insist on using instead. Think of it as job security. And yes, I know it's infuriatingly repetitive.
Because LLMs introduce unpredictability. No matter how hard you work to train them, expect some conflict. Different answers will float around your company. Any time two departments report different numbers for the same metric, the meeting stops being about the business and starts being about whose number is right. We covered how to handle that fight in the "We're All Reporting Different Numbers" section of our change management article.
With that in mind, here are the four flavors of AI-driven analytics I've seen:
Each has pros, cons, and its own brand of gotchas.
This isn't a bad option. Whether it works well depends a lot on who owns it and how responsive they are to requests.
LLMs have made query languages and logic adjustments easier in many ways. You no longer need to write every line of SQL from scratch to answer a new question. But this setup still requires:
The LLM connection itself can take a few forms, ranging from a simple connector to a full Claude Enterprise account pointed at your warehouse. In the Claude scenario, consistency relies on skills: packaged instructions that tell the model how to calculate your metrics and which tables to use. Skills help a lot. But the model sometimes forgets to use them, and when it does, it goes right back to guessing at your definitions.
Ownership is where this option gets complicated. If IT or a BI team owns the warehouse, change requests and new dashboards can get hung up in a queue. (To be fair, this happens when RevOps owns it too.) The bigger problem is translation. Business context gets lost between the person asking the question and the person writing the query. That's the most frustrating part if you're in RevOps and you have the skills but not write access to IT- or BI-owned tools.
How to make the most of it:
This is my preferred path when I'm contracting for smaller organizations, for three reasons:
That said, not all tools are created equal. Ask a ton of questions before you sign anything. Here are mine:
The answers to the first three questions tell you most of what you need to know. A vendor that links records by guessing domain matches, has no governed definition, and queries raw tables on the fly is just a nicer-looking version of the inconsistent-metric problem you already have.
Ask the pricing question early, too. Credit-based pricing can look cheap in a pilot and get expensive fast once the whole sales team starts asking questions.
MCP servers and connectors let people across your company ask an LLM questions about CRM data, format notes, pull segments, and plenty more.
I’m not saying that these tools are bad. Sales leaders can get answers about specific deals without pulling reps into another meeting. Reps can get help drafting next steps. Managers can prep for pipeline reviews in minutes, not hours.
The downside is just as real. It's very easy for everyone to get a different answer when they ask about key metrics. One person's LLM decides pipeline means open opportunities in every stage. Another decides it means stage two and beyond. Both people walk into the same meeting with confidence and different numbers.
The fix is building out a lot of skills, or pre-built prompts, that tell the LLM exactly how to calculate your core metrics. You'll still have more issues than you would with options one or two. But the odds that your team is also using this option (or option four) on top of your source-of-truth tool are so high that you'll need to build this layer even if you insist your reporting tool is the source of truth.
The exception I've found: an MCP connection directly to the tool you built in option one or two, one that uses the data structure, definitions, and schema already living in the originating database or visualization tool. That's the most reliable version of this setup. Even these hiccup occasionally, and you'll get "bug" tickets that turn out to be poorly worded prompts from your stakeholders.
How to make the most of it:
In-app AI is the most difficult flavor to navigate. You'll need to do a lot of training with your team, and even then, it can be a bit of a disaster.
The core problem is anchoring. These tools are built to answer questions about whatever data sits in the CRM, and the CRM doesn't know that your company counts pipeline from stage two onward or excludes renewals from new business bookings. Your reports know that. The AI often doesn't.
I went into this section convinced it was almost impossible to anchor these tools on metric definitions. After digging into the vendor documentation, I haven't changed my mind:
The catch with the Salesforce version is that it's really option one or two wearing a Salesforce badge. You need Data 360, a semantic model someone has built and maintains, and the licensing to match. If you have all of that, great. Most of the teams I talk to don't, and their reps are using whatever AI is built into the screen in front of them.
How to make the most of it:
Regardless of which flavor you pick (and you'll likely end up supporting more than one), the order of operations is the same:
The biggest battle you’ll have when layering LLMs into your analytics environment is managing conflict if you don’t start with data definitions set in stone. Calculate your metrics with consistent, reviewed logic first, and then let the LLM explain, slice, and explore from there.
Do it in that order, and those stakeholders who stopped logging into their dashboards may become your most engaged users. That's been my experience, anyway.
Have you found a way to keep in-app AI anchored to your definitions? Or a vendor that nails the questions above? We'd love to hear about it (and the horror stories, too) in the community.
