Claude + MCPs Isn’t a Semantic Layer: What Breaks When You Analyze Your Business Without One

TL;DR

• Ask Claude the same metric question twice and you can get two different numbers. Your playbook tells the model what a metric means. It does not tell it how to calculate it.
• Most MCPs give the model a partial view of your data by default. Neither you nor the model can tell from the answer.
• The AI is doing statistics in its head, not in a real database. Ask for the same correlation twice and you may get two answers.
• A semantic layer is where metrics get defined once, with an owner and a record of every change. A playbook cannot do any of that.

Ask Claude for your average sales cycle today. Ask again next Tuesday. Same prompt, same MCPs, same playbook. There is a real chance the two numbers do not match.

The reason is not exciting, and it costs you decisions.

If you have wired Claude to your CRM, product, and finance tools through MCPs and given it a detailed playbook, you are already ahead of most companies asking whether AI can analyze the business. The gap between that and running the company off the answers is real, though. The stack works, but it is missing a layer.

Here are the four things that break without one.

Your metric definitions drift between sessions

Your playbook tells Claude what a sales cycle is. It does not tell Claude how to calculate one. So every session, the model picks: median or average, deal-created or deal-first-touched, closed-won only or closed-won plus closed-lost. A different pick each time, and you cannot see the pick from the answer.

A real example from our own data: 20 days by median, 66 days by average. Same deals. A handful of long enterprise deals pull the average up. The median ignores them.

Two defensible numbers. One question. The playbook cannot tell you which belongs on the board slide.

A metric that shifts between sessions is not a metric. It is a range the model picks from. If your “sales cycle” is anywhere from 20 to 66 days depending on the day, your team cannot plan against it, and your board cannot compare it to last quarter.

The fix is not a longer playbook. Playbooks control vocabulary, not calculation. A semantic layer defines the math once, runs it the same way every time, and returns the same number for every person and every AI that queries it.

Your MCP is probably showing the model a partial view of your data

Most MCPs were not built to pull entire datasets. They expose what the source API makes easy to expose. Which usually means paginated results, filtered views, and default caps that trim months of history down to whatever fits in one response.

Claude does not know it is working from a partial view. Neither do you. The model runs the analysis on what it received and returns a confident number. If the retrieval covered 40% of your deals, the answer covers 40% of your deals, presented as if it covered all of them.

Pulling data well across sources is boring engineering work. Pagination, rate limits, retries, auth token refresh, schema changes as sources evolve. Plus keeping a stored copy of the data in sync over time, so the AI is querying a warehouse, not scraping the source live on every question.

The stored copy is what lets you run a correlation across six months of pipeline data, or a lag test comparing this quarter’s spend to next quarter’s demand. Without it, the AI is working with whatever the source API is willing to hand over in the current session. Which is rarely six months.

The Databox engineering team has written about the rate-limiting problem alone in real depth. Rate limiting is one of maybe eight problems in that stack. None of it is glamorous. All of it is the difference between “I asked Claude and got a number” and “the number reflects our actual business.”

Test your own setup fast. Ask Claude for a deal count in your CRM for a specific quarter. Then pull the same count from the source’s admin panel. Match? Your MCP is doing more than most. Off by 10%? Every analysis you ran last quarter was off by more than you think.

The AI is doing math in its head, not in a real database

When Claude calculates a correlation or a regression in the chat, the model is approximating. Sometimes very well. Sometimes badly, and you cannot tell which from the output. Ask for the same correlation twice and you may get two answers. Not because the data changed. Because the model reasoned through the math again.

A real query engine runs the same calculation the same way every time. Faster than the AI approximating it. Cheaper on tokens. And the answer holds up when someone asks how you got it.

Trend detection, correlation, regression, anomaly detection. Every calculation that decides whether a signal in your business is real belongs in a database engine, not in a language model doing arithmetic in its head. The AI’s job is to ask for the calculation and explain the result.

One fair counter: for directional questions, a rough answer is fine. Most day-to-day queries do not need audit-grade math. Agreed. The problem shows up the moment a rough answer becomes the number in a comp review, a forecast, an annual plan, or a board deck. At that point, “roughly right” is the same as “you cannot defend it.”

A semantic layer is not a smarter MCP or a longer playbook

Three separate jobs get collapsed in a Claude + MCP + playbook setup:

• Getting the data to the AI. That is retrieval. That is the MCP.
• Telling the AI how you think about the business. That is context. That is the playbook.
• Defining the metric once, with an owner, a version, and a record of every change. That is governance. That is the semantic layer.

Playbooks solve context. They do not solve governance. The moment two people in your company disagree about what “activation” means, the playbook cannot settle it, because the playbook was rewritten last Tuesday and nobody logged the change. A semantic layer can. Every metric has a named owner. Every calculation has a version. Every change has a timestamp. When Claude queries the layer, it pulls from the verified version by default. You get an auditable answer to “what exactly are we measuring, and says who.”

The same layer is where cross-source questions get answered. “Deals per rep by product tier” needs data from your CRM joined to data from your product analytics tool. That join has to happen somewhere. Inside a semantic layer, it is a defined view. Inside an AI’s chat window, it is a guess the model is making across tables it barely holds in memory.

If you act on a number, there is a record of what that number meant when you acted on it. Nobody wants that record until the moment they need it. By then, the playbook has been rewritten six times.

What good looks like

A common growth question: “How is our signup volume trending?”

In a Claude + MCP + playbook setup, the AI grabs signup data from whichever source it hits first (Mixpanel, HubSpot, or GA4), calculates a trend, and returns a chart. If your team defines signup slightly differently in each tool (unique users in one, account creations in another, tracked events in the third), the number the AI returns depends on which tool it landed on. You cannot see that from the answer.

In a semantic layer, the same question surfaces the ambiguity before the AI runs any math. The AI comes back with: “Signup volume from Mixpanel, HubSpot, or GA4?” Once you pick a source: “There are three metrics defined here. Here is how each one is calculated. Which one?”

The AI can ask those questions because the definitions exist underneath it. And you get an answer to the question you actually asked, not the one the AI guessed you meant.

Define the metrics once, and the AI’s first move becomes a question back, not an answer.

Test it against your own setup

Four questions worth running against any Claude + MCP setup already in use:

• When Claude runs statistics, is it doing the math itself or handing off to a real engine?
• Are your metric definitions holding steady from one session to the next, or moving under you between reviews?
• When you pull through an MCP, are you certain you are getting the full dataset, or a partial view that looks complete?
• If you act on a number, is there a record of what that number actually meant?

Your playbook and definitions answer some of these on paper. The real question is whether that lives in prompt files you maintain, or in a layer the data flows through before it reaches the model. Only the second one answers all four questions in practice. The difference is not sophistication. The difference is whether the numbers your team runs on can survive being audited by the executive who acted on them.

To learn more why AI agents need a semantic layer read: databox.com/semantic-layer-for-ai