Watch or Ask

Most of your dashboards exist to answer a question someone asked once. That was always the wrong tool for the job.

A sales engineer wants to know something specific at 9:40 on a Tuesday. Not a number on a screen. A question: a prospect she’s been circling just announced it’s acquiring a competitor, and she needs to know whether the deal pulls them toward the product she sells or away from it, and which piece of the platform the acquisition suddenly makes relevant to them. She isn’t asking what the pipeline dashboard shows. The dashboard was built eighteen months ago for some standard sales process, by someone who has since left, and it answers the question that person was handed back then. It doesn’t answer this one.

So what does she do. She pings the data team. Someone says they can pull it, give them a day. By the time the view exists she’s already had the call, or the window’s closed, and the question has already moved to the next thing.

This happens a hundred times a day in every institution. The question is alive and the tooling is frozen.

two jobs wearing one name

We built business intelligence around a thing we called the report, and we never noticed it was doing two completely different jobs.

The first job is watching. You have a number you care about and you want to see it the same way, day after day, so you can tell when it moves. Daily P&L. Risk limits. Pipeline coverage against target. Cloud spend against budget. This is what dashboards are good at, and nothing here is going anywhere. If you watch something repeatedly, you want it to look the same every time you look. The dashboard is the right tool. Keep it.

The second job is asking. You have a question you’ve never asked before, in a shape nobody anticipated, and you need the answer now, once. The sales engineer’s question is this. Nobody built a dashboard for it, because by the time they did you’d be asking something else.

We jammed both jobs into the same tool because, for thirty years, there was no other way to do the second one. Asking your data a new question required someone who could write the query. So every new question became a ticket, every ticket became another bespoke view, and that’s how you end up with two hundred Tableau workbooks that three people have bookmarks to and nobody trusts.

The watching job was served well. The asking job was served terribly. We just couldn’t see it, because the constraint was so total that we mistook it for the nature of the work.

what changed

The constraint is gone.

You can now put an agent in front of your data that understands your domain, has access to the systems and the tools, and can go answer a question it has never seen before. Not build you a report. Answer the question.

The sales engineer asks what the acquisition changes for this prospect. The agent pulls the prospect’s stack and the competitor they’re buying, pulls the deal’s stated rationale, weighs it against what her platform actually does, and tells her: this pulls them toward you, not away. The integration work they’ll need to fuse two systems is exactly what your platform handles, and here are the two modules to lead with. Ten seconds. No ticket. And when she asks the follow-up, because there’s always a follow-up, it answers that too.

This isn’t domain-specific. It’s the same shape everywhere.

The head of finance wants to know why gross margin slipped this quarter. Not the margin dashboard that shows the slip and stops there. The actual question: which product line, which segment, and whether it’s discounting that ran loose or a cost that crept in quietly. The agent has the billings data, has the cost ledger, and it goes and looks.

The engineering lead wants to know why the cloud bill jumped eleven percent. Not the cost dashboard that shows the jump and nothing else. The question is which service, whose workload, and whether it’s the thing that just shipped or something that’s been quietly creeping for months. The agent has the billing export, has the deploy history, and it correlates them.

Three people, three worlds, one shape. A specific question no standing report was ever going to answer, handed to something that can actually go dig.

the danger nobody mentions

Here’s the part that should make you cautious, because it made me cautious.

A one-off answer is more dangerous than a dashboard, for exactly the reason it’s more useful. When you watch a number every day, you develop a feel for it. If the dashboard says revenue tripled overnight, you know to doubt it. The one-off question has none of that. You ask it once, you get a confident answer, and there’s no baseline sitting next to it whispering that something looks off. A wrong answer has nothing to catch it.

A few weeks back I wrote about an economist who asked a model for an inflation figure and got a wrong one, because the model tried to remember the number instead of looking it up. He decided AI was unreliable. The real lesson was narrower: don’t ask it to know things, ask it to find things.

That distinction is the whole game here. An agent that guesses at your numbers is worse than the stale dashboard, because at least the dashboard is wrong in a way you can predict. An agent that runs the actual query against the actual data and shows you what it pulled is a different animal. It isn’t remembering. It’s fetching. The answer is only as good as the agent’s discipline about going and getting it rather than sounding like it already knows.

And the trust doesn’t come from the answer sounding fluent. It comes from watching it be right, over and over, on questions where you could check. You trust the agent the way you trust an analyst: not because they speak confidently, but because they’ve been right before and they show their work. Fluency is free now. The track record isn’t.

the proof

Here’s what changed for me.

As I mentioned in a previous post, we replaced our CRM. The Salesforce consultants lost their minds. You couldn’t possibly have replaced all of Salesforce, they said, and so on. And yeah, of course I didn’t replace everything. An important part of the process was confirming that an agent could replace the need for a number of custom built reports.

The data lives in a structured database and the way I interact with it is conversational. The agent understands the domain. The deal stages. The relationship types. The difference between someone I’m prospecting and someone I’m already building with, between a warm intro and a cold one.

When I need to know something, I ask. The agent queries the actual data and answers. If it should be a table, it’s a table. If it should be a list, it’s a list. If I need an export, I get one. But I’m not building reports. I’m not maintaining dashboards. I’m not wondering whether the number on the screen matches the number in the system, because there’s no screen and no snapshot. There’s just the data and a question.

The data is alive because it’s being asked, not frozen into a picture taken last quarter.

the smallest version that works

You don’t need a platform to feel this. You need one dataset and a model that’s allowed to use a tool.

Here’s the whole trick, and it’s the same trick the early prompts.finance posts were built on: the leverage is in the instructions, not the query. You don’t write the SQL. The agent writes the SQL. What you write is the operating discipline that keeps it honest.

Start with one source. Not six. Take a single dataset you already trust, last quarter’s pipeline export, your revenue numbers, your cloud billing CSV, and put it somewhere a model can query, even a local database on your laptop. Then give the model exactly one tool, the ability to run a read-only query against that data, and a system prompt that encodes three rules.

You are an analyst with read-only access to [one dataset].

You answer questions by querying the data, never from memory.

Rules:
1. Never state a number you did not retrieve with a query. If you
   cannot answer it with a query, say so and stop. Do not estimate.
2. With every answer, show the exact query you ran. The user must
   be able to check your work.
3. If a question is ambiguous (which date, which location, which
   definition), ask one clarifying question before querying.
   Do not guess the interpretation.

Domain notes:
- [what the tables and columns actually mean, in plain language]
- [the stages, relationship types, and categories that matter]
- [the distinctions a smart newcomer would get wrong]

Those three rules aren’t cosmetic. They’re the entire difference between the thing I just described and the economist’s wrong inflation number. Rule one is fetch, not guess. Rule two is a track record you can audit on every single answer. Rule three is the agent admitting it doesn’t know which question you’re asking, instead of confidently answering the wrong one. Headquarters or the contact’s location. Committed or called capital. The agent that asks is worth ten that assume.

The domain notes are where it actually gets good, and where you, the person who knows the business, matter more than the model does. A generic agent pointed at your data will call a lapsed account active and a warm intro cold. The notes are how a borrowed analyst becomes yours. That’s not a one-time setup. It’s the thing you keep sharpening every time the agent gets something subtly wrong, which is also how you come to trust it.

what actually stays

So no, as much as I’d like to say they are, reports aren’t dead.

If you need to track a KPI consistently, you want it to look the same every time. Keep the dashboard. If you have a real-time operational view, a control room watching live telemetry, an ops team watching a system, the dashboard is the right interface and an agent is a worse one. Board decks need fixed formats. Regulatory filings need fixed formats. The watching job is alive and well.

What’s not long for this world is the standing BI team whose entire function is to manufacture bespoke domain dashboards, one per question, forever. That model existed to solve the asking problem with watching tools, because watching tools were all we had. The moment you can just ask, the dozens of half-trusted views stop being an asset and start being a liability. Nobody is going to staff a team to keep building them.

The questions your business actually runs on were never report-shaped. We just didn’t have anything better to pour them into.

We do now.