For years, I recommended two KPIs to measure whether a data function was delivering: data utilization rate and data ROI. I was wrong about both. Not because they are irrelevant, but because they answer a question that is disconnected from the decisions you actually need to make. This article explains why, and what to measure instead.
The appeal of utilization and ROI
Data utilization sounds like exactly the right thing to track. If more of your business teams are using the data products your data team builds, that is a signal things are working. And ROI, return on data investment, is the language of every CFO conversation. Of course you should be able to say what the data team is returning on the spend.
The problem is not the intent. The problem is that neither metric tells you what to do next.
When data utilization drops, what do you change? Is it a training problem? A product quality problem? A trust problem? The number does not say. When ROI is hard to calculate, and it almost always is, because attributing business outcomes to specific data products is genuinely difficult, you end up with an imprecise number that invites debate instead of driving decisions.
These are vanity metrics. They can look fine while the underlying system is quietly failing.
What the system actually needs you to track
There are two things that determine whether a data function is healthy or heading toward a crisis. The first is the cost the business is paying to work around the data. The second is the cost the data team is paying to keep the current portfolio alive.
Neither of these appears on a utilization dashboard.
The business cost is all the time your non-data employees spend compensating for data that does not quite work: manual quality checks before any number gets shared, spreadsheets built to bridge the gap between the official dashboard and how the team actually operates, re-running analyses that should have been self-serve but require a conversation with an analyst. This is real cost, denominated in hours per week, multiplied by salary. In most organizations, it is significant, and it is invisible to leadership because it hides inside existing headcount.
The data team cost is the weight of the existing portfolio. Every data product a team has shipped still requires ongoing attention, monitoring, bug fixes, schema changes when upstream data changes, support for users who cannot interpret the output. As the portfolio grows, that maintenance burden grows. The question is not how many products you have shipped, it is whether the team still has the capacity to ship more without the existing portfolio collapsing.
When these two measures are tracked together, you get a picture of the system's health that utilization and ROI cannot give you.
The connection that makes these metrics useful
What makes these two measures more than just interesting numbers is that they connect directly to the decisions you are trying to make.
Choosing which initiative to fund next becomes a different conversation when you can ask: which capability is going to reduce the business cost most significantly? Not which dashboard looks most valuable in a sprint review, but which one frees the most hours from your marketing team's weekly workaround process, or eliminates the reconciliation step that finance runs before every board report.
Deciding whether to hire comes down to something concrete: is the team's capacity being consumed by maintaining what already exists, or is there genuine room to build? If the answer is the former, hiring adds people who immediately get absorbed into the same maintenance load. If the answer is the latter, there is a case for growth.
Evaluating whether to retire a product stops being political and becomes financial: what is this product costing the team to keep running, and is the business benefit it provides worth that ongoing cost?
Why the standard KPIs survive despite being wrong
Utilization and ROI persist as the default metrics for the same reason most bad metrics persist: they are easy to present in a slide deck and hard to argue with in a room. Nobody challenges you for tracking utilization. Nobody challenges you for wanting to prove ROI.
The harder conversation, here is exactly what the business is spending on data workarounds, and here is the maintenance debt the team is carrying, requires measurement that most data functions have never done. It requires time to map how business teams actually use data, not how they are supposed to use it. It requires an honest assessment of the portfolio's true cost, not just the build cost.
That work is not glamorous, and it does not produce a compelling chart for the quarterly review. But it produces something more valuable: a number that survives a CFO conversation, because it is denominated in cost rather than sentiment.
The shift in conversation
The first time a leadership team sees the real cost of the business workarounds, expressed as hours per week and annual salary, the reaction is usually surprise. Not because the data team was hiding it, but because no one had ever added it up before.
That number changes the budget conversation. It stops being about whether the data team deserves more resources and starts being about which investments will recover the most organizational capacity. That is a question any CFO can engage with directly.
The second measure, the weight of the existing portfolio, changes the roadmap conversation in the same way. Instead of debating which initiatives are most exciting, you are asking which ones can actually be delivered given what the team is already carrying. That is a productive question. The other question leads to a shopping list of initiatives that is out of date by March.
What to do with this
If you do not currently track either of these measures, the starting point is a two-week measurement exercise. Ask your business teams how much time per week goes to data preparation, reconciliation, and workarounds. Ask your data team to categorize how their hours split between new work and maintenance on existing products.
Two weeks of honest data will show you more about the health of your data function than a year of utilization reporting.
If the numbers surprise you, which they usually do, that is the start of a real conversation. Book 30 minutes here to walk through what the numbers mean and what it would take to move them in the right direction.