"We have a policy" doesn't answer the question anymore. Regulators and boards want to know whether you're actually governing AI. That means quantified metrics: how much of your AI you can see, how much of your high-risk use is assessed, how fast you respond to new use cases and incidents, and whether governance is improving over time. A policy document is an input. KPIs are the output. Here are the metrics that belong on an AI governance dashboard and how to use them so you can answer "are we actually governing AI?" with data, not reassurance.
Why Metrics Matter
Documentation alone doesn't prove governance. You can have a policy and still have shadow AI everywhere. You can have an inventory that's 20% complete. You can have a process for high-risk use that takes six months, so nobody uses it. Metrics force honesty. How many AI systems do we have in the inventory versus how many we estimate exist? What percentage of high-risk systems have a completed impact assessment? How long does it take to get a new use case reviewed? How many AI-related incidents have we had, and are we closing them? When you put these on a dashboard and review them with the governance committee or the board, you stop claiming progress and start measuring it. You also give yourself a basis for prioritization. If inventory coverage is low, that's the focus. If high-risk AIAs are lagging, that's the focus. If mean time to review is too long, the process needs to change. KPIs turn governance from a narrative into a management discipline.
Inventory Coverage: Inventoried vs. Estimated Total
The first question is visibility. How many AI systems or use cases do you have in your inventory? How many do you estimate exist (from discovery runs, CASB, surveys, or extrapolation)? Inventory coverage is the ratio: inventoried divided by estimated total, expressed as a percentage. It doesn't have to be perfect. "Estimated total" can be a range or a point estimate that you refine. The point is to show whether you're gaining or losing ground. If you have 12 systems in the inventory and you estimate 40 in use, coverage is 30%. If next quarter you have 18 inventoried and still estimate 40, you're at 45%. The trend matters as much as the level. Define "AI system" or "use case" consistently (e.g., one entry per distinct system or per distinct use case per system) so the numerator is comparable over time. Track this monthly or quarterly. When coverage is low, the KPI drives investment in discovery and inventory hygiene. When it's high, it confirms that visibility is under control and you can focus elsewhere.
High-Risk Systems with Completed Impact Assessments
Not every system needs an impact assessment. High-risk ones do. The next KPI is: of the systems you've classified as high-risk (or equivalent), what percentage have a completed, current impact assessment (AIA)? Count high-risk systems from your inventory. Count how many of those have an AIA that's in date (e.g., completed in the last 12 months or updated since the last material change). The ratio is your coverage. If you have 10 high-risk systems and 4 have a current AIA, you're at 40%. Regulators and auditors will ask this. Boards will ask. Track it quarterly. Set a target (e.g., 100% within 12 months of classification) and report progress. This metric directly supports "we're assessing our high-risk AI." When the number is low, it drives prioritization: which high-risk systems get the next AIAs, and who owns them.
Shadow AI Detection Rate
Shadow AI is AI use you didn't sanction or don't yet have in the inventory. You can't measure "total shadow AI" precisely, but you can measure how much you're finding. Run periodic discovery (CASB, SSO, network, surveys). Count how many new or previously unknown AI systems or use cases you found in the last period. Shadow AI detection rate can be expressed as: number of previously unknown AI systems or use cases discovered in the period, or the ratio of new discoveries to the size of the inventory at the start of the period. You'll always find something. The goal is to see whether discovery is working and whether the rate is falling over time as governance matures. If you discover 5 new systems every quarter and your inventory grows by 5, you're keeping pace. If you discover 20 and your inventory grows by 2, you have a visibility problem. Track discoveries per quarter and the size of the inventory. Use the metric to justify discovery cadence and to show that you're actively looking, not assuming the inventory is complete.
Mean Time to Review New Use Cases
Governance that's too slow gets bypassed. If it takes 10 weeks to get a new AI use case classified and approved, teams will go around. Measure how long it takes from intake (someone submits a use case for review) to decision (classified and approved, conditionally approved, or denied). Mean time to review (MTTR) is the average of that duration for use cases closed in the period. Track it by risk tier if you can: high-risk might take longer than standard. The metric tells you whether the process is fit for purpose. If MTTR is 8 weeks and the business expects 2, you have a bottleneck. Use the metric to drive process change: delegation, parallel review, or clearer intake so that cycles shorten. Report MTTR monthly or quarterly. Set a target (e.g., standard use cases within 5 business days, high-risk within 4 weeks) and track against it. This KPI answers "does governance slow us down?" with data.
Incident Count and Closure
AI-specific incidents (hallucination reaching users, bias flagged, prompt injection, data exposure, etc.) are a direct signal of risk and of response effectiveness. Count incidents per period. Count how many were closed (contained, remediated, and post-incident review done) within a target SLA (e.g., Severity 1 within 24 hours, Severity 2 within 5 days). Track open incidents and aging. Report incident count, closure rate, and mean time to close. The board and regulators will ask "how many AI incidents have you had?" and "how did you handle them?" These metrics answer that. They also drive improvement. If incident count is rising, you may have more AI in production, better detection, or a real increase in failure. If closure is slow, your IR process needs attention. Don't hide incidents. Count them, close them, and show that you're learning from them.
Governance Maturity Score
A single number that summarizes where you stand can be useful for boards and for trend over time. Use a simple maturity model (e.g., the five-level model: ad hoc, aware, defined, managed, embedded). Score your organization on a set of dimensions (inventory, policy, classification, impact assessment, committee/ownership, incident response, integration with release/procurement, metrics). Average or weight the dimension scores to get an overall maturity score (e.g., 1 to 5). Update it annually or when you've made a material change. The score is an approximation. It's useful for "are we improving?" and for communicating upward. "We were at 2.2 last year; we're at 2.8 this year" is a story. Don't over-precision it. Half-point movement is meaningful. Use it as a trend and a conversation starter, not as a substitute for the underlying KPIs.
The Dashboard That Answers the Question
Pull these into one place. Inventory coverage (inventoried vs. estimated total, trend). High-risk systems with completed AIA (count and percentage). Shadow AI discoveries (count per period, inventory size). Mean time to review (by tier if possible). Incident count (open, closed, MTTR to close). Governance maturity score (annual or when updated). Add a short commentary: what's improving, what's not, what we're doing about it. Review the dashboard with the governance committee monthly or quarterly. Use it in board or risk committee materials so that "how do we govern AI?" is answered with the same kind of metrics you use for other risk and compliance domains. The dashboard doesn't have to be fancy. It has to be current, consistent, and used. When regulators or auditors ask for evidence of governance, the dashboard and the underlying data are the evidence. "We have a policy" becomes "here are the numbers." That's the shift.
Defining and tracking AI governance KPIs? We help with independent AI risk assessments and governance program design. Reach out.