AI usage data should start a conversation about the work

An AI usage dashboard can tell a leader where the activity is. Deciding what that activity is worth still requires someone who understands the work. I would use the dashboard to choose an investigation, then put the business owner in charge of explaining what changed.

OpenAI’s September 16 guidance describes analytics that combine usage and spending information with task categories across ChatGPT Work and Codex. Its engineering measures include contributions to merged code. The guidance also tells business owners to compare workflow results, including review and correction time. Its worked ROI calculation is explicitly hypothetical, rather than a measured customer result.

That gives a learning leader somewhere concrete to start. Suppose a support team uses AI heavily to draft replies. High usage could mean the tool is useful. It could also mean employees are repeatedly revising answers that are difficult to trust. The same activity count could accompany either situation. A discussion with the team is more useful than assigning training because its number is above or below another department’s.

I would select one recurring task and inspect a small sample of completed work. Keep the standard simple enough that a manager can use it: whether the reply is accurate, resolves the issue, and requires further work. Include what happened after the first draft. If the employee finishes sooner but a specialist spends longer correcting it, the work has moved rather than disappeared.

The capability question follows from that inspection. An employee who cannot recognize a weak answer needs different support from one who knows what is wrong but lacks access to the relevant information. A course could help the first person. For the second, a permissions or knowledge-maintenance decision may matter more. Usage data can identify where to look; it cannot make that diagnosis by itself.

Time released also needs an owner. If a team gains capacity, its manager should decide whether to reduce a backlog, improve service, or take on additional work. Finance can then assess the consequence using the evidence appropriate to that choice. Multiplying every reported hour saved by salary cost can produce an attractive estimate while leaving the actual operating benefit unexplained.

The vendor’s guidance is useful without proving that these analytics cause better investment decisions. Start with a manageable use case, record what is known, and revise the decision as results emerge. Learning leaders contribute by making the required judgement teachable and observable. The dashboard becomes valuable when it helps someone make a better decision about the work.

Correction, October 8: An earlier version attributed MetricHQ’s definition of incremental business value to OpenAI. This version describes OpenAI’s actual analytics guidance and removes that attribution.