AI Status Reports from Azure DevOps That Leaders Trust

It's the first Monday of the month and leadership wants to know what your teams delivered. You open Azure Boards, export three queries to Excel, and start translating titles like "fix stuff on login" and "spike part 2" into sentences a director understands. By Wednesday you have a report. By Thursday someone points out that half the "completed" work never shipped.

The obvious fix is to let AI write the report. That works, but not the way most people try it. Paste a raw query export into a chatbot and you get a confident, well-formatted summary of outcomes that never happened.

In this post I show the pipeline I recommend for an Azure DevOps status report that an AI drafts and a human can trust: one scoped query, a minimum standard of evidence on every completed work item, a grounded prompt, and a short review step. The AI part turns out to be the easy part. The data is the report.

Why status reporting still eats days in Azure DevOps

Dashboards show numbers, leaders want a story

Azure DevOps has solid reporting mechanics. You can build dashboards with charts and widgets for velocity, burndown, cumulative flow and lead time. Those are useful for the team. They don't answer the question a leadership review actually asks: what changed for customers or the business this month, and which priorities did it move?

That answer lives in the text of your work items, not in a chart. So somebody has to read the items, group them, and write the story. Usually that somebody is the engineering lead, and usually it takes a day or more.

What Microsoft learned from its own reporting agent

Microsoft recently described how one of its internal engineering teams automated this exact job. In the Inside Track write-up, the team runs a reporting agent that reviews last month's completed work in Azure DevOps, groups it under strategic priorities and drafts leadership-ready summaries. A report that used to take weeks is now ready at the start of the next month.

The interesting part is what made it work. Early outputs were sometimes inaccurate or incomplete. The team fixed that by strengthening the underlying planning data and adding a second agent that checks work items against data hygiene rules. As one of them put it: "the AI can only be as accurate as the data it has in front of it." Humans still review the output.

That matches everything I've seen building AI tooling on top of Azure Boards.

Why AI-generated status reports go wrong

Garbage titles in, confident nonsense out

A language model will always produce fluent text. If the input is a work item titled "Update API" with an empty description, the model fills the gap with something plausible. "Improved API performance and reliability" sounds great in a report and may be completely untrue. The model isn't lying on purpose; you gave it nothing to be accurate about.

"Closed" is not evidence of delivery

Teams close items for many reasons: done, duplicate, won't fix, moved to another backlog, or "closing to clean up the board". A query on State = Closed mixes all of these. Unless the item records what was actually delivered and how it was verified, the AI can't tell a shipped feature from a bulk cleanup.

Work that isn't linked to a priority disappears

Leadership reports are organized by priority: epics, objectives, initiatives. Work items without a parent link have no place in that structure. The AI either drops them or guesses where they belong. Both are bad, and the second is worse because it looks right.

Step 1: Scope the report with one saved query

Start with a single shared query that defines the report's scope. Everyone, human or agent, works from the same list.

Filters: closed date, state, area path

For a monthly report, filter on items that reached a completed state last month. The date macros in Azure Boards queries make this repeatable without editing dates every month:

• Work Item Type In User Story, Bug (add Feature if you report at that level)
• State In your completed states, for example Closed or Done
• Closed Date >= @StartOfMonth - 1 and Closed Date < @StartOfMonth
• Area Path Under the teams in scope
• Reason excludes duplicates and "removed" if your process tracks it

Save it as a shared query such as Reporting / Delivered last month. Now the scope is visible to everyone, reviewable, and identical for every run.

Group by epic or feature, not by team

Use a tree query or add the parent as a column so each item carries its epic or feature. Leaders think in outcomes, not team names. Grouping by parent gives the AI the structure of the report up front, so it doesn't have to invent one.

Step 2: Make every completed item self-explaining

This is where most of the accuracy comes from. You don't need perfect backlog hygiene across the board. You need a minimum standard of evidence on items as they reach Done, because those are the only items the report reads. If you want the broader backlog clean-up, I covered that in Azure Boards hygiene for 2026.

Minimum evidence per item

For each completed User Story or Bug, I'd require:

• An outcome title. "Customers can reset their password from the app" instead of "Password reset".
• Acceptance criteria that describe what is now true, so the AI can quote real behaviour.
• A parent link to a feature or epic, so the item lands under the right priority.
• A short delivery note: where it shipped (release, environment, feature flag) or why it was closed without shipping.

That's four things. None of them are new to a decent team. What is new is that a machine now depends on them every month.

Enforce it at Done with a checklist, not a reminder

Reminders in the retro don't hold. What holds is a gate at the moment an item moves to Done. A Definition of Done checklist on the work item, with the evidence items above, makes the requirement visible while the developer still remembers the details.

In the Checklist Extension for Azure DevOps you can attach a reusable template to User Stories and Bugs and block the state transition until the checklist is complete. I wrote about that pattern in more depth in Make "Done" Mean Done in Azure DevOps Boards. For reporting, the side effect matters: every item the query returns has passed the same evidence gate, and the checklist history shows who confirmed what.

Step 3: Let AI draft, then review like an editor

A prompt structure that stays grounded

The prompt matters less than the data, but a good structure prevents the most common failures. The rule is simple: the model may only summarize what is in the items, and it has to say when evidence is missing.

1You are drafting a monthly delivery report for engineering leadership.
2Input: the work items below, grouped by parent epic. Each item has an id,
3title, acceptance criteria and delivery note.
4
5Rules:
6- Only describe outcomes that are stated in the items. Do not infer impact.
7- Cite work item ids in brackets after every claim, for example [#4812].
8- Per epic: 2-4 sentences on what is now possible for users or the business.
9- List items with missing acceptance criteria or delivery notes under
10  "Needs clarification" instead of summarizing them.
11- End with a short "Not delivered as planned" section if any item was
12  closed without shipping.
Copy

The ids in brackets are the most useful line. They make every sentence checkable in seconds, and they make hallucinations obvious because the cited item won't say what the sentence claims.

Where the agent lives: MCP, Copilot, or inside the work item

You have three realistic options in 2026:

• An agent over the Azure DevOps MCP Server. GitHub Copilot or another MCP client runs your query and reads the items. It's flexible and works well for a monthly batch. I covered the setup, including read-only mode, in Azure DevOps MCP Server: Connect AI Agents Safely.
• Export and paste. It works for a one-off report, but you lose links back to the items and nothing is repeatable.
• AI inside Azure Boards. ClearSpecs AI is the extension I build for this. Its agentic chat queries live work items from within Azure DevOps, so you can ask it to summarize what was delivered under an epic, and drill into a single item when a sentence looks off. It also helps on the input side: it rewrites vague titles and drafts acceptance criteria while the work is still fresh.

Whichever you pick, keep the agent read-only for reporting. A report generator has no reason to edit work items.

The human review checklist

The draft is a draft. Before it goes to leadership, someone who knows the work spends 20 to 30 minutes on it:

1. Spot-check three cited ids per epic. Does the item say what the sentence says?
2. Read the "Needs clarification" list and fix the items, not the report.
3. Check the "Not delivered as planned" section for anything sensitive that needs context.
4. Add the one thing AI can't know: why this mattered this month.

Fixing the items instead of the text is the habit that compounds. Next month the same gaps don't come back.

A monthly reporting routine that takes an hour

Put together, the routine looks like this:

• During the month: items pass the Done checklist with an outcome title, acceptance criteria, parent link and delivery note.
• First working day: run the shared Delivered last month query and hand it to your agent with the grounded prompt.
• Same morning: review the draft with the checklist above, fix flagged work items, regenerate once.
• Send: the report goes out with item ids intact, so any reader can click through to the evidence.

The first month takes longer because the gaps surface all at once. From the second month, the evidence gate does most of the work and the review shrinks to a coffee.

Conclusion

AI can turn an Azure DevOps status report from a two-day chore into an hour of editing, but only if the work items carry real evidence. The pattern is the same one Microsoft landed on internally: scope the data with one query, raise the quality of completed items at the source, let the model draft with ids attached, and keep a human in the review seat.

If you want to try the drafting part inside Azure Boards, install ClearSpecs AI from the Visual Studio Marketplace and ask its chat to summarize last month's delivered work under one epic. It has a free tier, so you can see quickly whether your items are ready for it. If the answer is "not yet", add a Done checklist first. The report will get better on its own.

Frequently asked questions