Compare delivery with and without AI

Use this guide to see whether AI is changing how your teams deliver. You can check how much of what they merge is AI-assisted or written by agents, what AI costs per pull request, and whether flow and cost per pull request moved over the same months.

Before you start

• Engineering insights is switched on with the AI impact tab, and GitHub is connected. See Measure what delivery returns.
• Your engineers' AI sessions reach Flowstate. Without them, only agent-written pull requests count as AI. See Track AI in your workflows.
• You need access to effort reporting to open Engineering insights, and access to financial figures to see cost. Ask your Flowstate admin.

Compare with the previous period

1. Go to Insights → Engineering insights → AI impact.
2. Choose the dates with the period control: Quarter, Year, Since or Custom.
3. To narrow it, select Filter and choose Team, Repository or Initiative.
4. Read the figures across the top: AI-assisted PRs %, AI-assisted PRs, Agent-authored %, AI cost / merged PR and Total AI cost.

Each figure shows its change against the period of the same length just before the one you chose. Choose a quarter, for example, and each figure is compared with the same number of days immediately before that quarter began.

The Overview and Flow tabs don't compare periods. Read their charts month by month instead (see below).

Read the months either side of a change

To see what happened around a change, such as rolling out an AI tool:

1. Choose a Custom period that starts well before the change and ends well after it.
2. On AI impact, read Delivery mix over time. It shows the split of Agent-authored, AI-assisted and Human pull requests for each month they merged.
3. On Flow, read Cycle time (open → merge) for the same months.
4. On Overview, read Cost per merged PR and PR throughput for the same months.

The period and filters you choose apply to every tab, so the charts line up month for month.

Compare teams

Each tab can be filtered to several teams at once, but then it shows them combined. To compare teams, save a view for each one:

1. On AI impact, select Filter → Team and tick one team.
2. Open the views menu and select Save current view. Name the view after the team, then select Save.
3. Repeat for each team you want to compare.

Switch between the views under Saved views to read one team after another. See Save a view.

Find AI-assisted and agent pull requests

On Insights → Engineering insights → Pull requests, rows are marked Autonomous agent or AI-assisted (human). Human pull requests carry no mark. For an agent's pull request, the author column shows the agent, with the person who ran it beneath.

Select a row to see its Authorship, Line authorship and Linked sessions.

You can't filter the list to AI-assisted or agent pull requests. Use AI impact or the report below for totals.

Compare cost per pull request by delivery kind

The AI Delivery — Cost vs Human Baseline report template is a table for each team. For agent, AI-assisted and human pull requests, it shows Merged PRs, Cost per PR and AI Cost.

1. Go to Insights → Analytics & reporting → Dashboards and open a dashboard you made, or create one.
2. Open the gear menu (Actions) and select Edit Layout.
3. Select Add Report, then Templates.
4. Find AI Delivery — Cost vs Human Baseline. Select Preview, then Add to Dashboard, or drag it onto the dashboard.
5. Select Stop Editing.
6. Choose the dates with the dashboard's period control. The table covers the dates you choose.

Rows under Delivery Kind read agent, ai_assisted_human and human. To see how the table changed, set the dashboard's dates to one period, read it, then set them to another.

The report shows cost, so you need access to financial figures. It becomes one of your own reports, so you can change it like any other.

Compare like with like

A lower cost per pull request for AI-assisted or agent work only means something if the pull requests are similar. An agent's dependency updates aren't the same work as a person's new feature. Read Cost per PR alongside PR size distribution on Flow, and alongside what the project delivered.

Compare before and after an AI rollout

Not available yet

There's no setting to mark the date you rolled out an AI tool, and no screen that splits delivery into before and after that date. Instead:

• For AI impact's figures: choose a Custom period that starts on the rollout date. Make it as long as the stretch before rollout you want to compare against. Each figure's change is then measured against that stretch.
• For flow and cost: choose a Custom period covering both sides of the rollout and read the monthly charts, as in Read the months either side of a change.
• For a table: set the AI Delivery — Cost vs Human Baseline dashboard to the months before, read it, then set it to the months after.

Read the result carefully

• More pull requests isn't more value on its own. It can mean smaller pull requests or a growing backlog. Read throughput with Cycle time (open → merge), PR size distribution and the project's verdicts.
• Look at quality next to speed. If cycle time falls while Median post-open churn % rises, more work is carrying on after pull requests open.
• Faster delivery is capacity, not a saving. It only becomes money when headcount comes down or the team takes on more work.
• Outside engineering, use outcomes instead of pull requests. For a support team's AI assistant, read the project's drivers, such as deflection, handle time and satisfaction, on its Value tab. Read what a resolved contact costs with and without a person in Business metrics.

If something's not right

AI-assisted PRs % is zero. Your engineers' AI sessions aren't reaching Flowstate. See Cloud Proxy. Agent-written pull requests still count.

The Flow or AI impact tab shows a sample with Talk to us to turn on Engineering insights. Those tabs aren't switched on. Ask your Flowstate contact.

The report tile says "Financial data restricted". It shows cost, and you don't have access to financial figures. Ask your Flowstate admin.

A team isn't in the Team filter. It has no pull requests in the dates you chose. Widen the period.