The AI paradox in software engineering
Code got cheap. Review became the system.
AI increased code output, but deliveries did not keep up. STRATAGO treats this bottleneck as a decision problem under uncertainty and models it with three complementary formalisms: Petri nets, simulation and influence diagrams.
Local output after agent adoption
The cost shifted to human judgment
No statistically significant increase
Figures from the Chen and Stratton study (≈300M events, 700+ companies, 2021–Mar 2026), as reported in the source document. Source
01 · Does it make sense?
Yes, because each formalism answers a different question.
No single model captures the bottleneck. Flow structure, performance over time and policy choice are distinct problems, and mixing them produces confident wrong answers.
| Formalism | Question it answers | Output for the decision | Limit |
|---|---|---|---|
| Petri net | Where does work get stuck? Is there concurrency, rework or deadlock? | Verifiable flow structure and its invariants | On its own, it does not say how long the wait is |
| Simulation (GSPN / queues) | How long does a PR wait and how many deliveries ship per day? | Lead time, utilization and throughput per scenario | Depends on parameters calibrated with real telemetry |
| Influence diagram | Which review policy maximizes value given what we do not know? | Optimal policy and value of information | Requires an explicit, debated value function |
| AI Decision Matrix (MIT CISR) | Who frames, who acts and who learns on each PR? | Review track and owners per activity | Static, unless computed per PR |
| STRATAGO epistemic fusion | The human reviewer and the AI reviewer disagree. Whom to trust? | Evidence state per PR (DS + LPA₂v) | Belief masses need calibration |
Condition for it to make sense
The model is only worth something if it is fed with the team's own data (commit, review, deploy and incident timestamps). Without calibration it is an elegant illustration. With calibration it becomes an auditable decision instrument.
02 · Concepts
Seven ideas to read the rest of the page.
Each concept links to its primary source at the end of the page.
Petri net
A graph of places (circles, states), transitions (bars, events) and tokens (work items). A transition fires when every input place holds tokens. It models concurrency and shared resources. Classic reference: Murata (1989).
GSPN
Generalized stochastic Petri net: it separates immediate transitions (zero time) from timed ones (random time). It is equivalent to a Markov chain and can be solved or simulated. Ajmone Marsan, Balbo and Conte (1984).
Little's Law
L = λW. Items waiting = arrival rate × time in system. If PRs arrive faster and capacity does not change, the stock of idle code grows. Little (1961).
Utilization ρ
ρ = λ / (c·μ): the fraction of time reviewers are busy. Waiting grows non-linearly as ρ approaches 1. That is why small volume increases cause large delays.
Influence diagram
A graph of decisions (rectangles), uncertainties (ovals) and value (hexagon). It shows what depends on what and what is known at decision time. Readable by people and solvable by computers. Howard and Matheson.
DORA
Four delivery metrics: two for throughput (deployment frequency, lead time) and two for stability (change failure rate, recovery time). The review bottleneck shows up in all four, not just the first two.
Complementarity
The NBER working paper by Demirer, Musolff and Yang estimates an elasticity of substitution of 0.25 between AI-generated code and review effort. Execution and judgment are complements: more of one does not replace the other.
Dempster-Shafer and LPA₂v
Dempster-Shafer combines evidence from several sources and measures the conflict between them (K). The paraconsistent logic LPA₂v classifies each proposition by degree of belief and disbelief, without collapsing contradiction into an average. A PR with high belief and high disbelief at the same time is not "50% approved": it is inconsistent and calls for human judgment.
03 · Structural layer
The Petri net of the review flow
Each PR is a token. Reviewers are tokens too, in a resource place. Review only fires when there is a PR in the queue and a free reviewer: that is where the bottleneck becomes visible in the structure.
R · free reviewers (c tokens). T2 takes one token from R when it starts and returns it when it ends. It is the resource that limits throughput.
T3′ · request changes (prob. r). It sends the token from P3 back to P2. It is the rework loop that multiplies the load: each PR visits review 1/(1−r) times.
Boundedness
Is P2 bounded? If λ ≥ c·μ·(1−r), the queue grows without end. Reachability analysis shows this before it happens.
Resource invariant
Free reviewers + busy reviewers = c, always. A P-invariant that checks whether the model represents the real team.
Liveness
Does any PR stay stuck forever? Policies such as "two mandatory approvers" can create deadlocks the net reveals.
04 · Performance layer
Review bottleneck simulator
With exponential times, the net above reduces to an M/M/c queue with rework. Adjust the parameters and watch the wait explode near ρ = 1.
Fixed assumptions: each reviewer spends 4 h/day on review; 8 h working day. Lead time includes rework passes.
Four policy scenarios
Same team (5 reviewers, 4 h/day). Click to load the scenario into the simulator. Illustrative parameters, chosen to reproduce +23% PRs; results were checked with discrete-event simulation.
| Scenario | Assumption | ρ | Lead time/PR | Failure-free deliveries/day | Incidents/day |
|---|---|---|---|---|---|
| A · Pre-AI | 10 PRs/day, 1 h/pass, r = 15%, failure 12% | 0,59 | 0,33 d | 8,8 | 1,2 |
| B · AI + instance review | 12.3 PRs/day, larger PRs (1.15 h), r = 18%, failure 15% | 0,86 | 0,70 d | 10,5 | 1,8 |
| C · AI + AI triage | Faster but shallower review (0.8 h): failure 17% | 0,60 | 0,27 d | 10,2 | 2,1 |
| D · AI + rule review | Contracts and property tests: 0.85 h, r = 8%, failure 10% | 0,57 | 0,25 d | 11,1 | 1,2 |
Reading: B doubles lead time with the same team. C recovers speed but exports the problem to production (more incidents). Only D improves throughput and stability together, which reproduces in numbers the thesis "review the rule, not the instance". Button E adds routing by the decision matrix: 45% of PRs go to automated tracks and ρ drops to about 0.31.
05 · Decision rights
The AI Decision Matrix applied to review
The MIT CISR matrix places each decision by ambiguity (how clearly data determines the answer) and risk (the cost of being wrong), and defines who frames, who acts and who learns. Applied to PRs, it says which ones leave the human queue and where judgment is irreplaceable.
Exploratory
PRs: prototypes, features behind a disabled flag, spikes, internal tools.
Approach: experiment cheaply; the human extracts the learning.
Strategic
PRs: architecture changes, public API contracts, pricing or billing logic.
Approach: deep, prioritized human review; AI supports execution.
Routine
PRs: dependency bumps with green tests, lint, formatting, documentation.
Approach: auto-merge with sampled monitoring by the process owner.
Consequential
PRs: payment webhooks, authentication, data migrations, infrastructure routing.
Approach: review the rule. Contracts, policy-as-code and property tests; the human reviews guardrails and postmortems.
Reading: line-by-line human review is only justified in the Strategic quadrant. In the other three, the human acts before (framing) or after (learning), not as an inspector of every instance.
Does it make sense? What the matrix gets right and where it fails
It decomposes the decision
Framing, acting and learning have different owners. It is the separation the Bauhaus tried to bring together, now redistributed on purpose.
Human accountability
Humans answer for outcomes in every quadrant, and the company makes explicit which risks it accepts and who bears them.
Policy, not tool
The matrix defines decision rights by type of problem, not which AI to use.
| Problem | Why it matters | Fix |
|---|---|---|
| Circulating translation: Routine with "Learn" assigned to AI | Contradicts the original, in which a human process owner monitors patterns | Learn = human in every quadrant |
| Circulating translation: Consequential with "Frame" assigned to AI | In the original, humans set the guardrails, and setting guardrails is framing | Frame = human |
| Ambiguity mixes two phenomena | Missing evidence and conflicting evidence call for opposite responses: collect data or arbitrate | Separate ignorance from conflict (DS + LPA₂v) |
| Risk without reversibility | A large error reversible in minutes is different from a large irreversible error | Risk = P(failure) × impact × (1 − reversibility) |
| Static classification | Decisions migrate: the exploratory becomes routine as learning happens | Recompute the quadrant per PR, continuously |
Making the axes computable
Ambiguity = LPA₂v state
The PR's evidence is fused by Dempster-Shafer. Belief μ and disbelief λ, computed independently, give the state:
True / False: low ambiguity, lower quadrant.
Paracomplete (low μ and λ): high by ignorance. Collect evidence.
Inconsistent (high μ and λ): high by conflict. Human arbitration.
Risk = value node
P(failure | evidence) × impact × (1 − reversibility)
Impact: whether the PR touches critical paths (payments, authentication, data) and how many users it affects.
Reversibility: feature flag, automatic rollback, destructive migration or not.
| Quadrante | Value of information | Cost of error | Rational strategy |
|---|---|---|---|
| Exploratory | High | Low | Buy information cheaply: experiment |
| Strategic | High | High | Buy information at a high price: deliberate first |
| Routine | Low | Low | Not worth measuring more: automate |
| Consequential | Low | High | Shield execution: guardrails |
In practice, the matrix is a heuristic of the influence diagram below: STRATAGO computes what the matrix only classifies.
Effect on the bottleneck
In the Petri net, the matrix becomes a router after triage. Only the Strategic and Consequential tracks consume free reviewer tokens.
Scenario B with 45% of PRs routed to automated tracks: ρ 0.86 → 0.47. Combined with rule review (scenario E): ρ ≈ 0.31.
How to roll it out
- Inventory PR types and place them on the matrix by category.
- Name owners per activity; "learn" is always a human name.
- Instrument the axes per PR: DS + LPA₂v and blast radius × reversibility.
- Route each PR to its quadrant's track, with an auditable record.
- Measure the four DORA metrics per track; high failure on Routine = loose threshold.
- Review migration between quadrants monthly.
06 · Decision layer
The influence diagram of the review policy
The matrix says who decides. The influence diagram shows what the decision depends on: evidence becomes the matrix axes, the axes choose the track, and the track determines wait, defects and value.
AI reviewer · tests · static · history
and reversibility
ignorance · conflict
(DS + LPA₂v)
size
P(failure) × impact
× (1 − reversib.)
Routine
Consequential
Exploratory
Strategic
sets who frames, acts and learns
in the human review queue
generated by agents
that escape
failure-free deliveries
− cost of waiting
− cost of incidents
| Arc | Meaning |
|---|---|
| Evidence → Ambiguity | Dempster-Shafer fuses the sources; LPA₂v separates ignorance from conflict. |
| Blast radius → Risk | Impact weighted by how irreversible the error is. |
| D2 → Risk | Smaller batches reduce the blast radius. It is the lever DORA points to for improving throughput and stability. |
| Ambiguity, Risk ⇢ D1 | Information arcs: known at decision time, they place the PR in a matrix quadrant. |
| D2 ⇢ D1 | The batch is decided before the review track. |
| D1 → W, f | The track defines who reviews and how deeply: it changes the wait and the defects that escape. |
| λ → W | More generated volume, more queue. |
| W, f → V | Wait and incidents are the costs; failure-free deliveries, the gain. |
| V ⇢ Evidence | The matrix's "Learn" activity, always human: the outcome recalibrates belief masses and thresholds (V_ALTO, V_BAIXO). If measuring the evidence first costs less than the expected error, the diagram signals the value of information. |
07 · STRATAGO epistemic layer
When the AI reviewer and the human disagree
AI triage is only safe if disagreement is treated as information, not noise. Each PR receives evidence from different sources, which STRATAGO fuses and classifies.
Dempster-Shafer fusion
Sources: AI reviewer, test suite, static analysis, author history. Each assigns mass to {approvable, not approvable, uncertain}. Dempster's rule combines the masses and returns the conflict K between sources.
LPA₂v classification
Belief μ and disbelief λ are computed independently (λ = Bel(not approvable), never 1 − μ). The pair (μ, λ) falls into one of the logical states, and each state has a route.
| LPA₂v state | Reading | Route in the flow |
|---|---|---|
| True (high μ, low λ) | Evidence converges on approvable | Light human review, by sampling |
| False (low μ, high λ) | Evidence converges on not approvable | Back to the author or agent, at no human cost |
| Inconsistent (high μ, high λ) | Sources in real conflict | Prioritized human review: this is where judgment lives |
| Paracomplete (low μ, low λ) | Missing evidence | Ask for more tests or specification before reviewing |
Result: human attention, the scarce resource of the Petri net, is allocated where evidence is contradictory. The cut-off thresholds (V_ALTO, V_BAIXO) are parameters to calibrate with the team's history.
08 · Implementation
From data to dashboard in five steps
Each step reuses engines STRATAGO already has: Petri, Dempster-Shafer, LPA₂v and sensitivity analysis.
Telemetry
Collect commit, open, review, approval, deploy and incident timestamps (Git, CI, incident management).
Calibration
Estimate λ, time per pass, effective c, r and failure rate per team and period, with confidence intervals.
GSPN engine
A FastAPI service that builds the net, checks boundedness and invariants and simulates scenarios by discrete events.
Decision
Evaluate the influence diagram over the scenarios, with Sobol analysis to show which parameter dominates value V.
Routing
DS + LPA₂v fusion per PR, deciding who reviews and how deeply.
Validation
Compare predictions with the four DORA metrics for the following period. A model that does not predict gets recalibrated.
09 · Algorithmic Bauhaus
Review the rule, not the instance.
The Bauhaus answered the assembly line by bringing conception and execution back into the same workshop. The equivalent today is to move human judgment upstream: specify the constraints that make code reviewable instead of inspecting every output. Scenario D in the simulator is that thesis in numbers.
10 · Caveats
What this model does not say
Times are not exponential
Real reviews have long tails. The GSPN admits other distributions via simulation; the closed formula is only a first approximation.
Reviewers are not identical
Specialists form their own queues. A multiclass model by code area is the next level of detail.
Value is not just throughput
Learning and knowledge transfer happen in review. The V function must include them, or it optimizes what is easy to measure.
Live · review.stratago.com.br/api
The engine in production
The same models on this page run in the STRATAGO Review Engine service, which receives GitHub telemetry, routes each PR and measures the four DORA metrics per track.
| Track | PRs | Deploys/working day | Median lead time | Failure rate | Median recovery |
|---|---|---|---|---|---|
| Loading… | |||||
11 · Sources
References
- Murata, T. (1989). Petri Nets: Properties, Analysis and Applications. Proceedings of the IEEE, 77(4), 541–580. Bibliographic record
- Ajmone Marsan, M.; Balbo, G.; Conte, G. (1984). A Class of Generalized Stochastic Petri Nets for the Performance Evaluation of Multiprocessor Systems. ACM Transactions on Computer Systems, 2(2), 93–122. Abstract
- Little, J. D. C. (1961). A Proof for the Queuing Formula: L = λW. Operations Research, 9(3), 383–387. DOI 10.1287/opre.9.3.383
- Little, J. D. C. (2011). Little's Law as Viewed on Its 50th Anniversary. Operations Research, 59(3), 536–549. INFORMS
- Howard, R. A.; Matheson, J. E. (2005). Influence Diagrams. Decision Analysis, 2(3), 127–143. RePEc
- Howard, R. A.; Matheson, J. E. (2005). Influence Diagram Retrospective. Decision Analysis, 2(3), 144–147. RePEc
- Sebastian, I. M.; Weill, P.; Haskamp, T.; vom Brocke, J. (2026). Designing Decision Rights for AI. MIT CISR Research Briefing XXVI-6. MIT CISR
- DORA. Software delivery metrics: the four keys. dora.dev
- Chen and Stratton: study on AI agents and the review bottleneck (2021–2026). Coverage
- Demirer, M.; Musolff, L.; Yang, L. (2026). NBER working paper on AI coding tools. Summary
Parameters for scenarios A–D are illustrative, not measured. STRATAGO · Decision Intelligence · Florianópolis