Earlier version · Leading through technology shifts
The exception queue is part of the product
An AI workflow can save handling time and still make customers wait longer. One missing variable is the work arriving at the people who resolve its exceptions.
Before historical examples and clearer titles — preserved text. Read the current version.
Start with the promise made to the customer
An automated process rarely ends where the model stops generating. Someone still has to resolve the ambiguous document, correct the wrong identifier or decide what to do after a tool times out. If that person is unavailable, the customer's request remains unfinished even when the automated portion ran quickly.
I would treat the exception queue as part of the product. Its staffing, decision rights, priority rules and recovery paths determine whether automation returns useful capacity or moves delay into a less visible place. A system can improve its average processing time while leaving the hardest cases waiting longer.
This changes the question I would ask at a rollout review. How many requests can the whole service finish, including the cases that need a person? The model's throughput is one input. The capacity of the slowest necessary resolution step can determine what the customer experiences.
Google's SRE guidance describes how resource limits, queues and retries interact in software services. Human review has different behavior, incentives and constraints, so the analogy needs care. I use the same basic accounting question: what work arrives, what capacity can remove it, and what happens when the difference persists? Handling overload, cascading failures
Count demand in minutes before counting headcount
Consider a fictional service handling 1,000 cases a week. Eight hundred enter an AI-assisted workflow. Four reviewers each have 25 hours a week genuinely available for resolving exceptions, after other duties. The review pool therefore supplies 100 hours, or 6,000 minutes, a week.
At a 10% exception rate and 20 minutes per exception, the workflow creates 80 exceptions and 1,600 minutes of review demand. That uses about 26.7 hours. At a 35% exception rate and 25 minutes per exception, it creates 280 exceptions and 7,000 minutes of demand: about 116.7 hours.
The model can complete all 800 automated attempts in both scenarios. The second scenario still creates more review work than the team can finish. Calling this a temporary launch issue makes sense only if there is a credible reason the incoming demand will fall or the resolving capacity will rise.
Headcount alone hides important assumptions. A reviewer may have meetings, specialist duties and interruptions. A case requiring a particular legal or technical judgment may have only one eligible resolver. Four people in a roster do not imply four interchangeable servers. I would measure time available for the actual decision and the work each person is authorized to resolve.
| Constructed weekly scenario | Exceptions | Minutes each | Review hours needed | Review hours available |
|---|---|---|---|---|
| Lower review demand | 80 | 20 | 26.7 | 100 |
| Higher review demand | 280 | 25 | 116.7 | 100 |
A queue records a flow imbalance
With 280 exceptions arriving and capacity for 240 at 25 minutes each, 40 remain each week. Starting empty, the simple model reaches 200 unfinished cases after five weeks. That calculation assumes fixed arrivals, fixed handling time, no abandonment, no rework and no additional capacity. It exposes the consequence of the assumptions; it predicts no particular institution's backlog.
The relation is simple: next backlog equals current backlog plus arrivals minus completions, with a floor of zero. I would keep cancellations and transfers as separate departures. Otherwise the queue can look healthier because work was renamed, moved or abandoned while the customer still waits for an answer.
A falling backlog is also ambiguous. Did the team resolve more cases, narrow eligibility, let requests expire or stop recording some work? The operating review needs these flows alongside the stock. An improvement claim should identify which movement helped the customer and which merely changed the accounting boundary.
A stable average arrival rate is insufficient for a promise about waiting time. Bursts, variable handling times, absences and priority work can create long waits even when average demand fits average capacity. The five-week table deliberately leaves those effects out. I would use event-level simulation or a measured pilot before offering a service-level commitment.
| Week | New exceptions | Completed | End backlog |
|---|---|---|---|
| 1 | 280 | 240 | 40 |
| 2 | 280 | 240 | 80 |
| 3 | 280 | 240 | 120 |
| 4 | 280 | 240 | 160 |
| 5 | 280 | 240 | 200 |
Routine review consumes the same scarce attention
Suppose the same reviewers also spend two minutes checking each of the 800 AI-assisted cases. That adds 1,600 minutes, or 26.7 hours, before exceptions. Only about 73.3 of the pool's 100 hours remain for deeper resolution.
At 25 minutes per exception, the remaining capacity handles 176 exceptions a week. With 280 arriving, the backlog grows by 104 a week in the simplified model. The small routine check has changed the queue considerably. If routine checks belong to another team, move the demand to that team's capacity rather than removing it from the economics.
This is one reason I would record ordinary review separately from exceptional repair. A statement such as 65% needs no intervention can coexist with every case receiving a two-minute inspection. Both observations can be true, but the staffing implications differ. A cost model that counts only obvious failures misses the repeated touch across the rest of the work.
The same discipline applies to evaluation. Human annotation, periodic quality sampling and investigations consume effort even when the automated workflow behaves correctly. Some of that work is an investment that reduces future risk; some becomes ongoing operating cost. I would show the distinction so a launch budget does not quietly become an unfunded permanent obligation.
Saved labor can be stranded in the wrong place
Assume every case previously required 20 manual minutes. The baseline for 1,000 cases is about 333.3 hours. After 80% adoption, the 200 manual cases still require about 66.7 hours. Routine checks require 26.7 hours and exceptions require 116.7 hours. Total required human work becomes 210 hours. Each adopted case receives one routine check; exception repair is additional, and this illustration excludes other implementation, monitoring and operating effort.
On paper, demand falls by about 123.3 hours. Yet the review pool has only 100 hours for 143.3 hours of routine and exception work. The aggregate labor calculation improves while the specialist queue remains overloaded. The saved time and the required expertise may sit with different people.
I would describe the 123.3 hours as a potential reduction in labor demand under the assumptions. The service still has to finish its work. Unresolved cases prevent us from treating that difference as delivered capacity, and turning released capacity into cash savings needs a staffing or demand decision.
This is an organisational design problem as much as a model problem. Can general handlers learn the new resolution work? Can specialists delegate lower-consequence decisions? Does the tool provide the evidence reviewers need, or force them to reconstruct the entire case? Those interventions may improve completed throughput more than shaving another second from inference.
Rework can turn one exception into several visits
A case can reach a reviewer, return to the automated path and come back again. Counting queue entries as unique customer problems overstates how much new work arrived. Counting only the first entry understates the effort. I would track both the customer case and each visit.
In a simple model, each review has an independent probability r of needing another visit, with the same time required each time. Expected visits are 1 + r + r² and so on, giving 1/(1−r) when r is below one. At a 20% return probability, expected visits rise to 1.25 per case.
The weekly cohort of new exceptions therefore generates 8,750 expected eventual exception minutes before routine review, rather than 7,000: about 145.8 hours. Return visits can arrive in later weeks; this workload calculation makes no claim of a stable queue. Real rework can be correlated, and later visits can be harder than the first. The calculation exposes a neglected input while leaving the timing of those visits to a richer model.
Software retries create a related risk. Google SRE describes how delayed responses and retries can amplify overload. In an agent workflow, a retry can also repeat an uncertain action and create duplicate records. I would distinguish recovering a failed interaction from starting a new business case, then make that distinction visible in the work record. Cascading failures
A smaller visible queue can hide more escaped errors
One tempting response to review overload is to make the evaluator pass more work. That can reduce referrals while letting more unacceptable outputs reach customers. The review queue gets smaller and the unresolved problem moves downstream, where it may become harder to detect and repair.
I would examine three quantities together: unnecessary referrals, correctly caught failures and failures that escaped. Their consequences differ. A more selective reviewer can reduce false alarms and preserve safety; a less sensitive reviewer can also reduce the queue while damaging both quality and trust. The dashboard needs enough information to distinguish those changes.
This connects to the evaluator's own calibration. A percentage of cases sent to review describes routing behavior. It cannot establish the underlying failure rate. A team that improves the threshold should test the passed group and the referred group, keeping the sampling probabilities clear.
There is another failure mode at the human boundary. Reviewers under persistent pressure may shorten investigations or approve familiar-looking cases with less evidence. I would treat that as a hypothesis to examine through sampled decisions, handling times and escalation patterns. Queue pressure alone cannot prove that standards fell, but it gives a reason to look before declaring the rollout successful.
Put a capacity constraint into the adoption decision
Return to the simplified example with one visit per exception and no rework. Without routine checks, exception demand equals total volume × adoption × exception rate × minutes per exception. Dividing available minutes by the other terms gives a capacity ceiling for adoption. With 6,000 minutes, 1,000 cases, 35% exceptions and 25 minutes per exception, the ceiling is about 68.6%.
Add the two-minute routine check across all adopted cases and required minutes per adopted case become 2 + 0.35 × 25 = 10.75. The ceiling falls to about 55.8%. Eighty percent adoption exceeds that ceiling while the assumptions hold.
I would treat the ceiling as a warning boundary for this arithmetic, not an operating target. Running right at average capacity leaves no allowance for variability, incident work or absences. The right margin depends on the service promise and observed demand. There is no universal utilization percentage that makes a human review queue safe.
This creates a practical rollout choice. Restrict the eligible work, add trained resolution capacity, reduce actual exceptions or redesign the review step. Each option has a different cost. Reducing adoption can be the responsible short-term response while the team improves the constraint that limits completed service.
Choose the repair that matches the failure
More people help when demand exceeds trained capacity. They help less when every case waits for the same missing policy decision. A better model helps when incorrect outputs generate repair. It helps less when a correct result still needs a legally required review. The queue's categories should guide the intervention.
I would separate missing evidence, ambiguous policy, wrong extraction, broken integration, denied permission and uncertain completion. The resolver and fix differ across these categories. Treating them all as model errors invites prompt changes that cannot remove the underlying dependency.
Tool design can reduce investigation cost. A reviewer who sees the source passage, relevant version, intended change and actual result can make a focused decision. A reviewer who receives only a completion summary may need to repeat the work. The question is how much context is sufficient to resolve the exception accurately, without overwhelming the reviewer or exposing unnecessary information.
There is a cost to classification itself. For a small queue, a human triage pass may be enough. Automated clustering becomes useful when volume and repeated patterns justify it. I would validate the categories against actual resolution decisions, because a convincing label adds little if it sends the case to the wrong person.
Make priority and ownership explicit
A first-in-first-out queue is easy to explain and can be appropriate for similar cases. Urgency, customer harm and irreversible deadlines may justify other priorities. A high-priority lane, however, can starve ordinary work. I would track age and outcomes by class so a good aggregate turnaround cannot conceal a neglected group.
An exception also needs someone responsible for its next decision. Sending a case to a queue without a resolver, due condition or escalation path is a transfer of storage rather than responsibility. The owner should be able to distinguish work awaiting evidence from work ready for a decision and work that has no valid path forward.
The right interface depends on how often this occurs. A short internal workflow may need a named person and a reason. A larger service needs routing rules, capacity commitments and a way to return a malformed case to its producer. More coordination is justified when it reduces lost work, not by the number of agents or dashboards involved.
For customers, the useful promise is what happens when the ordinary path fails. Can they see that a case is pending? Will they be asked for evidence already supplied? Who can resolve a contradictory decision? I would design those answers alongside the successful path, because exceptions are part of the service customers actually use.
Test the whole flow before scaling it
The companion reproduces the demand, capacity, backlog and rework calculations from this essay. It also specifies scenarios for a burst of arrivals, one reviewer absent, longer handling times, duplicate referrals and an evaluator that silently misses failures. These are designed challenges for a future operating experiment.
In a pilot, I would record arrival and departure times, unique case identifiers, every visit, handling effort, waiting time, resolution outcome and escaped errors found later. Compare a similar manual baseline and preserve unfinished cases at the end of the window. A report limited to completed cases can look excellent while the queue accumulates the hardest work.
A credible alternative is to retain the manual workflow until automated routing and specialist capacity are credible. That choice can be rational when volume is low, integration is expensive or failures are hard to reverse. Another is to redeploy or train people whose ordinary handling work has fallen. Compare these choices with narrower adoption on completed service, quality and transition cost, rather than assuming the model must improve before the queue can improve.
My rollout rule would be to expand the work the whole system can finish well. Adoption volume then follows from model quality, reviewer capacity and clear decision rights. The exception queue belongs in the product review because it records the work the product has promised to do and has not yet completed.
Practical companion
Run the calculation script with Node.js to reproduce workload, backlog, rework and adoption ceilings, then inspect the authored operating challenges.
Sources and attribution
This is an original, self-published piece by Vihang Patel. The sources below support the referenced frameworks; fictional examples and personal judgments are identified in the text.