Your Performance Review Measures the Wrong Thing Now: A Rubric for Grading Operators Who Manage Agents

Here is the uncomfortable part first: the review form you used last cycle was built to measure how much work a person did with their own two hands. Tickets closed. Lines shipped. Calls made. Campaigns launched. Every one of those numbers assumed the human was the unit of production.

That assumption broke the moment agents joined the team. The human is no longer the unit of production. The human is the unit of judgment — the person who sets the rules the agents execute against, and who catches it when the agents are confidently wrong. Grade that person on output volume and you will promote the loudest and fire the most valuable.

This is the rubric fix. Not a philosophy of the future of work — an actual rubric, with a scale, a re-plotted grid, and the pay math to back it.

The metric that used to predict performance now predicts nothing

Activity metrics were always a little soft: they signal participation, not impact. Mature engineering orgs had already moved toward outcome frameworks like SPACE and DORA over raw commit counts, precisely because counting keystrokes told you who was busy, not who was good.

Agents severed the last thread connecting activity to value. When one operator can point an agent at a backlog and generate 40 first-draft tickets before lunch, "tickets created" starts measuring how loose their standards are, not how productive they are. High output has become weak evidence — sometimes negative evidence.

The cleanest cautionary tale on record is Klarna, and it is worth telling straight because the myth around it is wrong.

Klarna reported its AI assistant handled 2.3 million conversations in its first month — which it framed as the "equivalent work of 700 full-time agents" — with resolution time dropping from 11 minutes to under 2, and repeat inquiries down 25%. The company projected roughly $40M in profit improvement for 2024 (Klarna, 2024). If you were grading on activity, this looked like a flawless season.

Then in 2025, CEO Sebastian Siemiatkowski publicly acknowledged they had cut too deep and that quality had dropped, and Klarna reopened human hiring for premium support (Bloomberg, 2025).

Read the two facts together. Activity scaled to 700-agent-equivalent throughput and the outcome degraded at the same time. And note the myth worth killing: the "700 agents" was never a clean headcount replacement — it was a modeled equivalence in Klarna's own marketing figures. Volume was the thing that looked like a win right up until the outcome told the truth.

If your review measures the Klarna-month-one number, you are measuring the exact quantity that misled Klarna's own executives.

What you actually manage now: agents, and the judgment to verify them

The gains are real when the operator is good. The controlled study of generative-AI assistance in customer support by Brynjolfsson, Li, and Raymond (NBER, 2023) found average productivity gains of about 14%, concentrated among less-experienced workers; other studies of coding assistants and marketing tasks report larger but noisier effects. Treat the headline percentages as directional, not settled — but the direction is clear: real value, unevenly captured.

And none of that value comes from the agent. It comes from the operator who knows what to ask for, how to check it, and when to throw it away. That is the competency your rubric has to name and grade. So name it.

The rubric: BARS for agent operators

Gut ratings are biased and noisy. A well-established fix is BARS — Behaviorally Anchored Rating Scales — where every point on the scale is tied to a specific, observable behavior instead of a vibe. BARS has a long research pedigree associating anchored scales with better reliability and less rater bias than unanchored ratings. Honest tradeoff up front: BARS are expensive to build. Each competency takes several hours of real work to anchor properly. That cost is exactly why the rubric reads as rigorous rather than free — you are buying discrimination between a level 2 and a level 3, which is the whole game.

Here is one competency, fully anchored, as the template for the rest.

Competency: Agent Orchestration & Output Verification

Level	Anchored behavior
1 — Autocomplete	Uses agents as a faster autocomplete. Ships raw agent output with no verification step. Cannot explain why the agent produced what it did, so cannot predict when it will be wrong.
2 — Delegator	Hands discrete, well-defined tasks to agents and spot-checks the results. Catches obvious errors. Still personally the bottleneck on anything novel or ambiguous — reverts to doing it by hand.
3 — Operator	Designs the workflow: writes the rubric the agent executes against, builds a verification loop into the process, and knows the agent's failure modes well enough to catch them before they ship. Recovers measurable hours and can show the before/after.
4 — Systems builder	Encodes their judgment into reusable rubrics and evals that other operators run agents through. Team-wide agent output quality rises because of scaffolding they built. Their leverage compounds across people, not just tasks.

Build three to five of these — Agent Orchestration, Output Verification, Judgment Under Ambiguity, and whatever is load-bearing for the discipline — and you have a review that measures the thing that actually creates value now. A level-4 operator producing "less" than a level-1 by activity count is your most promotable person. The rubric makes that visible. The activity dashboard hides it.

Re-plot the 9-box — you're finishing a fight, not starting one

Most large HR functions still run some version of the McKinsey 9-box: performance on one axis, "potential" on the other. Do not throw it out. The grid is fine. The Y-axis definition is the problem.

Its core weakness is well documented in HR practice: the 9-box shows where an employee is now but doesn't explain why, and routinely mislabels people whose context — not aptitude — drove the score. "Potential" has too often meant "reminds the calibration room of a future VP." That is the bias vector, and practitioners have complained about it for years — so you are not attacking a sacred cow.

The re-plot: redefine the potential axis as agent-leverage aptitude — the demonstrated ability to move up the BARS scale, to turn one person's judgment into a system others run. Someone can be a high performer today (crushing it by hand) and low on this redefined potential (a level-2 delegator who will plateau as agents absorb the delegable work). That box — high performance, low agent-leverage — is your quiet fragility risk, and the old grid rendered it invisibly as "solid contributor."

The pay math: recalibrate compa-ratios before the market does it for you

Here is where most orgs will get quietly robbed, so let's do the arithmetic.

Compa-ratio = actual salary ÷ pay-band midpoint × 100. A common convention is an 80%–120% band; 100% means paid exactly at midpoint. The mechanic people miss: when the market rate for a role moves and you refresh the band, the midpoint moves — so an employee's compa-ratio drops even though you never touched their salary.

Worked example. Take an operator at $95,000, in a band whose midpoint was $95,000 when it was last set. Compa-ratio: 95,000 ÷ 95,000 × 100 = 100%. Dead center. No flag.

Now the market for agent-fluent operators moves and you refresh the band to a new midpoint of $118,000. Same person, same salary, no raise:

95,000 ÷ 118,000 × 100 = 80.5%.

They just fell to the floor of the band. Nothing changed about them — the ground moved. If your comp review only looks at "did we give raises," you miss it entirely, and the way you find out is a resignation letter. Refresh the bands, re-run the compa-ratios, and fix the sub-90% level-3-and-4 operators before the market reprices them for you.

The ladder they climb is flattening — level honestly, don't cheerlead

The structure underneath all of this is flattening. Spans of control have widened and organizations have cut management layers — Gartner has predicted that through 2026, 20% of organizations will use AI to flatten their structure and eliminate more than half of current middle-management positions (Gartner, 2024). Charan's classic six-passage Leadership Pipeline is compressing. Our own studio bet is that in agent-heavy teams the director/manager/senior-IC rungs functionally collapse toward a single "lead operator" role — hold that as our projection, not an established statistic.

Now the honest counterpoint, because flatter is not automatically better: delayering without re-scaffolding just spreads the confusion thinner. When there are fewer rungs and a manager owns a far larger span, the BARS anchors become the alignment artifact — how you keep a big team of operators pointed the same way without a chain of middle managers translating.

And re-scaffold for retention while you do it, because the exits are expensive. SHRM puts the cost of replacing an employee at 50–200% of their annual salary depending on level (SHRM) — and losing a level-4 systems-builder is the loss that actually hurts.

Run it quarterly, because the annual review is already fading

None of this survives an annual cadence. When the skill being measured is compounding week over week, a once-a-year snapshot grades a person who no longer exists. The field has already been moving away from annual-only reviews toward continuous-feedback models for years, and research on frequent check-ins is generally associated with higher engagement and a stronger sense that the process is fair.

Prescription, stated as ours and not as industry standard: run the agent-competency rubric quarterly, and recalibrate compa-ratios against refreshed bands on a rolling basis rather than once at annual-review season. In a market moving this fast, an annual band refresh is a year of silent under-payment on your best operators.

The one-line version

Stop grading people on the work. The agents do the work now. Grade people on the rubric they hold the agents to — and measure that with anchored behaviors, a re-plotted grid, quarterly cadence, and pay bands that track the market instead of lagging it by a year.

---

If you're rebuilding your review, comp bands, and ladder for a team where agents do the producing and operators do the judging, that's the ground the People & Culture in the AI Era playbook covers — the full BARS competency set, the re-plotted 9-box calibration guide, and the compa-ratio recalibration worksheet, tradeoffs included. It won't reorg your company for you. It'll save you the hours per competency of figuring out where the anchors go.