The Human-in-the-Loop Fallacy.
The Human-in-the-Loop Fallacy is the belief that the continued presence of humans in an AI-mediated workflow proves that humans remain central to value creation. It mistakes existence for position.
A human may remain in the loop. The question is where that human sits in relation to the algorithmic system. Does the human define the problem, own the customer, shape the workflow, and absorb accountability? Or does the human review, correct, validate, and patch the machine? The first position is above the algorithm. The second is below it.
What does everyone get wrong about human-in-the-loop?
The phrase has institutional weight. AI safety guidance treats meaningful human oversight as a horizontal requirement. Regulation writes it into law for high-risk systems. CEO talking points and consulting decks borrow the same shape and apply it to labor markets directly.
Everyone agrees: humans stay in the loop, so jobs are safe.
Almost nobody asks: a loop terminating where?
- Everyone says
“AI augments. Humans stay involved.”
NIST AI RMF (2023) and EU AI Act Article 14 (2024) write meaningful human oversight into law. McKinsey State of AI 2024 names augmentation as the dominant enterprise frame.
But the loop terminatesInvolvement is not ownership. The translation post-editor is involved with the AI assistant. The Klarna gig CS agent at $41/hr was involved with the OpenAI assistant. Both are below the algorithm. Augmentation says nothing about where the output goes.
- Everyone says
“Humans handle what AI can't.”
Salesforce Agentforce launch (Sep 2024). CEO talking points across the Magnificent Seven. The consulting-deck residual-theory of work.
But the loop terminatesStarting with 'what AI can't' produces compensating-layer jobs - validation, exception handling, relationship management, ethics review. The map of those jobs is mostly below the algorithm.
- Everyone says
“Humans validate AI output before it ships.”
Anthropic RLHF era discourse, AI safety guidance, compliance-review job listings.
But the loop terminatesDell'Acqua-Mollick 2023: GPT-4 users produced more polished output on every BCG task - including the tasks where the model was systematically wrong. Surface validation collapses as buyers learn this. Most validators do surface.
Where does the output go after the human touches it?
Same human. Same workflow. Two destinations. The destination is the position.
The presence of the human tells you nothing. The destination tells you everything. Everyone gets the question wrong.
Where does the human actually sit?
Everyone gets augmented. Position decides outcome.
The premise of this map is that everyone in knowledge work is now augmented by AI. The real question is what AI does to your seat. Is AI inside the offering you sell - the thing the customer pays for? Or is AI inside the workflow you operate - the tool you use to do work that someone else owns? And does the position route the output to a client, or back into someone else's pipeline?
AI as leverage
Commoditized labor
AI as discipline· The HitL trap
Hover any role chip for why it lands there. Click a quadrant for the practitioners and the durability test.
Craft
AI is in your daily workflow as a tool, but the offering you sell is your human act, signature, or institutional seat. The customer pays for the act (the surgery, the courtroom appearance, the sworn attestation, the CPA seal) - AI just helps you prep. The institutional bind keeps you above the algorithm.
Holds because the regulator refuses to let AI hold the seat. The FDA's AI/ML device approval pathway still routes the final read through a radiologist. The PCAOB still binds audit signoff to a CPA. Courts still bind immigration translation to a sworn human. The seat erodes the day the regulator publishes a framework allowing AI to sign directly - watch the FDA AI/ML pathway, FAA autonomous-flight certification, and bar association rulings on AI legal practice.
- Trial litigators Courtroom is non-delegable; AI accountability docket adds adversarial work
- Surgeons Procedure is the deliverable; AI has not entered the OR as actor
- Sworn / court translators Court, asylum, immigration translation requiring sworn signoff; substance liability binds to the practitioner
- CPAs (audit signoff) Big Four 'Client Zero' enterprise sell-through requires CPA seal; AI accelerates production but the seal binds
- M&A senior partners Deal-thesis judgment; capital-side counterparties pay for upstream inquiry, not memo production
AI as leverage
AI is inside the offering you sell. You own the customer relationship, the question the AI is for, or the capital stack the AI runs on. The AI multiplies what you sell instead of replacing you.
Test: who decides what your AI economics look like next quarter? Bureau Works owns its own MT layer, so its margin survives an OpenAI price change. Founders building thin wrappers on GPT-4 had to renegotiate their P&L in 60 days when OpenAI launched GPT-4 Turbo Nov 2023. Sovereign capital is the difference between catching the expansion and reading about it in a vendor's pricing email.
- Gabriel Fairman Bureau Works - translator-founded platform with AI inside the offering
- Brian de Haaff Aha! - bootstrapped product-management platform, AI inside the offering, no VC
- Senior software architects + Copilot/Cursor System owners shipping faster while boilerplate compresses below them
- M&A partners + Harvey Deal-thesis framing premium; AI inside the workflow they own. A&O Shearman profit-share with Harvey signed Apr 2025
- David Holz Midjourney - ~107 employees, no VC, ~$500M ARR; AI inside the offering they own
- Aengus Tran Harrison.ai / Annalise.ai - radiologist-founded; 35% NHS chest-X-ray share
Commoditized labor
AI is in your daily workflow as a tool, but the firm above you owns the offering. You're a fungible operator inside a traditional pyramid where the AI compresses your tier. Your output goes to a paying client - through your firm's hierarchy - but your role is increasingly substitutable.
The cost curve has crossed the wage line for your tier. Two paths from here: absorption (your seat goes away as AI gets better) or relocation as cleanup (you migrate into the discipline quadrant on the left). Brynjolfsson-Chow-Hoffmann 2025: developers aged 22-25 lost ~20% employment share post-ChatGPT. The senior tier above you captures the differential.
- Tier-1 customer support (with deflection AI) Post-Agentforce/Sierra/Decagon era CS; the AI handles 70%+ containment, the human handles the rest
- Junior software engineers (with Copilot) Routine code production accepting Copilot/Cursor/Cognition diffs in someone else's repo; cascade displacement firing
- Junior paralegals (with Harvey) Routine discovery and contract review under partner signature; Harvey/Spellbook absorb the substrate
- Junior data analysts (with copilots) Routine pull-and-pivot work that analyst copilots absorb
- Workday-operating recruiters Operating the AI scoring system that decides which candidates the firm sees; Mobley v Workday discovery surfaced 1.1B rejected applications
- Per-stamp architect-of-record reviewers Reviewing AI-generated documents for sign-off at per-stamp rates; criticality high, compensation throughput
AI as discipline· The HitL trap
AI is the offering - someone else's vendor product is what the customer pays for. You operate inside that vendor's AI as residual labor. The output you produce feeds back into the vendor's pipeline rather than going to a paying client. Augmented, throughput-priced, the canonical HitL trap.
Two failure modes firing. First: criticality real, compensation throughput - Translation post-editors at $0.03-$0.05 per word despite native fluency, Klarna gig CS at $41/hr handling AI residual. Second: if the work is grading AI output for surface coherence, it dies as buyers learn Dell'Acqua-Mollick 2023: GPT-4 produced more polished output even when systematically wrong.
- Translation post-editors Machine-translation post-edit at per-word piecework rates; translation memory feeds the next model
- Data annotators / RLHF labelers Labeling examples that train the AI they are paid to correct; canonical ghost-work configuration
- Klarna gig customer-service agents Pre-reversal: ~$41/hr cleanup for the AI assistant handling two-thirds of chats. May 2025 walk-back reabsorbed the function in-house
- AI-output reviewer paralegals Reviewing AI-drafted documents at content platforms; surface coherence test, no substance check
- Content moderation reviewers Reviewing classifier-flagged content; ghost-work pattern documented by Gray-Suri
What if your human-in-the-loop work trains the AI that absorbs it?
The matrix above shows snapshot position. The dynamic underneath is more damning.
Every human-in-the-loop worker generates training data through the act of being in the loop. Pick a job below to see the loop play out.
The HITL fallacy assumes the human is the safety perimeter around the AI. The loop is the opposite. The human is the training signal for the AI - and the better the signal, the smaller the perimeter the next model needs.
Is human-in-the-loop one thing happening to one role?
No. Seven configurations firing across thirteen roles. Most cells fire below the algorithm. The fallacy isn't isolated. It's a pattern.
| Cleanup | Surface | Substance | Training | Eval | Edge case | Fail-over | |
|---|---|---|---|---|---|---|---|
| Translators | below Translators · Cleanup Translation post-editing; the canonical cleanup HITL | below Translators · Surface review Compliance copy review at LSP QA tier | above Translators · Substance validation Sworn / court translation under signature liability | below Translators · Training data TM corrections feed vendor MT | below Translators · Edge case Low-resource language pairs | ||
| Paralegals | below Paralegals · Cleanup First-pass doc review | below Paralegals · Surface review AI-output review at content platforms | above Paralegals · Substance validation Citation-audit specialists post-Mata v Avianca | below Paralegals · Training data Edits feed Harvey, Spellbook, EvenUp retrain cycles | below Paralegals · Edge case Junior tier handling escalations Harvey cannot resolve | ||
| Customer-support agents | below Customer-support agents · Training data Chat transcripts feed assistant retrain | below Customer-support agents · Edge case Klarna gig CS overflow at $41/hr; canonical | below Customer-support agents · Fail-over Salesforce HITL pool inside Service Cloud | ||||
| Copywriters | below Copywriters · Cleanup AI-draft post-edit at content mills | below Copywriters · Surface review Brand-voice + tone check at agencies | below Copywriters · Training data Corrections feed agency in-house models | below Copywriters · Edge case Compliance-copy + regulated-domain copy | |||
| Performance marketers | below Performance marketers · Cleanup Andromeda + Advantage+ creative cleanup | below Performance marketers · Surface review Brand-safety + creative review | below Performance marketers · Training data Conversion signals feed platform models | contested Performance marketers · Eval criterion Cross-platform incrementality measurement | below Performance marketers · Edge case Anomaly review on auto-bid swings | ||
| Radiologists | above Radiologists · Substance validation FDA AI/ML pathway overread; image against impression | contested Radiologists · Training data Annotated overreads feed Aidoc / GE Imaging models | above Radiologists · Edge case Uncertain cases the model declines | above Radiologists · Fail-over FDA mandates human-in-the-loop fallback | |||
| Junior software engineers | below Junior software engineers · Cleanup Copilot / Cursor diff review | below Junior software engineers · Surface review PR review of AI-generated code | contested Junior software engineers · Substance validation Senior architect review of AI architecture choices | below Junior software engineers · Training data Accept/reject signals feed Copilot/Cursor models | below Junior software engineers · Edge case Debugging the cases AI cannot trace | ||
| Junior bankers | below Junior bankers · Cleanup Deck polish on AI-drafted decks | below Junior bankers · Surface review Memo polish on AI-drafted memos | below Junior bankers · Edge case Manual research the AI cannot source | ||||
| Journalists | below Journalists · Cleanup Wire-copy edit on AI-drafted news | below Journalists · Surface review Style check on AI summaries | above Journalists · Substance validation Investigative fact-checking against ground truth | below Journalists · Training data Edits feed editorial AI training | below Journalists · Edge case Beat-specific knowledge AI cannot replicate | ||
| Claims adjusters | below Claims adjusters · Surface review Routine claim review on AI-triaged claims | above Claims adjusters · Substance validation Substance validity check + fraud review | below Claims adjusters · Training data Decisions feed insurer underwriting models | below Claims adjusters · Edge case Escalations AI cannot adjudicate | above Claims adjusters · Fail-over Regulatory bind on decision review | ||
| Content moderators | below Content moderators · Surface review Classifier-flagged content review (Sama, Telus) | below Content moderators · Training data Labels feed surface classifier retrain | below Content moderators · Edge case Worst-of-worst edge content the model declines | ||||
| Financial analysts | below Financial analysts · Cleanup Model polish on AI-drafted analysis | below Financial analysts · Surface review Memo polish on AI-generated reports | above Financial analysts · Substance validation Audit-prep substance validation | below Financial analysts · Edge case Manual research AI cannot source | |||
| Recruiters | below Recruiters · Surface review Resume review on AI-scored candidates | below Recruiters · Training data Hire/no-hire signals feed Workday + LinkedIn HA | below Recruiters · Edge case Workday escalations |
Hover any marked cell for the source.
The HITL trap is not a translator problem or a paralegal problem or a CS-agent problem. It is seven configurations firing across the knowledge-work corpus. The fallacy assumes "human-in-the-loop" labels a safe seat. The grid shows what it labels: a pattern of below-the-algorithm work distributed across the roles where AI has already crossed the cost curve.
How did the human-in-the-loop end up below the algorithm?
Four roles where the descent is documented, dated, and load-bearing in the index. Pick a role and watch the algorithm line clear each rung of the bundle. Each chapter cites the platform event, the wage delta, and the named platform or practitioner that absorbed the rung.
What does "human in the loop" mean in practice?
There is no single 'human in the loop.' There are seven.
Each configuration is a distinct pattern of how a human gets relocated as cleanup labour for an AI pipeline. The configuration determines whether the practitioner sits above or below the algorithm. Most configurations land below.
Cleanup
BelowHuman cleans AI output before it reaches the buyer.
Post-edit machine-translation output at per-word piecework rates. The output then feeds the next model's training corpus.
Surface review
BelowHuman grades AI output for coherence and tone.
Scan AI-drafted documents for surface coherence without a statute-grounded checklist. AI is structurally better at producing coherent text than humans are at checking it.
Substance validation
AboveHuman tests AI output against ground truth.
Verify AI-generated citations against actual case law. Compare AI radiology impressions against the underlying image. Attest fidelity to source text under legal liability.
Training data
BelowHuman labels examples that train the AI.
Label examples, write preferences, write demonstrations. Direct ghost-work configuration: the worker's output is the AI pipeline's input, with no client on the other side.
Eval criterion
AboveHuman writes the rubric / golden set the AI is measured against.
Author the criteria, build the golden set, design the test suite the model is measured against. Owning the eval is owning the question the AI is for.
Edge case
BelowHuman handles cases the AI can't.
Take the residual cases the AI declines or fails on. Compensation prices the per-case throughput, not the criticality. Workflow owned by the vendor.
Fail-over
BelowHuman is the fallback when the AI breaks.
Stand by to take over when the AI fails. The human's presence is what lets the AI ship in production; the human is structurally on call rather than producing the offering. Throughput-priced.
What was the human-in-the-loop conversation worth in capital terms?
While the popular conversation argued whether humans would stay in the loop, the capital answered. Six AI-native vendors raised or compounded into the tens of billions on the work that used to belong to translators, paralegals, CS agents, copywriters, and engineers. The labour those vendors run on was priced as throughput.
- Translation post-editor $0.03-$0.05 per word 2024-26 mid-market rate
- Data annotator (Appen) $1.32-$2.00/hr Kenya / Philippines baseline
- Klarna gig CS agent ~$41/hr Pre-May 2025 reversal
- Per-stamp architect-of-record review $80-$120 per stamp AEC AI-draft sign-off
Is your work an input to the AI, or an output to a client?
The line between above and below is not a switch. It is a position on a spectrum. Where on the spectrum does the customer pay for what you ship, versus where does the AI pipeline consume what you produce?
Why does the framing keep producing roles below the algorithm?
The residual theory produces four canonical roles. Three of them land below the algorithm by default.
When the framing starts with 'what can AI not yet do?' it produces a predictable set of compensating-layer jobs. Each one looks defensible because it answers a real gap in the model. But most of them sit below the algorithm unless a specific condition is met.
Validator
below"Humans check AI output to make sure it's right."
The test must be against ground truth, not the AI's coherence. Mata v Avianca (S.D.N.Y. June 22 2023) is the canonical sorting event: the attorney filed an AI-drafted brief with fabricated citations that sounded plausible, the court sanctioned him, citation-audit specialists became the new scarce role. If the validator only grades how well the AI writes, Dell'Acqua-Mollick 2023 predicts the seat collapses.
Relationship manager
below"Humans handle the empathy AI lacks."
The contract has to run through the practitioner, not the platform. Gabriel Fairman at Bureau Works owns the customer relationship and the AI stack inside the offering. The Klarna gig CS agent (pre-May 2025) did not - the contract sat with Klarna, the workflow sat with OpenAI, and the agent carried the residual emotional labour at ~$41/hr until the framing flipped.
Accountability officer
above (rare) or below (common)"Humans bear liability AI cannot."
Formal authority must bind to the seat. Radiologists hold the final read under the FDA AI/ML device pathway; CPAs hold audit signoff under PCAOB rules; trial litigators hold the courtroom; sworn translators hold court / asylum / immigration attestation. An AI-ethics-officer title without enforcement authority collapses on the first incident - watch IBM, Microsoft, and Google's 2023-2024 'responsible AI' layoff rounds for what a non-binding accountability seat is worth.
System architect
below (default) or above (rare)"Humans design the agent workflows AI runs inside."
The practitioner must own the question, not the configuration. Senior software architects who decide what the system is for capture rent; prompt engineers configuring vendor workflows don't. M&A partners framing the deal thesis Harvey accelerates capture rent; Workday operators clicking through scoring screens don't. The test: if you were removed, would the AI produce work for the wrong question or no work at all?
What does the fallacy do to roles?
At the level of job design, the fallacy produces roles that sit below the algorithm. Humans are asked to review outputs, handle exceptions, correct errors, label edge cases. These jobs preserve employment and often reduce leverage. The worker is present; the workflow, customer relationship, pricing power, and learning loop sit elsewhere.
The real question for any role is not whether a human remains in the loop. It is where that human is positioned to own the constraint, the context, the accountability, or the outcome.
What does it do to organizations?
At the level of organizational design, the same framing produces a deeper error. Firms redesign around what AI can and cannot do inside yesterday's workflow. Agents take over the scalable work. Humans fill the gaps. The org becomes a map of machine capability and human leftovers.
That is automation with a human safety layer. It is not the organization a new capability portfolio makes possible.
A genuinely AI-native organization starts from a different question. Given the combination of human and agent capabilities now available, what new problems can the firm solve that it could not solve before? The org chart follows the frontier of solvable problems, not the residual limitations of the tools.
Where do you sit on the matrix?
Six questions. The first asks whether AI is inside your daily work. The next five ask where you sit in relation to the workflow, the customer, and the offering. The result places you on the matrix above and names the durability test firing on your seat.
Is AI inside your daily work?
Whether you use it directly (Copilot, Cursor, Harvey, Smartcat, Workday) or operate downstream of it.
Where does your output go after you produce it?
The next consumer of your work - the AI pipeline, another reviewer, or a paying client.
What does the buyer actually pay you for?
The unit of value the engagement transacts on.
Who is your buyer-of-record?
Who the contract or invoice runs through.
Do you hold a constraint the system can't replicate?
A signature, seal, licence, regulatory seat, or institutional authority.
Who decides which problems the AI gets routed to?
Question framing - who chooses what the AI is for.
-
-
Practitioners in the same quadrant