Onboarding
Slop

01

The Plaque and the Guide

A placard and a guide are both explaining the painting. Only one of them decides what you needed to hear.

A visitor stops in front of a painting she does not understand. To the right of the frame there is a placard: three paragraphs, the artist’s dates, the provenance, a sentence about the treatment of light. She reads the first paragraph, skims the second, and moves on. Twenty feet later a guide is standing with a small group in front of a different painting, and she drifts over. The guide is not reciting anything. Someone has asked why the figure in the corner is turned away, and the guide is answering that, and in answering it says three things about the period that were not on any placard in the building.

Both were explaining the painting. Only one of them left her understanding it.

This is the intuition behind a great deal of unease about how companies are onboarding people right now, and it is worth stating precisely before we start testing it, because the precise version and the loose version behave very differently under evidence.

The loose version says: text is cold, humans are warm, so text onboarding is worse. That version is wrong, and this book will show you the studies that break it.

The precise version says something narrower. It says the placard and the guide differ not in warmth but in who decides what matters. The placard has already decided, identically, for everyone who stands in front of it. The guide decides in the moment, for the person in front of them, on the basis of what that person just revealed by asking. A placard can be exhaustive. It cannot be responsive, and exhaustiveness is not what was missing.

The shape of the problem

Museum researchers have known for decades that visitors do not consume exhibitions the way exhibitions are designed. They do not read every label. They do not stop at every case. Tracking studies — where researchers follow visitors and record what they actually attend to, rather than asking them afterward — find that in a typical exhibition, visitors spend around thirteen minutes and stop at 29 percent of the exhibits.^c-017

Twenty-nine percent. Not because the other 71 percent were badly written, but because attention is finite and the exhibition was designed as though it were not.

The reason to start here, in a book about software engineers, is that the generative-AI era has done something specific to this ratio. It has made completeness nearly free. An organization that could previously afford to document its three most important systems can now generate documentation for all forty. The constraint that used to force prioritization — somebody has to write this — has largely dissolved.

But attention did not become more abundant. If anything it became scarcer. So the ratio between how much explanatory material exists and how much of it any one newcomer can absorb has moved sharply, in the direction of more material and the same person.

That is the first thing to hold onto: the scarce resource was never the explanation. It was the judgment about which explanation you needed.

What a guide actually does

It is tempting to describe the guide’s advantage as brevity — the guide tells you less, so you retain more. That turns out to be wrong, and the study that shows it is the closest thing we have to a direct test of this book’s whole argument.

Researchers at a home-appliance manufacturer built a system that lets a senior engineer record a walkthrough of a codebase as a reusable “code tour,” with a language model drafting the explanation and the expert then correcting and curating it. They compared learners using those expert-curated tours against learners using tours the AI generated alone. On a comprehension quiz scored by an expert who did not know which condition each learner had been in, the expert-curated group scored 83 percent and the AI-only group scored 57 percent.^c-021

Here is the part that undoes the brevity theory. The expert-curated tours were longer — three to five steps longer than the AI-generated ones. And learners got through them about ten minutes faster. The authors read this as expert curation producing more focused learning paths.^c-048

Longer, and faster, and better understood. The expert did not subtract. The expert sequenced: decided what had to come before what, where a newcomer would stall, which detail was load-bearing and which was scenery.

That is a claim about structure, not about volume, and it is the version of the plaque-and-guide distinction that survives contact with data. A guide is not a shorter placard. A guide is a placard that knows what you already understand.

I should tell you immediately how thin this result is. Seven participants: two domain experts and five developers, at one company, on one legacy codebase.^c-022 It is the largest direct measurement of this question that exists, and it would not survive a strong wind. We will come back to what that means in Chapter 8.

Why the metaphor is not just a metaphor

There is a body of theory behind the museum comparison, and it is more useful than the comparison itself.

Falk and Dierking’s Contextual Model of Learning proposes that free-choice learning — the kind that happens when nobody is grading you — arises at the intersection of three contexts: the personal (what the visitor already knows, wants, and is curious about), the social (who they are with, what gets said), and the physical (the building, the objects, the layout). Understanding is not delivered by any one of them. It happens where they overlap.

The mapping onto technical onboarding is almost embarrassingly direct. The personal context is what the new engineer already knows and what they believe they need to learn. The social context is who they talk to and, crucially, who they learn to ask. The physical context is the codebase, the tooling, the documentation, the room or its absence.

Stated in those terms, the worry about AI-mediated onboarding becomes testable. The worry is not that text is cold. It is that a text channel collapses three contexts into one — it delivers the physical context beautifully, serves the personal context only to the extent the newcomer already knows what to ask for, and does not touch the social context at all.

Whether that collapse matters is an empirical question. Most of this book is about the surprising fact that nobody has answered it — and about what companies did in the meantime, which was not to wait.

The argument of this book

Three claims, in order of how well the evidence supports them. The strongest one is not the one you would expect.

One: the feedback loop on onboarding is broken, and broken in a known direction. This is the best-supported claim in the book, and Chapter 7 is devoted to it. When people are asked how an onboarding experience was, their answers come apart from what measurement shows they learned — demonstrated twice, in unrelated settings, by unrelated groups. The error is not random. It favors whichever option produced the smoothest experience, which is usually the automated one. An organization evaluating its onboarding by satisfaction is running an instrument that cannot see what it is losing.

Two: what is missing from AI-mediated onboarding is unmeasured rather than disproven, and the gap is in a specific place. Chapter 8 shows where. The studies that measure understanding run for about an hour. The studies that run for weeks count code. The interval where onboarding actually lives — weeks of accumulating half-understanding — has never been observed by anyone in this literature. Few people discussing AI onboarding know this, and it is the reason the confident claims on both sides are unearned.

Three: the error companies are making is not adopting AI. It is adopting autonomy. This is Chapter 9, and it is the practical heart of the book. The dominant pattern — here is the knowledge base, here is an assistant, ask it anything — treats self-service as an improvement rather than as a cost. It is a pull interface: it can only deliver what the newcomer already knows to request. Onboarding is the period defined by not knowing what to request. Deploying a pull interface against an unknown-unknowns problem is not a tuning error; it is a category error, and the field’s own literature says so in language sharper than mine.

Notice what is not on that list. “AI onboarding produces worse engineers” is not a claim this book makes, because nobody has measured it and I am not going to pretend otherwise. The claim is that a large-scale bet has been placed on the basis of a broken instrument, in a region no one has surveyed, and that the specific design choice being made — remove the human from the loop — is the one the evidence gives least support to.

A caution about that framework, since this book’s method requires me to say so: the quotation most often used to summarize it — that virtually all visitors read some labels but no visitor reads all labels — I could trace only as far as an encyclopedia entry citing page 66 of the 1992 edition. I did not obtain the book. The tracking data above stands on its own and does not depend on it, which is why the tracking data is what appears in the text. Chapter 5 explains why I am telling you this rather than quietly citing the book.

In practice

• Count what your onboarding actually asks a person to absorb. Pages, documents, wiki depth. Then estimate what fraction a new hire reads in week one. If your honest guess is above 30 percent, check it — the museum number is 29 percent for people who chose to be there.
• Separate the two failure modes before you fix either. “We do not have documentation” and “we have documentation nobody can navigate” look identical from the outside and have opposite remedies.
• Ask which of the three contexts your onboarding touches. Most engineering onboarding is entirely physical-context — artifacts and access. If nothing in your program is designed to build the social context, that is a decision, and right now it is usually an accidental one.
• Notice what got cheaper. If your organization is producing more onboarding material than it did two years ago, the relevant question is not whether the material is good. It is whether anyone decided what a newcomer should read first.

02

What Is Actually at Stake

Onboarding's strongest measured predictor is the least writable thing in the company.

Nobody argues that onboarding is unimportant. That is part of the problem. A belief everyone holds and nobody has to defend tends to be held loosely, and loosely held beliefs lose arguments to whatever has a number attached.

So before the interesting questions, the boring one: how badly is this going, and how do we know?

The number everyone cites

Gallup reports that only 12 percent of employees strongly agree their organization does a great job of onboarding new employees.^c-001

It is worth being careful with that figure, because it is one of the most recycled statistics in the entire human-resources literature and recycling damages things. It comes from Gallup’s own publication, not from a blog summarizing Gallup. Gallup does publish a closely related finding elsewhere in different terms — that only one in 10 employees strongly agree their organization does a good job of onboarding^c-004 — which is not a contradiction, but does mean the 12 percent belongs to the specific article that says 12 percent, and citing it to the other one would be wrong in the small way that becomes a large way when somebody checks.

Two things about how Gallup frames the failure are more interesting than the number itself.

The first is that Gallup describes the consequence in relational terms. Poor onboarding, it says, “gets in the way of the formation of an emotional bond between the new hire and the company.”^c-002 Not a knowledge deficit. Not a productivity lag. A bond that fails to form.

The second is which factor moves the outcome most. Across Gallup’s analysis of what predicts an employee calling their onboarding exceptional, the largest multiplier is not about information delivery at all. Employees who strongly agree that they have a good understanding of how things are done at their organization are 4.7 times more likely to strongly agree their onboarding was exceptional.^c-051

Sit with that for a moment, because it is the quiet center of this book. Understanding how we do things here — the least documentable thing in any company, the thing that lives in habits and precedents and what people wince at in code review — is the strongest correlate of onboarding that worked.

Manager involvement is the more famous multiplier. When the manager takes an active role, employees are 3.4 times as likely to feel their onboarding process was successful.^c-003 It is a real finding and a large one. It is also smaller than the culture-understanding effect, and it gets quoted more often because it comes with an obvious action item.

I want to flag the limits of all of this before building anything on it. These are correlational, self-reported on both sides of the relationship, and produced by an organization that sells employee-engagement consulting — which does not make them wrong, but does mean they were gathered by someone with an interest in onboarding being judged inadequate. And the crucial gap: none of these findings say anything about how an understanding of “how we do things” is acquired. That it matters is measured. That talk is what carries it is not.

The felt absence

If the argument of this book is right, the thing to look for is not dissatisfaction in general but a specific complaint: that something human was missing.

A survey of 1,156 US employees hired within the previous twelve months, run jointly by TalentLMS and BambooHR in 2025, asked something close to that. Forty- one percent of Gen Z employees said their onboarding lacked human interaction, against 33 percent of Millennials, 29 percent of Gen X, and 18 percent of Baby Boomers.^c-007

The same survey found that 32 percent of employees said they used AI tools more often than they asked a manager, HR, or colleagues to answer questions during onboarding.^c-005

That 32 percent is the statistic most likely to appear in a slide deck about this topic, and it is the weaker of the two. It measures a substitution, and a substitution can be read benignly — asking a machine at 11pm is not evidence of anything wrong. The 41 percent measures the complaint directly. It is the number to carry.

Both come with the same caveat, stated plainly: this is vendor-run research. TalentLMS sells a learning-management system and BambooHR sells HR software. Both have a commercial interest in onboarding being found broken and in AI being found underused. Nothing in the corpus assembled for this book independently corroborates the 32 percent figure, and the argument should not be built on it alone.

The finding that complicates the story

The same survey contains a result that cuts against the reflexive version of this book’s thesis, and it belongs here rather than in a footnote.

The delivery format with the highest reported satisfaction was not in-person onboarding. It was hybrid, at 75 percent.^c-008

If you believe that the human channel is what matters and text is the problem, that number is not what you would predict. The survey’s own answer is a blend: digital delivery for the parts that survive being written down, human contact for the parts that do not. This book will end up somewhere close to that position, but by a different route and for a more specific reason, and it is worth noticing at the outset that the evidence pointed there before the argument did.

There is also a real limit on how much weight the satisfaction number can bear. Chapter 7 is about why satisfaction ratings are unreliable evidence about learning — a finding that applies to the 75 percent as much as to anything else. Take it as a signal that the answer is not “replace one with the other,” not as a measurement of what works.

What the stakes are not

One category of claim has been left out of this chapter deliberately.

The onboarding literature is full of dramatic cost figures: turnover of up to 50 percent in the first 18 months, replacement costs of six to nine months of salary. Both appear on the same Gallup page as the 12 percent, attributed there to the Society for Human Resource Management.^c-001 They may well be right. But they are second-hand at the point where I could read them, they are the kind of round figure that gets repeated far more often than it gets computed, and the argument in this book does not need them.

If the case for taking onboarding seriously depends on a replacement-cost multiplier, it is a weaker case than the one available. The case available is that the strongest predictor of onboarding that worked is an understanding of how things are done, that this understanding is the least writable thing in the organization, and that we are in the middle of a large-scale bet on writing.

In practice

• Cite the 41 percent, not the 32 percent. One measures the thing you are worried about; the other measures a behavior with an innocent explanation available.
• Ask new hires the culture question directly. “Do you understand how things get done here?” is a better six-week check-in than any satisfaction scale, because it maps to the largest measured effect.
• Treat vendor research as evidence of what was said. Not worthless — the TalentLMS survey is the only measurement of its kind in this corpus — but never as the sole leg of an argument, and always labeled at the point of use.
• Stop quoting replacement-cost multipliers you have not traced. If the number matters to your business case, find the study. If it does not, drop it; it is doing rhetorical work rather than evidentiary work.

03

The Bottleneck Is Not Information

New engineers are not short of information. They are short of the reason the system is shaped this way, and of anyone to ask.

Third week. The new engineer has a failing test in a service she did not write, in a language she knows well, in a repository whose README she has read twice. The README is accurate. It describes what the service does, how to run it locally, and which environment variables it needs. None of that is the problem.

The problem is that the test asserts something odd about retry behavior, and she cannot tell whether the assertion encodes a real requirement or a workaround somebody added during an incident two years ago. The commit message says “fix flaky test.” The author left the company. There are two documents describing the retry policy and they disagree, and nothing tells her which one won.

She has all the information. She is stuck anyway.

What actually blocks new engineers

Google ran a large internal study of engineer ramp-up, surveying more than 3,000 developers to validate objective ramp-up metrics against what engineers reported.^c-012 The three top hindrances it identified were: learning a new technology, poor or missing documentation, and finding expertise.^c-011

The first two are the expected answers. It is the third that reorganizes the picture.

Finding expertise is not a documentation problem. It is not solved by writing more, because the thing being sought is not written anywhere — it is the answer to “who would know?” A newcomer who cannot find expertise is not missing an explanation. They are missing a map of people.

I should be straight about the provenance here, since this book’s method demands it. Google’s column on this research sits behind the IEEE Software paywall; only its abstract was reachable to me. The findings above are bound to a summary published by DX, a company that sells developer-experience measurement tooling.^c-011 The summary is careful and its author is identified, but it is a summary, and DX has a commercial interest in ramp-up being a measurable problem. Read the claim at that strength.

Two kinds of thing a document can hold

The distinction the retry-policy story turns on is old and useful.

A README holds what. What the service does, what it depends on, what the endpoints are. This is genuinely useful and genuinely writable, and language models have made it dramatically cheaper to produce.

The engineer’s actual question is why. Why the retry count is three and not five. What was tried before. Which of two conflicting documents reflects current practice, and whose judgment settled it.

The industry knows this gap exists. There is an entire named practice built around closing it: Architecture Decision Records, which capture a decision together with its rationale rather than merely its outcome.^c-034 The practice exists precisely because outcome-without-rationale is a known failure mode of technical documentation.

Here I have to stop short of the sentence I want to write. The natural next line is that most teams do not maintain ADRs consistently — it matches every practitioner’s experience, mine included. But I could not find a measurement of ADR adoption anywhere in this corpus. The practice’s own reference site advocates it and says nothing about how widely it is followed. So the sentence is not in this book, and its absence is a small demonstration of how the book was built: this is the kind of claim that is true in every conversation and unsupported in every citation.

What can be said is narrower and still useful. The rationale layer is the layer the industry has repeatedly identified as missing, has built a named practice to address, and has not measured its own adherence to.

Why this matters for the AI question

If the binding constraint in technical onboarding were absent information, then generated documentation would be close to a complete solution, and the argument of this book would be sentimental nonsense. Point a capable model at a codebase, produce the explanation that nobody had time to write, and the gap closes.

But if the constraint is unwritten rationale and unfindable people, then generating more documentation is an answer to a different question. It may still help — Chapter 6 shows a case where it measurably did — but it is not addressing the thing the survey data says is blocking people.

There is a related finding worth carrying forward. Work summarized in the code-tour study reports that experienced contributors produced expert-level insights 70 percent of the time when using AI tools, against 48 percent for less experienced participants.^c-027 I read that at one remove — it is another paper’s description of a third paper’s result, which is exactly the citation chain Chapter 5 is about, so weight it accordingly. But the direction is suggestive and it recurs across this literature: AI assistance appears to compound with existing understanding of a system rather than substituting for it. The people best positioned to get value from an agent that explains the codebase are the people who already know the codebase.

If that holds, it is close to a structural problem for AI-first onboarding, because onboarding is definitionally the period when it does not hold.

In practice

• Audit for rationale, not coverage. Pick three recent non-obvious decisions in your systems and try to find why they were made. Coverage metrics on documentation will not tell you what that exercise will.
• Treat “who would know?” as a first-class question. If a new hire cannot answer it for the five systems they will touch, no amount of generated documentation has addressed their top-three blocker.
• Write down the losing options. The cheapest durable ADR is three lines: what we chose, what we rejected, and what would make us revisit. The rejected option is the part that saves someone a week.
• Be suspicious of documentation volume as a health metric. It measures the thing that recently became cheap, not the thing that is scarce.

04

Very Few Heads

Most systems run on one or two heads, and that is a measurement rather than a complaint.

Ask an engineering leader how many people would have to leave before a given system became unmaintainable. Most will guess three or four. Most will be guessing high.

The metric has an unlovely name — the truck factor, or bus factor: the minimum number of developers who would have to be hit by a truck before a project is incapacitated. It sounds like a joke that hardened into a term, which is exactly what it is. But it has been measured.

A study of 133 popular GitHub projects found that the majority of them, 65 percent, have a truck factor of two developers or fewer.^c-014

Two. For most of the systems examined, the knowledge required to keep the thing alive sits in one or two heads.

What the number is, and what it is not

This figure is worth handling carefully, because it is more often quoted than examined, and because the way it is computed matters for the use this book wants to make of it.

The truck factor in that study is estimated automatically from version-history authorship — who wrote which lines, weighted by how much of each file they are responsible for.^c-015 It is not derived from interviewing anybody about what they know. It is a measurement of where code came from, used as a proxy for where understanding lives.

That proxy is reasonable and it is not the same thing. A developer who wrote a module three years ago and has not touched it since may remember less about it than a reviewer who reads it weekly. A system with three active authors may still have exactly one person who understands why the architecture is shaped as it is. The metric could be over- or under-stating the concentration of actual knowledge; the study does not, and cannot, tell us which.

The authors did check their estimates against reality in a limited way. They surveyed developers from 67 of the target systems, and in 84 percent of the valid responses the developers agreed or partly agreed that the identified authors were the main authors of their systems.^c-014 That is corroboration of the authorship estimate. It is not corroboration that authorship equals understanding.

I should also note where I read this: the paper’s abstract page, not its full text. The corpus size, the method, and the headline finding are all stated in the abstract, which is what the claim above rests on.

One further note on how this figure usually travels. It reached my own earlier draft of this argument by way of an encyclopedia entry rather than the paper — which is how most people encounter it, and which is fine until the encyclopedia entry and the paper disagree. They do not here. Chapter 5 is about the cases where they did.

Why concentration is the load-bearing fact

Chapter 3 argued that the binding constraint in technical onboarding is unwritten rationale and unfindable people. This chapter supplies the reason that constraint is hard rather than merely annoying.

If the rationale behind a system lived in twenty heads, the failure of any particular transmission would be cheap. Miss it from one person, get it from another. The newcomer’s difficulty would be a scheduling problem.

At a truck factor of two, it is not a scheduling problem. It is a single-point-of-failure problem wearing a human face. The knowledge is not merely undocumented; it is undocumented and held by approximately one person, who has their own deliverables, who is the same person everybody else also asks, and who will eventually change teams.

This is what makes the onboarding question structural rather than sentimental. The reason “just ask someone” is not a complete answer is that in most systems there is essentially one someone, and the demands on that person do not scale with the number of newcomers.

It is also the reason the AI proposal is genuinely attractive rather than merely fashionable. An agent that could absorb what those two heads know and answer from it would solve a real and expensive problem — not a comfort problem, a continuity problem. Anyone dismissing AI onboarding out of hand should sit with how badly this needs solving.

The question this book keeps returning to is not whether that would be valuable. It is whether what is currently being deployed does it, and whether anyone has checked.

The thing that walks out the door

There is a particular kind of loss worth naming, because it is invisible on every dashboard.

When a person with high truck-factor weight leaves, the code stays. The documentation stays. What leaves is the set of judgments that were never requests: knowing which test failures are real, which service is fragile under load in a way its metrics do not show, which apparently reasonable refactor will break a downstream consumer nobody has talked to in a year.

None of that is retrievable from the artifacts, because it was never in the artifacts. It was in the person, and it was transmitted, when it was transmitted at all, by working alongside them.

That transmission is the thing this book is asking about. Chapter 9 gets to the sharpest version of the question: whether an interface that answers every question well might prevent the newcomer from ever building the map of who to ask — and whether we would notice if it did.

In practice

• Compute it, do not estimate it. Truck-factor tooling is freely available and runs against your history in minutes. The gap between the guessed and the measured number is usually the interesting part.
• Look at the second name, not the first. A truck factor of two means one departure from a very short list. Ask who the second name is and whether they know they are on the list.
• Separate authorship from understanding when you act on it. The metric finds who wrote it. Pairing a newcomer with the person who reviews it may transmit more, and costs the scarce author less.
• Treat the concentration number as the business case. If you need to justify protected time for knowledge transfer, this is a stronger argument than any satisfaction score: most of your systems are two departures from trouble.

05

What Traveled, and What Broke

Four citations, three of them mine, and what their failure has in common with a badly onboarded engineer.

This chapter is about four citations, three of which were mine.

It sits in the middle of the book rather than in an appendix because the failures it describes are not clerical. They are the same failure the book is about, happening to the book itself: a claim that traveled through an intermediary, arrived sounding right, and was not checked because it did not feel like it needed checking.

The corpus

Twenty-two sources were captured for this book. Their distribution says something uncomfortable before any individual one is examined.

The apex is empty. There is no primary evidence in this corpus — no dataset, no filing, no original measurement I could recompute from. Every claim in this book sits at least one step from the thing it describes. That is a fact about the field rather than about the search: onboarding is studied through surveys and small experiments, and the surveys’ underlying response data is not published.

Nearly a quarter of the corpus is material that counts as evidence of what somebody said rather than evidence that it is true — vendor blogs, press releases, encyclopedia entries.

That is not a disaster. It is the shape of the available evidence, and a book that pretended otherwise would be lying about its foundations. But it does set a ceiling on how strongly anything here can be stated, and it makes the integrity of each individual citation matter more than it would in a field with primary data to fall back on.

The word that changed

Gallup’s manager-involvement finding is that employees are 3.4 times as likely to feel their onboarding process was successful when the manager takes an active role.^c-003

My earlier draft of this argument said they were 3.4 times more likely to call the experience exceptional.

It is one word. The number is right, the organization is right, the direction is right. And it is wrong, in a way that is invisible from inside the sentence and obvious the moment you open the page.

The mechanism is worth understanding because it generalizes. Gallup does use “exceptional” — for a different set of multipliers, in a different article, about different predictors.^c-051 Somewhere in the chain from Gallup’s page through an aggregator to a summary to my draft, the more striking word attached itself to the more striking statistic. Nobody lied. Each step was a small, plausible compression, and the compressions accumulated in the direction of a better story.

The quotation I could not trace

The Contextual Model of Learning is usually introduced with a line about labels: that virtually all visitors read some labels, but no visitor reads all labels. It is a good sentence. It does exactly the work an opening quotation should do.

I could trace it as far as Wikipedia’s article on The Museum Experience, which attributes it to Dierking and Falk, 1992, page 66.^c-016 Searching for the sentence itself returns Wikipedia and pages derived from Wikipedia.

I did not obtain the book. So the honest description of the situation is: I have an encyclopedia’s report of a sentence in a book I have not opened, and if I printed it as a quotation from Falk and Dierking, I would be presenting a second-hand attribution as a first-hand one.

There are two respectable ways out and one disreputable one. The respectable ones: obtain the book, which is a library problem rather than a research problem; or find independent evidence for the same point, which is what Chapter 1 does with the visitor-tracking data on the 29 percent stopping rate.^c-017 The disreputable one is to print the quotation and hope, which is what almost everyone does, because the sentence is real, the attribution is probably right, and the cost of being wrong is low right up until someone checks.

The forecast that grew a clause

Gartner forecasts that 40 percent of enterprise applications will include task-specific AI agents by 2026, up from less than 5 percent at the time of the forecast.^c-009 That is real.

My draft added two words: “onboarding included.”

Gartner’s own examples of what those agents will do are automating development, managing incidents, and resolving support cases.^c-010 Onboarding is not among them. The extrapolation is defensible — onboarding is a workflow, agents are coming for workflows — but it is my extrapolation wearing Gartner’s authority, which is a different thing from a forecast.

A note on the citation itself: gartner.com returns an automated-access error to both my capture tooling and a direct fetch, so the claim above is bound to a trade-press reproduction of the release rather than to the release.^c-010 This book’s link checker treats that host as a known-blocked warning rather than a failure. I mention it because “the primary source is unreachable” is a fact a reader deserves at the point of use rather than in a methods note.

The claim an auditor rejected

The last one is the most useful, because it was caught by machinery rather than by conscience.

Chapter 6 discusses a vendor study reporting that engineers using AI daily reached their tenth pull request in 49 days against 91 for engineers not using AI. My draft made a sharp observation about it: that the research is silent on tacit knowledge.

The verification process for this book routes claims that cannot be settled by quotation to an independent reviewer working from a fresh context, which sees the source and the claim but never the argument the claim is serving. It read the whole post and rejected the wording.

The rejection was correct. The post is not silent on the qualitative layer. Its closing takeaway asserts that AI helps new hires “understand codebases, internal processes, and best practices faster than traditional onboarding alone.”^c-020 That sentence leaves pull-request throughput entirely.

The accurate version turned out to be sharper than the one it replaced. Every measure in the post is a code-shipping count. Its conclusion is about understanding. The problem was never silence — it was an unbridged gap between what was measured and what was claimed.

I have kept this one prominently because of what it says about the method. The error was not a fabricated fact. It was an accurate-sounding characterization that flattered my argument, and I would not have caught it by rereading, because rereading is done by the person who already believes it.

Why this chapter exists

A reader is entitled to ask why they should trust the rest of this book more for having read a chapter about its author’s errors.

The answer is that the four failures above have a common shape, and it is the shape of most citation failure in ordinary professional writing. Nothing was invented. Every number was real, every organization was correctly named. What went wrong was transmission: a word swapped, a clause added, a source attributed one link upstream of where it was actually read. None of it is detectable by reading carefully. All of it is detectable by opening the page.

That is also, not coincidentally, an argument about onboarding. A claim that passes through three summaries arrives fluent and slightly wrong. So does an explanation of a system.

In practice

• Open the page. Not the summary of the page. The single highest-yield practice in this entire book takes about ninety seconds per citation.
• Distrust the striking word. When a statistic arrives with an unusually quotable adjective attached, that adjective is the part most likely to have been added in transit.
• Say where you actually read it. “Reported by X, summarizing Y” is a complete and honest citation. Citing Y directly when you read X is not.
• Route your sharpest claims past someone who has not seen your argument. The claim that most needs checking is the one that fits best.

06

The Strongest Case for the Machines

The case a competent advocate would make for handing onboarding to the machines. It is stronger than the sceptics admit.

This is the chapter that argues against the rest of the book. It is here in the middle, at full strength, rather than in a section near the end labeled “limitations,” because a counter-argument that appears after the conclusion has already been answered by placement.

Read it as the case a competent advocate would make. The two chapters after it are about why that case, made in exactly these terms, is how the whole field talked itself into the wrong design.

Three results. None of them is decisive. Together they are enough that anyone holding the plaque-and-guide position casually should stop holding it casually.

Result one: they ramp faster

DX analyzed data from six multinational enterprises, measuring how quickly new hires reached their tenth pull request — an established proxy for ramp-up time.^c-019 Engineers using AI daily hit their tenth pull request in 49 days. Engineers not using AI took 91.^c-018

Forty-two days. On a measure that predicts, according to research the same post cites, a great deal about a developer’s trajectory.

The weaknesses are real and I will state them, but notice first that this is not a small effect on a silly metric. Shipping merged changes is a reasonable observable for “is this person contributing yet,” and near-halving the time to get there would be worth a great deal to any engineering organization.

Now the weaknesses. This is vendor-published research from a company selling developer-productivity measurement. It is observational, not experimental: nobody assigned engineers to use AI or not. The engineers who chose to use AI daily are not a random sample of engineers, and they may differ in ways — tenure, confidence, the kind of work they were given — that would produce the same gap with no causal contribution from the tool at all. The post does not report a sample size for the onboarding cohort.

None of that makes the result worthless. It makes it a correlation of unknown composition, which is a different thing from nothing.

Result two: an agent-first design was rated helpful

Researchers built an onboarding assistant for a development environment, combining a language model with retrieval over the project and an automated chain-of-thought layer. Their stated design goal is worth quoting exactly, because it is the position this book is skeptical of, held deliberately: the system “is agent-based and provides customized assistance with minimal human intervention.”^c-054

Eight participants used it on a real codebase. Seven of the eight completed all tasks. Mean perceived helpfulness came out at 3.26 out of 4.^c-028

So: an onboarding tool explicitly designed to reduce human involvement, and the people who used it found it helpful and got their work done.

Its limits are severe and the authors say so themselves: eight participants, one codebase, and a stated intention to test more broadly later. There is no control condition, so we do not know how those eight would have done with a mentor or with nothing. And the outcome is self-reported perception, which the next chapter is entirely about.

Result three: better text beat worse text

This is the one that actually threatens the thesis, and the one I found hardest to argue with.

If the problem with onboarding documentation were that text is the wrong medium, then improving the text should hit a ceiling fast. Instead: researchers used a generative pipeline to restructure open-source onboarding documentation according to cognitive-load principles — segmenting it into task-based units, inferring workflows, removing redundancy, generating multimodal explanations. In a between-subjects evaluation with 14 newcomers, participants using the restructured documentation achieved higher task success, had significantly lower cognitive load, and perceived higher usability.^c-029

The medium did not change. Only the structure did, and the structure was improved by exactly the technology this book is suspicious of.

That result is a preprint with a small sample, evaluated by the people who built the prototype. Fine. It still says something the sentimental version of the argument cannot absorb: a large share of what gets blamed on text is attributable to badly organized text, and the machines are good at reorganizing.

What the three results share

Line them up and something becomes visible that none of them shows alone.

The first measures pull requests. The second measures task completion and perceived helpfulness. The third measures task success and cognitive load on a bounded exercise.

Every one of these is a measurement of performance on a defined task within a short window. Not one measures whether the newcomer understood the system, could find a person, or knew what they did not know. That is not a criticism of the researchers; those things are hard to measure and the ones they chose are legitimate.

But it means the case for AI onboarding, at its strongest, is a case about task throughput. Which would be fine, except that the field has started concluding more than that.

Return to the DX post. Its evidence is entirely code-shipping counts. Its conclusion, in its own words, is that “AI helps new hires understand codebases, internal processes, and best practices faster than traditional onboarding alone.”^c-020

Understand. Internal processes. Best practices. Nothing in the study measured any of those.

This is the exact move an earlier draft of my argument mischaracterized as silence, and the correction in Chapter 5 sharpened it: the problem is not that the qualitative layer goes unmentioned. It is that the qualitative conclusion gets asserted on quantitative evidence that does not reach it, in the takeaway paragraph, where a busy reader will take it as a finding.

The honest position after this chapter

AI-mediated onboarding demonstrably helps people complete tasks faster, in several independent settings, by measures that are legitimate.

Whether it builds understanding is unmeasured. Whether it builds the map of who knows what is unmeasured. Whether either degrades over the weeks a real onboarding takes is unmeasured.

The rest of this book is about that gap, and about a specific reason it has stayed open: the field’s default measurement instrument cannot detect the thing in question. That is the next chapter.

In practice

• Steelman before you deploy or refuse. The strongest case for these tools is a real, replicated speedup on task completion. Any decision that ignores it is not being careful, it is being sentimental.
• Restructure your documentation before replacing it. The result with the cleanest design here says the structure was the problem. That is the cheapest intervention available and you can start this week.
• Read the takeaway paragraph against the methods section. In vendor research the gap between the two is where the claim you will end up repeating usually lives.
• Ask what a study would have had to measure to support its conclusion. If the answer is “something it did not measure,” you have found the seam.

07

The Broken Feedback Loop

Two unrelated studies found the same thing: what people say about an AI tool comes apart from what measurement says about them.

Almost every claim in the previous chapter rests, somewhere in its chain, on somebody reporting how an experience felt. Perceived helpfulness. Perceived usability. Satisfaction with the format.

This chapter is about why that instrument does not measure what it is being used to measure, and about the two independent results that establish it.

It is the most important chapter in the book. Not because the finding is surprising on its own — everyone knows surveys are soft — but because of what it implies about every decision made downstream of one. An organization that evaluates onboarding by asking people how it went is not making a slightly noisy measurement. It is making a measurement whose error runs consistently in one direction: toward whatever felt smoothest, which is reliably the option with the least friction and the least human contact in it.

The first result

Return to the code-tour study from Chapter 1. Learners worked through expert-curated tours and AI-only tours, and their comprehension was scored by an expert blind to condition. Curated: 83 percent. AI-only: 57 percent.^c-021

Now the part that matters here. The researchers also asked the learners what they thought. Learners only moderately agreed that expert curation combined with AI was more valuable than AI alone — and, in the authors’ phrasing, their performance told a different story.^c-023

Twenty-six points of measured comprehension, and the people who lived through both conditions rated the difference as marginal.

The paper’s explanation is worth stating carefully because it is the mechanism this whole book turns on. AI-only tours produce smooth explanations that feel immediately intuitive, creating a sense of understanding.^c-023 The fluency of the explanation is doing the work that comprehension is supposed to do. Nothing in the experience signals a gap, because a gap in understanding does not announce itself — that is what makes it a gap.

The authors draw the organizational conclusion themselves, and it is the single most useful sentence in the corpus assembled for this book: organizations that rely on satisfaction ratings alone will systematically undervalue expert curation and default to the faster AI-only path.^c-024

Not “may misjudge.” Systematically undervalue. The measurement instrument has a direction to its error, and the direction points at whichever option is cheaper.

Seven participants. I will keep saying it.

The second result

If that were the only evidence, it would be a suggestive finding from a tiny study. It is not the only evidence, and the corroboration comes from a completely different setting, which is what gives it force.

METR ran a randomized controlled trial with 16 experienced open-source developers, working on real issues in repositories they had contributed to for years — 246 tasks, randomly assigned to permit or forbid AI tooling.^c-031

The measured result was that developers took 19 percent longer when allowed to use AI. But that headline has since been superseded by METR’s own follow-up work, and I will come back to it in a moment, because it is not the part that matters here.

The part that matters: before starting, the developers forecast that AI would speed them up by 24 percent. After completing the tasks — after living through the measured slowdown — they still believed AI had sped them up by 20 percent.^c-033

They were inside the experiment. They had done the work. And their self-report was wrong by roughly 39 percentage points, in the flattering direction.

Why two studies make a finding

These two results come from settings with nothing in common. One is an industry onboarding study at a home-appliance manufacturer, with novices learning a legacy finance system. The other is a randomized trial with senior open-source maintainers working on code they wrote. Different populations, different tasks, different research groups, different decades of experience. Neither cites the other.

Both found the same thing: what people report about their experience with an AI tool comes apart from what measurement shows about their performance with it.

That convergence is the strongest synthesized finding in this book, and it is stronger than either study alone precisely because the two are independent. Four articles rewriting one press release is one source. Two unrelated research groups arriving at the same divergence by different routes is evidence.

What this does to Chapter 6

It does not demolish the previous chapter. It reweights it, unevenly, and the unevenness is the useful part.

The agent-first pilot rated 3.26 out of 4 for perceived helpfulness. That is a satisfaction measure, and this chapter is the reason a satisfaction measure cannot establish that an onboarding tool worked. Discount it heavily.

The DX ramp-up result is not a satisfaction measure — pull requests are counted, not felt. It survives this chapter intact, with its original weaknesses unchanged.

The restructured-documentation study measured task success as well as perceived usability. The task-success half survives. It remains, for my money, the most serious challenge in the corpus to a text-is-the-problem position.

So the honest scorecard after two chapters: the case for AI onboarding stands on throughput and task completion, and loses the parts that rested on how people felt.

The figure you should not cite

A word about the 19 percent, because it is about to be quoted at you.

METR’s early-2025 slowdown finding is superseded. In a February 2026 update, the same group reported that they believe developers are likely more sped up from AI tools now, in early 2026, than their early-2025 estimates suggested — while also explaining that their newer data suffers from selection effects severe enough that they are redesigning the study.^c-032 Their newer estimates point toward a speedup with wide confidence intervals.

Citing “AI makes developers 19 percent slower” as a current fact is exactly the citation-transmission failure Chapter 5 catalogs, and it is now the most common one in this subject area. The perception gap survives. The productivity headline does not.

That distinction — which part of a superseded study still stands — is the kind of thing that only survives if somebody reads the follow-up. Most people cite the original.

In practice

• Never evaluate an onboarding tool on satisfaction alone. The error has a known direction, and it favors whatever is cheapest to run.
• Score comprehension against a rubric, blind to condition. It is more work than a survey and it is the only thing in this literature that has detected the effect at all.
• Ask people to predict their score before they take the test. The gap between predicted and actual is free to collect and is the most sensitive instrument available for the failure mode in this chapter.
• Check whether the study you are citing has been superseded by its own authors. In this field the follow-up is often published quietly and says something different.

08

The Missing Quadrant

The studies that measure understanding run for an hour. The studies that run for weeks count code. Nothing occupies the space between.

Everything so far has been about what the studies found. This chapter is about where the studies stood when they looked, and it contains the finding that reorganized this book.

Two axes

Sort the evidence on onboarding by two questions.

First: how long did the measurement run? Minutes, or months?

Second: what did it measure? Something about output — code shipped, tasks completed — or something about understanding?

Place every study in this book’s corpus on those axes and a pattern appears that none of them shows individually.

The code-tour study measured comprehension with a blinded rubric — the deepest measure in the corpus. It ran in single sessions of approximately 60 minutes per condition, with the quiz at the end.^c-049

The restructured-documentation study and the agent pilot both measured task performance, both in single sittings.

The one study that spans weeks is the vendor analysis of time to a tenth pull request. It measures code-shipping volume.

The long measure captures the shallow thing. The deep measure captures the short thing. The quadrant where understanding is measured across the duration onboarding actually takes is empty.

Why the empty quadrant is the whole argument

An hour is the best case for a text interface.

Consider what that hour looks like. Attention is fresh. The scope is bounded — one feature, one subsystem. The question has just been handed to you by a researcher, so you know what you are trying to find out. There is an end in sight.

Now consider week three of a real onboarding. You have been reading for eleven days. You have four half-understood systems in your head and no clear sense of which of them the thing you are working on depends on. Nobody has handed you a question; you are supposed to be generating your own, and the reason you cannot is that you do not yet know enough to know what is missing. This is the state in which people describe onboarding as isolating, and it is a state that no study in this literature has observed.

The gap is not hypothetical, and the sharpest evidence for its size comes from inside the deepest study. The code-tour researchers surveyed their learners about their own history. Forty percent reported needing five months or more to reach productivity; another 40 percent, one to two months.^c-050

Sixty minutes of measurement, standing in for a process the participants themselves described in months.

Why the field ended up here

This is not incompetence. It is the ordinary consequence of what is easy to run.

A systematic review of 32 studies of software-onboarding solutions found that laboratory experiments were the most common evaluation method, and — its own summary of the state of the field — that many onboarding barriers remain unaddressed by existing solutions.^c-039

Lab-shaped evidence about a workplace-shaped problem. A controlled comparison in a single session is publishable, affordable, and clean. An eight-week longitudinal study with repeated blinded comprehension probes across a real intake cohort is expensive, messy, and vulnerable to two people quitting.

So the literature accumulated where the light is. Which would be a familiar and forgivable state of affairs, except that deployment did not wait.

The other thing that is missing

There is a second absence, related and worse.

No source in this corpus measures relational integration, belonging, or the transfer of tacit context under AI-mediated onboarding. Not measured and found adequate. Not measured.

I want to be precise about the difference between three things that get blurred together whenever someone says “there is no evidence.”

Checked and absent: I looked, and the evidence is not there. The relational outcomes fall here. So does independent corroboration of the 32 percent figure from Chapter 2, and any measurement of ADR adoption from Chapter 3.

Not checked: outside this book’s scope, or beyond where I could reach. The Falk and Dierking book itself. Google’s column behind the IEEE paywall. General human-resources onboarding, deliberately excluded.

Unknowable from public sources: the data exists and is not disclosed. Gartner’s methodology for its 40 percent forecast. Whether DX’s six enterprises resemble anyone else’s.

Most confidently wrong research is a “not checked” written up as a “checked and absent.” The relational gap is genuinely the first kind, and that is what makes it usable as an argument.

What follows, and what does not

Here is what the empty quadrant licenses, and it is less than I wanted when I started.

It licenses saying that the question is open. It licenses saying that the confidence with which text-first onboarding is being deployed exceeds the evidence for it, because there is no evidence about the interval that matters. And it licenses the specific worry, because the unmeasured interval is precisely the one where practitioners report the failure.

It does not license saying that AI onboarding fails over weeks. Nobody has looked. An argument from absence is an argument that the case has not been made, never an argument that the opposite has been.

The honest position, then: the thesis of this book is defensible and undertested. The sources genuinely disagree. One small controlled study supports it directly; two others show AI-mediated and text-based onboarding performing well on their own measures; and the study that would settle it has not been run.

That is an unsatisfying place to arrive in the middle of a book. It is also the only place the evidence actually reaches, and the last chapters are about what to do from here — first as an organization, then as a researcher, because the question is answerable and somebody should answer it.

In practice

• Ask any onboarding vendor how long their evidence ran. If the measurement window is shorter than your onboarding, the result does not transfer, whatever it found.
• Instrument at week eight, not week one. Week-one measurement catches enthusiasm. Week-eight measurement catches whether anything stuck.
• Keep the three kinds of “no evidence” separate in your own reasoning. Most confident wrongness lives in the confusion between them.
• When the evidence is thin, say undertested rather than wrong. They point to different actions: one says stop, the other says measure.

09

The Autonomy Error

Here is the knowledge base, here is the assistant, ask it anything. The mistake is not the assistant. It is the word anything.

Here is how it goes now.

A new engineer’s first day includes a link to the knowledge base and a link to the assistant. The knowledge base is more complete than it has ever been — generated, cross-linked, current. The assistant sits on top of it and will answer anything, instantly, at three in the morning, without making anyone feel they are imposing. Somebody says, sincerely and helpfully: ask it anything, it knows more about the codebase than I do.

Everyone involved believes this is an improvement. On most of the dimensions anyone is measuring, it is.

This chapter is about why it is nonetheless the central mistake of the current moment, and the argument does not require the assistant to be bad at its job. It requires the assistant to be good at its job. That is what makes the mistake hard to see.

The thing that got optimized

Notice what the arrangement above is designed to reduce.

Not time-to-answer, primarily — a colleague at the next desk was already fast. What it reduces is the number of times a newcomer has to involve another person. That is the design goal, stated plainly, and it is not my inference: an onboarding assistant published in 2025 describes itself as providing “customized assistance with minimal human intervention.”^c-054

Minimal human intervention. As a feature.

The reasoning behind it is not stupid. Senior engineers are the scarcest resource in the building. Chapter 4 established that most systems run on one or two heads, and those heads have their own deliverables. Every question routed to a machine is a question that did not interrupt the person who is already interrupted too often. Chapter 10 takes that constraint seriously, because it is real.

But look at what has happened to the objective function. The organization set out to onboard people. It is now optimizing for not needing anyone, and it has started calling that success.

Why “ask it anything” is the wrong shape

Every question-answering interface, however sophisticated, is a pull interface. It delivers on request. Its coverage is bounded not by what it knows but by what the person in front of it thinks to ask.

For most tools that constraint is invisible, because the user knows what they want. A search engine works because you know the thing you are looking for.

Onboarding is the one period in a professional life defined by not knowing what you are looking for. That is not incidental to onboarding; it is the definition of it. Somebody who knew which questions to ask about a system would already understand the system.

The evidence for this is not just conceptual. When the code-tour researchers surveyed their learners about what stopped them asking questions, two barriers tied at 80 percent: reluctance to interrupt colleagues, and uncertainty about what to ask.^c-026 Five learners, so hold it loosely. But note that an AI assistant addresses the first barrier completely and the second one not at all.

We deployed a solution to the half of the problem that was easy to see.

This is the sharpest form of the argument in this book, and it is worth stating without hedging: a pull interface is a category error for an unknown-unknowns problem. Not underpowered — mis-shaped. You cannot fix it with a better model, because the limiting factor is not on the model’s side of the conversation.

The critique from inside the field

I would be more tentative about this if it were only my reasoning. It is not.

The authors of the code-tour system surveyed the existing AI onboarding tools — the retrieval-augmented assistants, the agent-based buddies — and their assessment is direct. Those approaches, they write, “operate as reactive tools, answering questions when asked rather than providing structured learning paths that guide newcomers through the codebase step by step. Furthermore, by minimizing human involvement, they eliminate the expert-newcomer communication.”^c-053

That is a peer-reviewed paper at a major software-engineering conference saying, in 2026, that the dominant design pattern eliminates the thing onboarding is for.

Two honest caveats. These are the authors of a competing system, so the criticism is interested. And it is a criticism of design, not a demonstration of worse outcomes — nobody has run the comparison over a real onboarding, as Chapter 8 established at length.

But it is not a fringe view, and it is not a nostalgic one. It is the assessment of the researchers who looked hardest at these tools.

Autonomy as an accident

The most uncomfortable part of this chapter is that nobody decided it.

Very few organizations sat down and resolved that new engineers should learn in isolation. What happened is subtler and worse: the self-service option scored well on the instrument they were using, so it won on the merits, repeatedly, without anyone noticing the merits were mismeasured.

Chapter 7 supplied the mechanism. Satisfaction ratings come apart from measured learning, and the code-tour authors state the organizational consequence outright: organizations relying on satisfaction ratings alone will systematically undervalue expert curation and default to the faster AI-only path.^c-024

Read that alongside the pull-interface problem and the shape of the error becomes clear.

1. The autonomous option genuinely feels better to the person using it — smooth answers, no social cost, no waiting.
2. Feeling better is what gets measured.
3. So the autonomous option wins the evaluation.
4. And what it costs is precisely the thing the evaluation cannot see.

That is not a story about foolish executives. It is a story about a control loop with a systematic bias, running unattended, at scale, across an industry. It will keep producing the same answer for as long as the instrument stays the same, and it will produce that answer more confidently every year, because each iteration generates more evidence of the kind the instrument can read.

What is actually being lost

Be specific, and stay inside what can be supported.

The map of who knows what. Chapter 3 identified finding expertise as one of the three top blockers to engineer ramp-up.^c-011 An assistant that answers every question well means the newcomer never has to discover who owns the payments service, who to trust when two documents conflict, who wrote the retry logic and why. They get the answer without acquiring the map. Whether the map then fails to form is, I have to say again, unmeasured — it is the measurement I propose in Chapter 12, because it is the cheapest way to find out.

The relationship the question would have created. A question asked of a colleague is not only a request for information; it opens a channel that carries things nobody would have thought to write down. Eliminate the question and the channel does not open. The paper quoted above names exactly this: minimizing human involvement eliminates expert-newcomer communication.

The signal that the newcomer is lost. This one gets no attention at all. When a new engineer asks a senior engineer three confused questions in a row, a human notices and intervenes — not because they were asked to, but because confusion is legible in person. An assistant answers all three cleanly and no one is alerted. The organization’s mechanism for detecting a struggling new hire was, historically, the irritation of the person being asked. We removed it and installed nothing.

I cannot cite that last one, so I will mark it as what it is: my inference from the shape of the mechanism, offered as a hypothesis worth testing, not as a finding.

What this chapter is not saying

It is not saying to remove the assistant. The evidence in Chapter 6 is real: people ramp faster on task measures with these tools, and restructured documentation genuinely reduced cognitive load. Withdrawing them would cost something measured to protect something unmeasured, which is a bad trade to make on a hunch.

It is saying that autonomy is a cost, not a feature, and that it is currently being counted on the wrong side of the ledger. The right design question is not “how few humans can this require?” but “which parts of this genuinely require a human, and have we protected those?”

The survey evidence from Chapter 2 pointed at the same answer from a different direction: the highest-satisfaction onboarding format was hybrid, not either pure form.^c-008 That number carries all the caveats of this book’s seventh chapter. But when the sentiment data, the design critique, and the one controlled comparison all point at blend rather than replace, the burden has shifted onto the people building for minimal human intervention.

In practice

• Count the human touchpoints in your onboarding, and treat the number as a budget rather than a cost. If it has gone down over two years while your tooling improved, you have run the experiment in this chapter without a control group.
• Stop shipping “ask it anything” as the whole program. It is a fine supplement and a poor spine. The spine has to push — a sequenced path someone decided on — because the newcomer cannot pull what they cannot name.
• Schedule the walkthrough; do not offer it. An offer of help routes through exactly the reluctance-to-interrupt barrier that 80 percent of those learners named. A calendar invite does not.
• Instrument for silence. A new hire who asks nobody anything for a week is either exceptional or lost, and your current dashboard reads those two states identically.

10

The Guide’s Work Is Ephemeral

Guidance is treated as a recurring tax on the people least able to pay it. One result suggests it need not be.

The previous chapter ended by asking which parts of onboarding genuinely require a human. This one is about why organizations avoid answering that question, and about a result that suggests the avoidance is less necessary than it looks.

The bind

Every engineering leader knows this shape. Guiding a newcomer takes real time from the people least able to spare it. The person best placed to explain why the system is like this is the person whose absence from their own deliverables is most expensive.

The problem is described directly by practitioners. Mentorship, one widely read engineering writer puts it, is time- and emotionally consuming, and it is often invisible to the mentors’ own managers.^c-036 That second half is the operative one. Work that is invisible to the person doing your performance review is work you do at your own expense.

I should be clear about what kind of evidence this is: a practitioner blog post with no sample and no method. It establishes that the problem is named and recognized in the field. It does not establish how common it is, and nothing in this book’s corpus measures that.

The same writer makes a more useful observation about scope. Onboarding is a bounded form of mentoring — it lasts from the first day until the new starter is onboarded, typically one or two months.^c-037 That matters because it makes the ask finite. Nobody is being asked to sponsor a career. They are being asked for six weeks.

And yet it routinely does not get resourced, which is the pattern worth explaining rather than deploring.

Why the bind resolves the way it does

Chapter 9 described a control loop with a bias. Here is the same loop, seen from the budget side.

The cost of human guidance is visible, immediate, and attributable: a named senior engineer, hours, a slipped deliverable. The cost of its absence is delayed, diffuse, and attributable to nobody in particular — a new hire who takes five months instead of two, an outage six months later that a better-oriented engineer would have avoided, a resignation that gets coded as “not a fit.”

Any organization comparing a legible cost against an illegible one will systematically choose to incur the illegible one. It does not require bad faith. It requires a spreadsheet.

Add the finding from Chapter 7 — that the instrument used to check the outcome reports satisfaction, which stays high because the automated path feels smooth — and the loop closes. The cheap option is chosen, the measurement confirms it was fine, and the confirmation funds the next round of the same decision.

The result that reopens it

This is where I expected to write a resigned paragraph about tradeoffs. One finding stopped me.

In the code-tour study, the experts did not only prepare tours; they were asked about the experience of preparing them. They found creating a reusable tour less burdensome than giving live walkthroughs.^c-025

Two experts. One company. I am not going to dress that up — it is the thinnest result cited in this book, and Chapter 8’s complaint about small samples applies to it with full force.

But consider what it would mean if it held, because it reframes the entire chapter. The bind assumes that guidance is inherently ephemeral: the senior engineer explains the system, the explanation evaporates, and the next newcomer requires the same hour again. Guidance under that assumption is a recurring tax that scales linearly with hiring.

The result above says the tax might be payable once. An expert-curated walkthrough is an artifact. It can be replayed, and its authors report that producing it cost the expert less than performing the live version — because drafting is AI-assisted, because it happens on the expert’s schedule, and because it does not have to be repeated for every arrival.

The reframe

If that is right, the question this book has been circling changes shape.

The old question: who pays for the guide? It has no good answer. Somebody gets pulled thinner so somebody else can be walked through, and the organization chooses whom.

The better question: why is the guide’s work still ephemeral?

Put that way, the problem stops looking like a resourcing dilemma and starts looking like a tooling failure. We have spent three years building systems that generate explanations from artifacts, in an industry where the explanations worth having exist only in a small number of heads and are destroyed on delivery. The generated documentation of Chapter 6 captures what the code says. Nobody is capturing what the two people who understand it say when they explain it out loud, which is the highest-value, least-recorded artifact in most engineering organizations.

The code-tour system is one attempt at capturing exactly that, and the same paper reports the outcome that makes it worth attempting: curated tours produced 83 percent comprehension against 57 percent for the AI-only version.^c-021 The capture preserved something the generation did not.

Note also what this does to the autonomy argument. The problem in Chapter 9 was never that a machine was involved. In this design the machine is heavily involved — drafting, quizzing, tracking. What differs is that a human decides what matters and in what order, once, durably, instead of the system inferring it fresh for every newcomer from the artifacts alone.

What I am not claiming

That recorded tours substitute for a person. The same study found that its learners still wanted both, and Chapter 9’s argument about unknown unknowns applies to a recorded tour as much as to a chat window: a tour also cannot answer the question nobody asked. What a curated tour changes is the pull problem — somebody decided the sequence in advance, so the newcomer is pushed through material they would not have known to request.

That is a partial fix to a specific failure. It is not the guide. It is the guide’s judgment, made durable, which is a different and more scalable thing.

In practice

• Make the walkthrough an artifact the first time you give it. Record it, annotate it, put it where the next person will find it. The marginal cost is small and it converts a recurring tax into a fixed one.
• Fund the capture, not the repetition. If the only budget line is “senior engineer time spent onboarding,” you are funding the expensive version.
• Make the invisible work visible in review. Mentoring that does not appear in a performance conversation is being done as a donation, and donations get cut first.
• Ask what your two highest-truck-factor engineers have never been asked to write down. That is the highest-value unrecorded artifact in your organization, and nobody currently owns capturing it.

11

What to Do on Monday

The evidence is too thin to pick a side. It is not too thin to stop flying blind.

Everything so far has been an argument that the industry is measuring the wrong thing and drawing confident conclusions from it. That is a fine position for a book and a useless one for a Monday.

So: what does an engineering leader actually do, given evidence this thin?

The honest constraint shapes the answer. Nobody has run the study that would settle whether AI-mediated onboarding degrades understanding over weeks. Until somebody does, the right posture is not to pick a side but to stop flying blind — to install instruments that would detect the failure if it were happening, and to protect the few things the evidence says are load-bearing. Everything below is cheap. None of it requires believing this book’s thesis.

Stop doing three things

Stop evaluating onboarding by asking how it went. This is the one that matters most and costs nothing. Two independent studies found self-report coming apart from measured performance, and the code-tour researchers state the organizational consequence directly: relying on satisfaction ratings alone systematically undervalues expert curation and defaults an organization to the faster AI-only path.
Your onboarding survey is not a weak instrument. It is a biased one, and the bias funds the cheaper option every quarter.

Stop counting documentation coverage as progress. Coverage measures the thing that recently became free. Ask instead whether anyone decided what a new hire should read first, and in what order — the code-tour result suggests sequencing is where the value sat, since the curated tours were longer than the automated ones and still produced better comprehension in less time.

Stop treating “they didn’t need to bother anyone” as a win. It is the metric that quietly reversed. An onboarding where the new hire interrupted nobody is not efficient; it is uninstrumented. You have no idea whether they were fine or lost, because the mechanism that used to tell you — a colleague’s mild irritation — was removed and nothing replaced it.

Install three instruments

None of these requires a research budget. Each takes under an hour per hire.

The who-would-you-ask probe. At week two and week eight, give the new hire ten realistic situations — the reconciliation job fails at 3am, two documents disagree about the retry policy — and ask who they would go to. Score the names against actual ownership.

This is the highest-value cheap measurement available, because “finding expertise” is one of the three top blockers Google’s ramp-up research identified,
and because it is the specific thing an answer-everything assistant might prevent from forming. If the week-eight scores are no better than week two, the map is not building, whatever else is going well.

The prediction gap. Before any comprehension check, ask the person to predict their own score. The gap between predicted and actual is free to collect and it is the most sensitive detector of the failure mode in this book — the same gap that showed up in the code-tour learners, who rated expert curation as marginally better while scoring twenty-six points higher with it,
and in METR’s developers, who believed in a twenty percent speedup after a measured slowdown.

A widening prediction gap means confident non-understanding, which is precisely what fluent generated explanations produce and what no satisfaction survey will ever surface.

The week-eight explanation. Ask the new hire to walk someone through a system they have been working in. Twenty minutes, no slides. Have a senior engineer who was not their guide score it against a simple rubric.

This is the only measure in this chapter that touches understanding directly, and the reason to run it at week eight rather than week one is the whole argument of Chapter 8: an hour of measurement is the best case for a text interface, and week three is when people report the wheels coming off.

Protect two things

Protect the sequence. The failure of “here is the knowledge base, ask it anything” is not that the assistant is bad — it is that a pull interface can only serve what the newcomer knows to request, and 80 percent of the learners in the code-tour study named uncertainty about what to ask as a barrier.
Somebody has to decide the order. That decision is the thing a machine cannot make for you, because it depends on a model of what a newcomer will get wrong rather than on what the codebase contains.

Concretely: a scheduled sequence of walkthroughs through the four or five systems the person will actually touch, in an order a human chose, with the assistant available for everything in between. Scheduled, not offered — an offer routes through exactly the reluctance-to-interrupt barrier that the same 80 percent named.

Protect the capture, not the repetition. Chapter 10’s finding is the one that makes this affordable: the experts in that study found creating a reusable tour less burdensome than giving live walkthroughs.
Two experts, so hold it loosely — but it points at the right budget line. Fund the recording of the walkthrough once, rather than the performance of it every time somebody joins.

Your two highest-truck-factor engineers hold the most valuable unrecorded artifact in the organization, and in most systems that is a very short list.

Keep the machines

Nothing here says remove the assistant, and it would be a bad trade if it did. The measured benefits are real: engineers using AI daily reached their tenth pull request in 49 days against 91,
and AI-restructured documentation produced higher task success and significantly lower cognitive load than the original.
Withdrawing that to protect something nobody has measured would be trading a known good for a hunch.

The cheapest high-value move available is in fact an AI one: take your existing onboarding documentation and restructure it — segment it into task-based units, strip the redundancy, infer the workflows. That is the intervention with the cleanest supporting evidence in this entire book, and you can start it this week.

What changes is only the accounting. Autonomy stops being a feature you are proud of and becomes a cost you are watching.

How you will know

Three signals, in order of how quickly they arrive.

A new hire who has asked nobody anything by the end of week one is either exceptional or lost, and right now your dashboard reads those identically.

A week-eight who-would-you-ask score no better than the week-two score means the social map is not forming, regardless of how many pull requests have merged.

And a prediction gap that widens between week two and week eight means people are getting more confident faster than they are getting more correct — which is the signature failure of fluent explanation, and the one thing in this book worth setting an alarm for.

In practice

• Delete the satisfaction question from your onboarding survey this week. Not because feedback is worthless, but because that number is currently making a budget decision it is not qualified to make.
• Run the who-would-you-ask probe on your last three hires, retrospectively. One afternoon, no new process, and it tells you whether the problem in this book exists in your building.
• Schedule the walkthroughs instead of offering them, and record them.
• Name the number you would act on before you collect it. Otherwise the measurement becomes a reporting line rather than a decision.

12

A Study Worth Running

An unclaimed question, a protocol for answering it, and the version you can run in one afternoon.

There is an unclaimed question here, and it is unclaimed for an unusual reason. Not because it is unimportant, not because it is hard to state, but because answering it requires waiting eight weeks, and the field has been optimizing for studies that finish in an afternoon.

This chapter is the protocol. It is offered to anyone with access to a real intake cohort — which means most engineering organizations of any size, and very few academics. If you run it, you will hold the only evidence in existence on the question this book has spent eleven chapters failing to settle.

The gap it fills

Stated precisely, from Chapter 8: the studies that measure understanding run in single sessions of around sixty minutes,
and the only measurement in the corpus spanning weeks is a count of merged pull requests. Meanwhile 40 percent of the learners in the deepest of those studies reported needing five months or more to reach productivity.

Sixty minutes of measurement standing in for a process the participants themselves describe in months.

The field’s own systematic review reports that laboratory experiments are the most common evaluation method for onboarding solutions.
That is lab-shaped evidence about a workplace-shaped problem, and it is why the contribution below is available to a practitioner rather than to a professor: the scarce resource is not statistical sophistication, it is access to people actually joining a company.

The method to steal

Take the instrument from museum studies rather than the metaphor.

Museum researchers do not ask visitors how much they learned. They observe what visitors attend to, for how long, in what order, and what they walk past — which is how the field knows that visitors stop at 29 percent of exhibits in a thirteen-minute visit,
a fact no survey would ever have produced, because nobody accurately reports their own inattention.

Onboarding research has done the opposite throughout. It measures outputs, or it asks. Nobody has instrumented a newcomer’s attention across an onboarding the way a museum instruments a visit.

Three arms

• A — Autonomous. Documentation plus an assistant. Human help available on request, not scheduled.
• B — Guided. Expert-curated walkthroughs on a schedule, plus the same documentation.
• C — Hybrid. B’s scheduled sequence plus A’s assistant for everything in between.

Arm C is not padding. The survey evidence puts hybrid onboarding highest on satisfaction at 75 percent,
and a two-arm design that omits it is built to produce the answer the researcher already prefers. Include it or the study is advocacy.

Randomize at the individual level if intake volume allows. It usually will not: alternate by start date and record every deviation.

Duration: eight to twelve weeks. This is the entire point. A four-week study reproduces the existing gap at greater expense.

Four measures, and not one of them a satisfaction score

The design rule that governs everything: no primary outcome may be a satisfaction rating. A study built on perceived helpfulness would reproduce the exact error it exists to detect, for the reasons in Chapter 7.

M1 — Comprehension, longitudinal, blind-scored. At weeks one, three, and eight: a twenty-minute verbal probe. Walk me through what happens when X. Why is this component shaped this way? What breaks if you change Y? Recorded, transcribed, scored against a rubric by an expert blind to condition.

This is the method the code-tour study used, and it is why its result is credible.
The contribution here is running it three times across the real arc instead of once at the end of an hour.

M2 — Transactive memory accuracy. The novel measure, and the cheapest.

At each checkpoint, present ten realistic situations and ask who they would go to. Score the named person against ground truth — actual ownership, or who in fact resolved comparable incidents.

This is the measure worth building the study around. Finding expertise is one of the three top ramp-up blockers,
a README cannot say whose judgment to trust, and an assistant that answers every question competently may prevent the map from ever forming. If the autonomous arm scores well on M1 and badly on M2, that is the finding, and it would be the first direct evidence for the mechanism this book can only argue for.

M3 — Calibration gap. Before each comprehension probe, have the participant predict their own score. Predicted minus actual is the measure. It operationalizes the divergence already observed twice
and tests the specific worry: that fluent generated explanation produces confident non-understanding.

M4 — Attention tracking. Across the full period, with consent: which artifacts were opened, dwell time, what was abandoned, what was returned to, how many queries went to the assistant versus to a person. Reconstruct the museum measures — attracting power and holding power — per artifact.

Likely yield: the onboarding equivalent of “visitors stop at 29 percent of exhibits.” Most of your onboarding corpus is never read, and nobody knows which part.

Secondary, for comparability only: time to tenth pull request. It is the shallow measure this study exists to reach past, and reporting it lets the result be compared against the vendor research.

Sample, and being honest about it

This will be small. For calibration: the code-tour study reached a major software-engineering conference with seven participants.
A well-instrumented longitudinal study at fifteen to twenty-four participants across two or three teams would be a stronger contribution than most of what is currently published — because of the horizon and the objective measures, not despite the n.

Preregister it. Given that the surrounding literature is lab-shaped and self-report-heavy, a preregistered protocol with a stated primary outcome is the cheapest credibility available, and it is what stops the study from quietly becoming a search for the result its author expects.

The theory it should be grounded in

Four constructs would give this work a spine, and I am naming them as pointers rather than as evidence: transactive memory systems for the who-knows-what layer that M2 measures; the illusion of explanatory depth for what M3 is detecting; desirable difficulties for why smooth explanations might optimize the wrong curve; and psychological safety for the question-asking barriers.

None of these appears in this book’s claim ledger. They were not verified against sources the way everything else here was, and anyone building on them should run them through the same check before leaning on them. Saying so is not modesty — it is the difference between a citation and a gesture, which is the subject of Chapter 5.

What would kill it

Attrition, first: eight weeks and three checkpoints across a small n, and two dropouts from one arm ends the comparison. Over-recruit.

Different codebases across arms is the single most likely fatal confound. Match the difficulty or keep the arms within one system.

Contamination is certain — the autonomous arm will talk to people anyway. That is not a flaw to engineer away. It is a finding to measure, and how hard people work to route around the design they were given may be the most interesting number in the study.

And the honest null: all three arms may converge by week eight, with everyone finding their own way to what they needed. That result deserves publishing and should be named in the preregistration as a possible outcome rather than discovered afterward.

The version you can run this month

If the full protocol is more than you can carry, run M2 alone, retrospectively.

Take the engineers who joined in the last six months. Give them the ten who-would-you-ask situations. Score against ground truth. Correlate with how much of their onboarding was assistant-mediated.

One afternoon. No new hires, no control arm, so it establishes correlation only. But if the transactive-memory gap shows up there, the full study is justified — and if it does not, this book’s central worry needs rethinking before anyone spends a quarter on it.

That is the cheapest possible test of the most interesting claim in these pages, and I would rather someone ran it and proved me wrong than that it stayed an argument.

In practice

• Preregister before the first participant starts. It costs an hour and it is the difference between a study and an anecdote with statistics.
• Build M2’s ground truth first. If you cannot establish who actually owns what, you have found something worth knowing before the study begins.
• Run the retrospective version first regardless. It tells you whether the effect is large enough to be worth eight weeks.
• Publish the null. The field’s problem is not a shortage of positive findings about onboarding tools. It is a shortage of measurements at the horizon where onboarding actually happens.