Essay · Innovation & the economics of discovery

The Price of Being Wrong

Who in your company is allowed to be wrong, and what does it cost them? The world spends three trillion dollars a year trying to find out what it is wrong about. Almost none of that money is spent on the person who has to say so out loud.

A portfolio essay

Who in your company is allowed to be wrong?

Not wrong in the safe way. Not the retrospective where everyone agrees the market shifted. I mean a person, with a name, saying out loud that the thing they argued for last quarter has not worked. Who can do that? What does it cost them?

In most places I have worked the honest answer is that almost nobody can, and the people who can are the ones who were never really on the hook. That is worth sitting with, because we spend an enormous amount of money every year on finding out we are wrong, and almost none of it on making it survivable for the person who has to say so.

I spent some time worrying about one version of this. It was a small conflict, about an evaluation suite for a security product, and I have been unable to stop thinking about it since.

Two options. One was frictionless. Nothing to maintain, nothing to own, plug it in and move on. The other was ours to build and ours to keep repairing. It taxed every new product line we brought in. It meant a new engineer had to learn a codebase before they could ship anything.

I chose the expensive one. In a security product, trust matters more than speed, and I wanted to own the definition of what counted as a good fix rather than rent it. I would choose it again.

But I could not explain that choice in any language a finance person would accept. The cost of the expensive option was a line item. The cost of the cheap one was that in eighteen months somebody would be sitting in a room unable to prove their own product worked, and there is no column for that.

So I went looking for the column.

What the world spends on finding out

In 2024 the world spent an estimated $2.87 trillion on research and development. That is nearly three times what it spent, in real terms, twenty-five years earlier. Measured in current dollars rather than constant ones, the OECD puts it closer to $3.8 trillion. Same mountain. Different light. Either number is larger than the GDP of every country on Earth except two. Somewhere between a fortieth and a thirtieth of everything we make in a year, we make in order to find out something we do not yet know.

And nobody can tell you what it buys. Boards approve research budgets the way governments approve defense budgets. As an article of faith. Priced by benchmark, not by expected return.

Nicholas Bloom and his co-authors spent a decade measuring what the spending actually produces, and the answer was unwelcome. Effort is rising sharply almost everywhere they looked. Productivity per researcher is falling just as sharply. Keeping Moore’s Law on schedule now takes more than eighteen times the people it took in the early 1970s. Across a panel of individual firms, research productivity was falling in more than 85 percent of the sample. At roughly 10 percent a year.

[ Eighteen times the people. For the same doubling. That is not a slowdown. That is a different industry wearing the old one’s clothes. ]

The comfortable explanations do not survive this. Have we stopped trying? No. Effort is up. Have we run out of money? Also no. Global research intensity climbed from under 1.5 percent of world output in 2000 to nearly 2 percent today. So, we are trying harder, paying more, and getting less. Why?

Is the low-hanging fruit gone? Partly. But watch what that sentence does. It tells you that productivity is falling. It does not tell you why an extra dollar buys less discovery. And it does not explain why a wartime aircraft division and a patent-poor telephone monopoly produced so much of the twentieth century’s best thinking on what were, by our standards, small sums.

So, what is a trillion dollars actually for? If it is not reliably buying us ideas, what is it buying? And why does nobody seem able to say?

Follow the money

Start with where the money piles up. Concentration is a clue.

Five top tier American technology companies spent $229.1 billion on research and development in the twelve months to March 2024. More than the next five largest corporate spenders put together. And that is now the smaller of the two numbers that matter.

Add what the same companies are committing to capital expenditure. The data centers. The chips. The power contracts to train and run frontier models. The largest hyperscalers are expected to spend on the order of $725 billion in 2026, up roughly 77 percent on an already record 2025. One investment bank puts the combined figure at $5.3 trillion between 2025 and 2030.

[ Investment-bank estimates of things two years out. Hold these loosely. The direction is the point, not the decimal. ]

So is that what we are buying? It is tempting to look at a number that size and call it the machinery for finding out which ideas are any good.

Mostly it is not. The large majority is inference-serving capacity. Chips to deliver a shipped product to paying customers. That is production capital, no different in kind from a factory or a fleet of trucks. A data center does not become a peer reviewer because a model happens to run on it.

Strip the serving capacity out, though, and something real is left. The training runs. The evaluation harnesses. The red-team budgets. The synthetic-data pipelines. That residual is the part that tests rather than serves, and it points at the same fact the corporate numbers do. Across the OECD the business sector now accounts for 73 percent of all research spending, up from 67 percent in 2010. And inside corporate budgets the overwhelming majority goes to development, not to basic or applied research. Taking an idea, somebody already had and making it shippable, verified, real.

Discovery is the smallest line item in the discovery budget.

Which was my argument in miniature. Nobody on that product was short of ideas about how to fix a vulnerability. What we were short of was a way to know, before a customer knew, whether a fix was any good. The evaluation suite was not overhead on the real work. It was the real work, and it kept getting costed as overhead because that is the column it fits in.

Two labs

The innovation literature is a parade of famous institutions and I am going to use two of them, because the parade is the problem. Every one of them gets written up as a case study in creativity. Culture, autonomy, brilliant hires. None of that is wrong. It just hides a better question. How cheaply, and how fast, could each of them find out that an idea was wrong?

Bell Labs ran for decades on a fixed slice of every American’s phone bill, insulated from quarterly earnings. More than 28,000 patents. Work recognized by ten Nobel Prizes. The transistor. Information theory. Unix. Its horizon was measured in decades, so it could afford to test any one hypothesis slowly. But once a hypothesis was ready, the peer challenge down the hall was dense and literate enough that a wrong idea rarely survived the week.

Slow, and cheap. Cheap because the reviewers were free, permanent and world-class. Bell Labs owned its evaluation suite. It had simply built it out of people.

[ Which is the version of my argument I wish I had had at the time. The expensive option was not the tooling. It was refusing to outsource the question of what counts as good. ]

Then there is Xerox PARC, and this is where my tidy version of the argument fell over.

PARC built the graphical user interface, the mouse, Ethernet and laser printing, years or decades ahead of the market. Xerox commercialized almost none of it as well as the companies that copied it. The convenient reading is that PARC could not cheaply find out whether a customer would pay.

It quietly redefines innovation as money Xerox captured. By any honest accounting PARC innovated spectacularly. The world runs on what it made.

What PARC lacked was not a way of finding out it was wrong. It had that. What it lacked was any way to keep what it found. David Teece called this appropriability. The value flows to whoever controls manufacturing, distribution and brand, and PARC controlled none of them.

So, my one-variable theory had two variables in it.

Cheap, legible testing is not what makes a lab great. It is a necessary condition. Something every winner had, and something that on its own guarantees nothing at all. For every lab we remember there were dozens with dense review and cheap tests that produced nothing anyone can name. Cheap falsification is common. Fame is rare.

Table 1 · Innovation labs compared

Why nobody wants to run the test

The individual-level research gets summarized as a call to think outside the box. The findings are less inspirational and a lot more mechanical.

Generating many candidate solutions is a different skill from choosing correctly among them. Most processes reward the second while claiming to want the first.

[ Every idea-generation workshop I have sat in was a convergence exercise in a divergence costume. The ideas were on the wall by eleven. The afternoon was spent killing them politely. ]

There is a structural reason too. As a field’s knowledge deepens, the training needed to reach its frontier lengthens. Teams grow, because nobody can hold the whole field in their head. And the easy cross-disciplinary insight gets rarer, because the entry cost to either field keeps rising. The Renaissance man is not extinct because we got less curious. He is extinct because the distance between informed layperson and frontier researcher now takes a career to close.

Which matters here in a specific way. As the burden of knowledge rises, the cost of checking whether an idea is even coherent rises with it. Fewer people are qualified to check. Checking takes them longer. And those people have other work, and a manager, and a review cycle.

The bottleneck is not intellectual. It is somebody’s Thursday.

Table 2 · Psychological mechanisms and their organizational analogue

Why the test is never clean

I want to stop and confront the hole underneath all of this, because anyone who knows the terrain has already spotted it.

This essay leans on falsification. Popper’s idea that we make progress by ruling wrong things out. But Popper’s clean picture was dismantled decades ago.

No hypothesis can be tested on its own. Every experiment rides on a bundle of assumptions, so when a prediction fails, all you learn is that something in the bundle is wrong. Not what. And Kuhn argued that theories rarely die by falsification at all. They persist until the people holding them retire. Science advances, in Planck’s paraphrase, one funeral at a time.

If that is how discovery works, a model that treats falsification as a clean purchasable step is describing a world that does not exist. So the model has to survive the objection or be thrown away.

It survives, by absorbing it. Ask why finding out you are wrong is expensive, and the philosophy hands you the answer. To isolate which assumption failed you need many people, a lot of apparatus, and repeated independent tests. The tangle is not an argument against the cost. It is the source of the cost.

And Kuhn gives the sharpest case rather than the refutation. Sometimes the assumption that has to be rejected is the one held by the person who owns the field. Then the cost of disproving it is measured in career lengths. Not in equipment. In somebody’s standing, somebody’s tenure, somebody’s remaining working years.

The most expensive falsification in the history of science is a funeral.

Every one of the auxiliary assumptions in my argument about that evaluation suite was held by a person. What counts as a good fix. What counts as a regression. Whether a false positive is a bug or a taste. Those were not technical questions we had failed to answer. They were questions nobody could answer without telling a colleague they were wrong about their own product.

So the cost is irreducible, and it is social rather than technical. That is the part no budget line has ever contained.

Security is not a metaphor here

It would be easy to treat threat modeling, red teaming and chaos engineering as a colorful analogy for what research labs do. They are not an analogy. They are the same activity under a different name. The deliberate manufacture of being wrong, on purpose, early, in private.

A red team’s whole job is to find, cheaply and on your schedule, the ways your system is already wrong. Before an adversary finds them expensively on theirs. Zero trust assumes every credential may be compromised and forces continuous re-verification, which is the same move as refusing to accept a hypothesis because it passed one test. Chaos engineering, deliberately breaking production to see what fails, forces engineers to meet the ways a system behaves outside its familiar range instead of waiting for a customer to find out at 2 a.m.

So what does security engineering have that most innovation processes do not? An honest price on being wrong, and a culture where finding it is a win rather than an accusation. A vulnerability our own red team finds costs an afternoon. The same one found by an attacker costs a breach disclosure, a regulatory inquiry and a share price.

Notice what that does to the person who finds it. On a red team, being the one who proves the thing is broken is the job. Everywhere else in the same company, it is a political act. Same finding. Same afternoon. Completely different cost to the human being holding it.

It is nearly always cheaper to disprove ourselves than to be disproven by the world. Nearly. That qualifier is about to matter.

What the labs are actually buying

This is the lens that makes the buildout stop looking merely enormous and start looking legible.

Frontier labs spend a conspicuous share of their effort on activities that produce no new capability at all. Red-teaming their own models. Building evaluation suites. Running reinforcement learning from human feedback. Generating synthetic data to probe edge cases no real dataset covers. None of it is optional. It is what a model needs before anyone can trust its outputs, and its cost has grown comparable to the cost of the model.

[ This is the part I did not have to go looking for. Same suite. Same maintenance tax. Same argument about whether owning the test is worth what owning it costs. Just with more zeroes and better press. ]

Reasoning models make it almost literal. The system generates a candidate answer, checks it against constraints, revises, checks again. That is an internal loop for finding out it is wrong, bought with compute at the moment of use. Which is a genuinely new thing for a research budget to buy. Not researchers. Not equipment. The disconfirmation step itself, unbundled and metered.

Which brings me to the question I least want to get wrong. Can a model discover anything? Or can it only recombine what it was shown?

The comfortable answer is that machines search a space humans define. That was true until recently and is now too tidy. In the last few years, systems have found matrix-multiplication algorithms faster than the one that stood since 1969, sorting routines now shipping inside the C++ standard library, and a genuinely new result on a problem in combinatorics that no human had reached. These are not retrievals.

And yet the line did not vanish. It moved.

In every one of those cases a person still specified the objective and the search space. Minimize multiplications. Maximize the set. The machine then searched with a divergence no human could match. So the frontier is migrating. From generate versus test, toward choose the question versus search the answer. Protein folding did not become important because a model said so. A generation of biologists decided that first.

Is that a temporary limit or a durable division of labor? I do not know. Perhaps nobody does yet. Either way it fits the pattern. What is scarce is not the search. It is the judgment about which space is worth searching, and the willingness to be the person who chose wrong.

Table 3 · What AI labs are actually buying

The last row is the honest one.

Cheaper is not better, and I should have known

Here is where the argument I started with stops being true.

Consider Katalin Karikó. She spent decades on messenger RNA in exactly the conditions this essay is supposed to disdain. Chronically underfunded. Unfashionable. Slow. Largely unsupervised. The University of Pennsylvania demoted her in 1995 when the grants would not come. The 2005 paper that cracked the immune-response problem was rejected by Nature and by Science before it found a home, and it took over a decade to be believed.

By any cheap-and-fast scorecard, she is a failure. She is the mRNA vaccines.

Somebody priced her work. The price they arrived at was her career. They were wrong, and the cost of their being wrong was paid entirely by her, for twenty-eight years, and by everyone who did not have a vaccine in the meantime.

Or take the Manhattan Project and Apollo. Enormous, bureaucratic, slow, expensive, rule-bound. Roughly $30 billion and $260 billion in today’s money. And among the most concentrated bursts of invention ever recorded. If cheap disconfirmation were the whole story, none of these should exist.

So two admissions, and the second one hurts.

First, sometimes the binding constraint is not the price of the test at all. It is the patience to survive a long generative phase before any test is worth running.

Second, the relationship between the cost of being wrong and the rate of discovery is not a straight line. Make it cheap enough and it stops working.

The replication crisis is the proof. When a significance threshold made confirmation cheap and mechanical, the result was not more knowledge. It was less. Roughly a third of 100 psychology studies reproduced. Six of 53 landmark cancer papers. Falsification made too cheap does not sharpen a field. It fills it with findings nobody really tested, and with careers built on them, and with graduate students who spent three years failing to replicate something that was never there.

So the naive version is false, and we know it is false because reality ran the experiment for us.

What replaces it is not a retreat. There is an optimum. Discovery is maximized not where the cost of being wrong is lowest, but where it is priced correctly. High enough that a confirmed result means something. Low enough that testing actually happens, and that the person running the test can afford the answer.

Call the target Minimum Viable Doubt. The least expensive test that still produces a result you would stake a decision on. Below it lies the replication crisis and the demo that certifies nothing. Above it lies bureaucratic freeze. The pathology at both ends is the same. A cost of being wrong that has come unhooked from the value of what is being decided.

Which is also why the security engineer’s arithmetic has a limit. Pharmaceutical regulation sits exactly on that edge. The randomized controlled trial is one of the most rigorous instruments ever built for finding out you are wrong, and the same apparatus that makes a drug’s claim trustworthy is why a new drug now costs, on the most-cited estimate, around $2.6 billion and a decade. The rule and the tax are the same rule. You cannot lower that cost without lowering the trustworthiness that was the entire point.

[ Critics reasonably note that about half of that $2.6 billion is capitalized opportunity cost rather than cash out the door, and that the underlying data are confidential. Halve it if you like. The shape does not move. ]

The goal was never cheap. It was calibrated.

Table 4 · When rules lower the cost of being wrong, and when they raise it

The trial appears in both columns on purpose. It is the cleanest evidence that calibration, not cheapness, is the variable that matters.

Two quantities, not one

Put all of it together and one model falls out.

What an organization can discover is not a function of how many ideas it has, nor of how cheaply it can disprove them. It is a function of how well the cost of being wrong is matched to the value of the decision at stake. And, separately, of whether it can keep what survives the test.

The first quantity is the cost of finding out you are wrong. The time and money to move a claim from untested to a result you would act on. It is not lower is better. It has a target, and the failure modes are symmetric. And it is measurable, at least in proxy. How long from proposal to kill-or-scale. How much per decisive experiment. How often a confirmed result later reverses.

Any team can ask those three questions on a Monday morning. Most do not, and I think the reason is not that the questions are hard. It is that all three of them have somebody’s name attached to the answer.

The second quantity is whether you can keep what you find. Whether you control the manufacturing, the distribution, the brand. This is the axis PARC fell off. The first governs discovery. The second governs capture. They are independent, which is why an organization can be brilliant at one and bankrupt at the other.

Plot one against the other and the famous cases sort themselves without special pleading.

Table 5 · Discovery versus capture

Notice what this fixes. PARC is no longer a failure. It is a gift to the commons. A triumph of discovery and a catastrophe of capture, which is exactly what the history shows. Apollo is no longer a freeze. It is expensive certainty, bought at a price only a superpower could pay.

And the map stops pretending to predict success. It predicts two different things. Whether we can discover, and whether we can keep. It refuses to collapse them into one. Perhaps that is all a good model ever does. It stops us asking one question when we were really asking two.

The AI labs are racing toward cheap, fast disconfirmation at a speed no prior institution could attempt. But the same map warns them. A lab can drive the cost of finding out it is wrong below Minimum Viable Doubt. Ship benchmarks that certify nothing. Mistake a passed evaluation for a tested claim. And land in the replication crisis’s quadrant without ever leaving the office.

[ Which is the version of this failure I have watched up close. It does not feel like failure while it is happening. It feels like velocity. ]

Back to the question I started with

Why can nobody reliably manufacture discovery, despite spending more on it than ever? The answer is less romantic than ideas being hard.

Ideas were rarely the scarce thing. What is scarce, and getting scarcer as the dollars pour in, is correctly priced proof of which ideas are worth keeping. Cheap enough to actually run. Dear enough to actually mean something. Legible enough that the answer survives the person who found it.

The trillions we spend are mostly a bet on the infrastructure of finding out. Laboratories, red teams, checklists, evaluation suites, chip fabs that test ten thousand hypotheses before breakfast. The return on that bet was never about how many ideas the infrastructure generated. It was, quietly, the whole time, about how honestly it priced the cost of being wrong.

But there are two prices, and only one of them is on the invoice.

There is what it costs the organization to run the test. And there is what it costs the person who has to carry the answer into a room and say it. A demotion in 1995. A paper rejected twice. A colleague told that the thing they built does not work. Three years of a graduate student’s life spent failing to find something that was never there. Every number in this essay is the first price. Almost none of it is the second.

And I think the second one is the binding constraint. Not compute. Not headcount. Not the burden of knowledge, though that is real. The reason organizations do not find out they are wrong is that in most of them, finding out is a thing that happens to somebody.

So the question on the board slide is wrong. It is not how do we generate more ideas. That machinery is closer to abundant than scarce. It is not even how do we test them faster, because faster, past a point, is how a field lies to itself.

For a given decision, what is the right price of doubt? And who is allowed to pay it?

I still think I made the right call on that evaluation suite. But I notice I am more confident about the decision than I am about the price I paid for it, and I notice I am not the one who paid most of it. The engineers who had to learn that codebase did. Perhaps those are not the same kind of certainty. Perhaps they never were.

Sources

Every figure and claim above comes from here. Numbers are as the source reports them. Where an estimate is contested I have said so in the text rather than quietly picking the flattering one.

• WIPO, “End of Year Edition — Despite the Odds, Global R&D Spending Grew Again in 2024,” Global Innovation Index blog, 2025. Figure in constant 2015 PPP terms; R&D intensity from the same source.
• Bloom, Jones, Van Reenen, and Webb, “Are Ideas Getting Harder to Find?” American Economic Review 110, no. 4 (2020): 1104–1144; NBER Working Paper No. 23782. A scholarly rebuttal argues part of the measured decline reflects dilution of the researcher pool by less-productive marginal entrants rather than a fundamental scarcity of ideas; the direction of the finding is not seriously disputed.
• Trendline, “Big Tech’s Big R&D Bill,” 2024, compiling trailing-twelve-month 10-Q filings to March 2024. Note that Amazon reports R&D as “technology and content,” a broader line than a strict R&D definition.
• Goldman Sachs Research, “Why AI Companies May Invest More Than $500 Billion in 2026” and “Tracking Trillions,” 2026. Figures are investment-bank estimates as of early 2026 and are volatile.
• OECD, March 2026 statistical release.
• NSF/NCSES Business Enterprise R&D (BERD) tables; development is consistently the largest component of U.S. business R&D, with basic and applied research a minority. Summarized in UIDP, “R&D in Review: Global Investment Trends.”
• Nokia Bell Labs, “Awards” (official count: 10 Nobel Prizes, 5 Turing Awards) and “History”; patent count “more than 28,000 since 1925.” Some sources count 9–11 Nobel-recognized researchers depending on method.
• David J. Teece, “Profiting from Technological Innovation,” Research Policy 15, no. 6 (1986): 285–305.
• Benjamin F. Jones, “The Burden of Knowledge and the ‘Death of the Renaissance Man,’” summarized in Nicholas Decker, “Are Ideas Getting Harder to Find?” 2025.
• “Underdetermination of Scientific Theory,” Stanford Encyclopedia of Philosophy (holist / Duhem–Quine underdetermination). Kuhn: The Structure of Scientific Revolutions (1962).
• Fawzi et al., “Discovering Faster Matrix Multiplication Algorithms with Reinforcement Learning,” Nature 610 (5 Oct 2022): 47–53.
• Mankowitz et al., “Faster Sorting Algorithms Discovered Using Deep Reinforcement Learning,” Nature 618 (7 Jun 2023): 257–263.
• Romera-Paredes et al., “Mathematical Discoveries from Program Search with Large Language Models,” Nature 625 (2023/2024): 468–475.
• Penn Today and Forbes coverage of the 2023 Nobel Prize in Physiology or Medicine; Karikó and Weissman, “Suppression of RNA Recognition by Toll-like Receptors,” Immunity (2005).
• Manhattan Project ~$1.9–2.0B (1940s) ≈ ~$30B today (Brookings, NPS). Apollo $25.8B nominal (1960–73) ≈ ~$260–310B today (Dreier, Space Policy, 2022; The Planetary Society).
• Open Science Collaboration, “Estimating the Reproducibility of Psychological Science,” Science 349 (2015); Begley and Ellis, “Raise Standards for Preclinical Cancer Research,” Nature 483 (2012): 531–533. Begley–Ellis has been critiqued for a small non-random sample; the later Reproducibility Project: Cancer Biology (eLife, 2021) corroborates the direction with substantially reduced effect sizes.
• DiMasi, Grabowski, and Hansen, “Innovation in the Pharmaceutical Industry: New Estimates of R&D Costs,” Journal of Health Economics 47 (2016): 20–33. Critics (Public Citizen, KEI) note ~half the $2.6B is capitalized opportunity cost and that the underlying data are confidential.
• Toyota Motor Corporation, “Toyota Production System: Vision & Philosophy”; The Systems Thinker, “Pulling the Andon Cord.”