An RL environment is a scorer in a container, and RL turns that scorer into weights anyone with an API key can rent. A vertical company that sells one sells the scarce thing its profits rest on, unless what it ships is a complement it wants cheap. The test is what fraction of your scarcity fits in the container.
There are two versions of the next five years being fought over right now. In one, a few general models do everything and vertical companies are user interfaces on top of them. In the other, vertical companies own their verticals and the general model is a rented input, like the cloud bill.
The labs are committed to the first version. They have spent everything on being general, and a general company can't credibly turn into fifty vertical companies: different sales motion, different liability, different capital. But a general model gets no vertical for free. The transfer research is consistent on this: RL gains move between structured domains like math and code and don't carry over to knowledge-heavy domains like law, finance, and medicine, and domains the base model barely saw in pretraining only improve with in-domain RL. To get a vertical, a lab needs a scorer for that vertical. The only people who have one are the people who would otherwise own the vertical.
So the labs are buying. Anthropic reportedly discussed spending over $1 billion on RL environments in a year. The “there's a ton of money in RL environments” story is how you recruit a supply side, and a crowded vendor field also pulls in investors who subsidize the cost. What I want to work through is why the sellers are financing the version of the future that kills them, and whether the labs' plan can work at all. I don't think it's settled.
An RL environment is three things: a world (the state and what actions do to it), a distribution of tasks, and a scorer that grades each attempt. In practice it is delivered as a Docker container. The lab pays for all three, and the scorer carries most of the value.
You can see that in where the labor goes. When Epoch interviewed 18 people across environment startups and labs, the quality criterion everyone named first was robustness against reward hacking: a model will search up the solution, check out a future commit, or find any crack in the grader, and sound graders take many iterations. A world is engineering. A scorer is the codified answer to “what does correct look like here,” which is the thing a vertical company spent years learning.
An environment is three parts; the scorer carries the value
The world and the tasks are rebuildable by anyone with engineers. The scorer is the part only a domain owner can write, and the part the training run exists to extract.
Profits are rent on scarcity. A vertical company's scarce thing is a capability: partly tacit (its people know what correct looks like), partly a private scorer. Selling an environment pushes that capability through a chain where every arrow strips excludability.
1. You codify it. To sell the scorer you turn “what correct means here” into a program that grades any attempt. That is the first loss. Tacit knowledge can't be copied; a program in a container can. If your moat fits in a Docker container, it isn't one.
2. RL moves it into weights, more completely than data ever did. Data labeling sold finite examples a model could memorize. A scorer is a function: it grades unlimited new attempts, so the model learns the policy, not the samples. Policy gradient pushes the model toward whatever the scorer rewards until it passes; practitioners told Epoch they retire a task once it hits around a 70% pass rate, because what it had to teach is now in the weights. And RL-learned skill carries to variants you never showed it (SFT memorizes, RL generalizes; RL's Razor on why online RL also forgets less). The model now does your job on inputs it has never seen.
3. The scorer is a product at inference, not just a training signal. Sample N attempts, keep what the scorer passes. Whoever holds your scorer can do your job today with a model they already have. This is verifier's law pointed at you: the ease of training an AI on a task is proportional to how verifiable it is, and you just made your vertical verifiable for them.
4. The weights are public. A general model is sold per token to everyone: your competitors, your customers, and the lab's own app. The capability went from one holder to anyone with an API key.
5. It propagates. Successful trajectories are fed back into mid-training, so it's in every future base. It gets distilled into small models. And the same vendors sell to the open-weight labs; SemiAnalysis believes purchased environments played a big part in Kimi K2 Thinking and GLM-4.6. “If the labs have it, everyone has it” isn't a slogan. It's the pipeline.
Every arrow strips excludability; the container is the one-way door
At the end of the chain the capability you refined is abundant, and the rent on it is zero.
The lab pays you the value of the capability to one buyer and then sells it to everyone. That spread is the whole business, and the numbers show it.
And the scale is wrong. Anthropic's reported $1B+ on environments sits against OpenAI's projected ~$19B in R&D compute for 2026. The entire supplier layer is a rounding error next to the buyer's compute bill, which is the position you want to be on the other side of.
The magnitude point is the simple one. An environment sale is a slice of a lab's input budget. What you gave up for it is the rent on a vertical. Cash doesn't buy scarcity back.
After the sale your workflow, your relationships, and your interface compete with every other interface on the same model, and with the lab's own app.
Imagine Harvey packaging its internal evaluation set, the private version of BigLaw Bench, together with its law-firm task distribution, as an environment for the labs. Within a model generation there is no Harvey, just Harvey-shaped UIs on GPT. Same for Rogo in finance, same for Glean in enterprise search. Nothing about their value was outside the container.
Palantir is the contrast. A lab can buy every bit of Palantir that fits in a container and still not have Palantir, because the deployments, the integrations into government systems, and the standing with the customer never fit in one.
OpenEvidence is the closer contrast, because from the outside it looks like the chatbots it competes with. But its answers draw on journals it licenses in full (NEJM and JAMA, among others), about two-thirds of US physicians use it, by the company's count, and it says it has logged hundreds of millions of consultations from verified doctors. A lab could buy its scorer tomorrow and still have none of the licenses, the doctors, or that stream. That's the test the next section is about.
The useful question for any company is: what fraction of your scarcity fits in a Docker container?
Bits fit. A scorer, a compiler, a file format, a corpus, a rulebook, a benchmark. All of it can be shipped as an image, trained on, and absorbed into a general model. Someone will sell it even if you don't, because the labs will pay and a competitor or a contractor can rebuild most of it.
The world doesn't fit. A live stream of what actually happened (what the customer actually paid, what the regulator actually accepted, what production traffic actually did). The standing to be sued when the answer is wrong. The balance sheet that takes the risk. The seat in the flow that everyone else has to route through. None of these can be put in a container, which means none of them can be sold as an environment, which means none of them can be trained away.
What fits in the container can be bought; what doesn't is the moat
Clay Christensen called this the law of conservation of attractive profits: when one layer of a value chain commoditizes, the profit moves to the layer next to it that is still hard. The labs can absorb everything that is bits. They can't absorb the world, which is why the companies they buy from are the ones whose moat was entirely bits.
atopile is a good product: a language for describing circuit boards as code, with a compiler and a constraint solver behind it. It is also a company whose scarcity was all bits.
They built EEBench: 13 circuit-design tasks graded by SPICE simulation across real part tolerances and cost, with the frontier models on a public leaderboard. They also pitch the benchmark as the base for RL environments in electronics, with simulation as the reward. Their enterprise page says the quiet part about the leaderboard: “Each model that climbs it gets better at your boards.”
The bet is “be the format”: if the models learn to design in atopile, the world writes atopile. But the hard part was the physics-graded scorer, not the syntax, and the skill a model learns from it is circuit design, not atopile. The same week EEBench went up, OpenAI launched GPT-6 Astra with a clip of it laying out a board in KiCad, the incumbent format atopile was trying to replace (Hackaday covered both).
I'm not claiming the labs have trained on atopile's environments; I can't know that. The point is structural. A format is a coordination anchor only once the world has standardized on it, and the biggest lab's demo went straight to the incumbent. A company that could have owned hardware design is building the scorer that makes general models good at hardware design, and every lab that trains on it takes some of the scarcity with it.
Take a company that can price any medical claim exactly. A lab would pay well for that scorer: the general models can't do it, and the transfer research says they won't pick it up from code.
Suppose the company packages it as an environment. The sale is seven figures, more with exclusivity. Within a model generation every billing vendor and every payer has the capability at API price. The company banked a few million dollars and gave up the vertical.
Nobody would make that trade on purpose. People make it because the cheque is real and the loss is a counterfactual.
None of this means “never sell an environment.” The line to hold onto is Joel Spolsky's from 2002: smart companies try to commoditize their products' complements. When something that goes with your product gets cheap, demand for your product goes up. His cleanest case is Microsoft, which licensed DOS to IBM non-exclusively so it could license the same thing to every company cloning the PC. The PC became a commodity, and the operating system each one needed became the asset.
So the question isn't sell or don't. It's whether the thing in the container is your moat or your complement. If the scorer you'd ship covers work that commoditizes anyway, an answer key anyone could rebuild from public documents in six months, then selling it early, to every lab that will pay, is Spolsky's move. You bank cash and lab relationships on a depreciating asset, the general models get better at the part of the work you never wanted to be scarce, and every improvement there raises the value of the part only you have. If the scorer is the only scarce thing you own, selling it is atopile.
Spolsky's warning case is Sun, which pushed to commoditize both software and hardware at once and was left with nowhere to sit when the music stopped. The container test is how you find out which chair is yours.
I'm bullish on environments. I'm not bearish on RL; I'm bearish on a kind of RL. The kind I like has three properties.
Bearish on black-box RL, the LLM-judged, rubric-graded, “in the middle” kind, because the signal is the noise. Bearish on selling scorers to labs, because the lab never gets the context, the liability, or the capital underneath the capability. The model ends up with the top of the funnel and nothing under it.
So: build the white-box model of the domain, then run RL on top of it. Karpathy's version is that he's bullish on environments and bearish on reinforcement learning specifically. Mine is narrower. RL is fine. It just has to be on a scorer you own.
I don't have a verdict. Both sides have evidence.
For the labs: purchased environments demonstrably install skills. When a model release advertises better Excel or PowerPoint, SemiAnalysis's read is that it's directly downstream of purchased RL environments. On OpenAI's GDPval, expert-graded work across 44 occupations, the best model was already tied with or preferred over human experts about 71% of the time by January. And the supply side is showing up: more than 35 environment companies, with the human-data incumbents pivoting in behind them.
Against: environments are consumables, and verticals keep moving. A general model's RL budget is a weighted sum across every domain, and the optimum for the sum isn't the optimum for your term; the multi-domain curriculum work finds some domains' gradients align weakly or even negatively with the rest. A vertical company spends all of its RL on one term: Cursor scaled RL 20x on the same base from Composer 1 to 1.5, until post-training compute passed what went into pretraining. And the companies doing best in verticals are vertical ones, holding the parts that don't fit in a container.
My read, stated as a read: verticals win their verticals, the labs win the general layer, and the environment trade is the labs trying to change that outcome, with the sellers paying for it.