This (mostly handwritten!) piece collects my thoughts on the data market after a short four-month stint in it. During that time, I had a front-row seat to the release of DeepSWE at Datacurve, and worked on similar projects. This is the first of a three-part series on my learnings from the industry.
Setting the scene
Pretraining and post-training rely on structurally different kinds of data. Historically, web-scale datasets such as Common Pile v0.1, FineWeb, and RefinedWeb have primarily served pretraining in the model development lifecycle; their value comes from breadth and scale. Post-training is different: the goal is to elicit specific capabilities through more signal-dense examples and environments, so feeding the same corpus back into the model wholesale would be a relatively inefficient use of compute.
Even before the current LLM era, data was the foundation of the ML stack. Before upper layers such as model architecture or optimization can be meaningfully explored, the training data must first contain sufficient signal for the model to learn from. Improvements higher up the stack can then extract that signal much more effectively. Models can recombine what they have learned in novel ways, but insights that require distinctions absent from their training distribution are unlikely to emerge reliably.
Having merely the right signal is insufficient, there also needs to be enough of it. The Chinchilla scaling laws made this relationship between model capacity and training-data scale explicit for language models. Scaling laws allow us to solve for how much data to use.
Now that the importance of scale has been established, how do we curate the data itself? Some of the most consequential advances in ML have come from the creation of the right data artifact. ImageNet made computer vision plausible by labelling millions of images. CommonPile and FineWeb determine the distributions that modern language models encounter during pretraining. We have go-to benchmarks like MMLU which defined early-era LLM progress. While these datasets serve very different purposes, their inherent value comes from a measured curation that enables either a model’s development or provides an assessment of a model’s capabilities. By extension, huge amounts of data need to be cleaned and filtered before becoming useful training fuel. The same problem of purposeful curation becomes substantially harder once the thing being curated is no longer a static dataset, but an environment in which an agent acts.
Much of the industry’s disdain for data has come from the very manual, sometimes underpaid process of data labelling and data cleaning. Yet some of the largest gains in language modelling have come from higher quality data while keeping model size, architecture and training token count fixed1. Procuring and labelling data is often dismissed as “dirty work”. Unfortunately, said dirty work is the most under-valued and de-glamorised aspect of AI.
The form this curation takes, however, is changing. What is more interesting (and the result of where I’ve spent my past 4 months) is how the object we call data has changed in the current post-training era. Instead of static corpora of text / multi-modal data, increasingly valuable data points can take the form of executable environments in which language-model agents act and receive feedback.
Yet the underlying problem has barely changed. The bar for how one determines the quality of data in this new paradigm has only gotten more challenging; I posit that this old problem of data curation has increasingly become a problem of verification, which is apparent in the scaling of RL environments.
Shifting Paradigms
Investor attention has followed this shift. By July 2026, more than 50 companies were selling training data and RL environments to frontier labs, collectively generating roughly ~$8.5B in annual revenue. What, then, are these environments in practice, and why have they become so valuable?
For the purposes of this piece, I treat a single RL environment as the atomic post-training datapoint2. Unlike a static text or multi-modal sample, this datapoint is executable: it defines a task world (“environment”), the actions that a LLM agent can execute, and an evaluator that determines how well it performed.
Running a solver agent against this datapoint produces a trajectory3: the sequence of messages and actions (tool calls) an agent takes while attempting to solve this task. The evaluator then grades the resulting outcome and returns some form of reward. The same environment can therefore generate multiple trajectories across repeated solver agent configurations (or even on the same configuration).
As language models have progressed from generating text to code, making tool calls and completing multi-step tasks, the field has borrowed heavily from classical RL terminology. In agentic RL for LLMs, Harbor provides one way to package such RL-environments into the final ingestible form for delivery to clients (which would be frontier LLM labs).
Concretely, what would an “RL environment” really look like, in code? Drawing from Harbor documentation, any task would consist of a task.toml, a prompt.md4, a Dockerfile or equivalent environment setup, a solution script to prove that this task is solvable, and an evaluator / verifier that grades the solver’s output. In this case, you could collapse the definition of a solver agent into any coding agent (Claude Code with some Anthropic model, Codex with some form of GPT model, etc.).
This solver agent then attempts the task that it’s given (in the prompt.md) using the tools that are installed in the Dockerfile, much as a human would use available libraries. In doing so, one must ensure that this solver agent does not have access to the solution script, as well as the evaluator / verifier – if not such a task would be too trivial / easy for the agent (some would consider this to be a case of “reward hacking”).
To understand why these environments matter for training, it is useful to place them in the broader history of post-training. In Supervised Fine Tuning (SFT), a model is trained on fixed demonstrations of desired behavior; in RL, the model can instead generate its own attempts and be optimized according to feedback on their outcomes. When outcomes are checkable, the feedback can come from a verifier rather than exclusively from human judgement.
This is where RL environments become particularly useful. Here, the coding / solver agent generates its own trajectories, receives a reward based on the outcome and under RL settings, its policy is then updated to favour behaviors that score more highly. In algorithms like GRPO, multiple trajectories (or rollouts) can be sampled for the same task and compared against one another to determine which behaviors to reinforce.
The value of these environments are a reflection of why the current paradigm of language models are useful: for their ability to use intelligence well through executing complex tool calls, in the hope of completing multi-step, long horizon tasks. This maps closely onto most knowledge work, which largely consists of interdependent tasks rather than isolated actions. More importantly, the difficulty for agents increasingly lies in stringing longer sequences of actions together successfully rather than merely performing individual steps. As labs push models toward automating more of this work, they need environments in which those capabilities can be practiced and evaluated. A well defined environment lets the model learn from the consequences of its actions. Such an environment (as elaborated further later) is both a source of training and an instrument for measuring model behavior.
Why, then, has so much attention and capital accumulated around these environments? At the simplest level, labs have large budgets allocated toward improving model capabilities. Several posts have explained why labs do not inherently create all of post-training data in house (more on this later, as well)5. Labs, with their ever-growing areas to hillclimb on, have mostly insatiable demand, with supply of data of their required task shape / form lagging very much behind. Strong environments therefore become reusable seeds for trajectory generation; their value comes from the model improvement they can induce, not merely the unit cost of creating one. While these environments are valued highly, there is often significant deviation in the quality of said environments – whether these environments even produce the intended training signal – which will be explored in the following section.
Notes
FineWeb’s ablations held model size, architecture and training token count fixed, yet filtering for higher-quality educational content increased MMLU perf from ~33% to 37%, and ARC from 46% to 57%.
The classical formalism underlying an RL environment is the Partially Observable Markov Decision Process (POMDP), in which an agent receives observations of an underlying state, takes actions that transition the environment and receives rewards for said behavior. An RL environment can be thought of as the runnable instantiation of this interaction. Classical RL examples include MuJoCo or MiniGrid.
Also known as trace(s)
A specification of what the solver agent is being asked to solve. For instance, it could be as simple as “Create a Python function that calculates the `n`th Fibonacci number”.
The reason that is the most intuitive to me would be that of data diversity. It is, in my opinion, a rather high leverage move to outsource this task of curating data to various vendors, who then have their own separate methods of curating said data (through contractors, in house experts, etc.), rather than for labs to spin up their own in-house efforts.