CodeMidas turned 3,185 open-source codebases into 5,545 training tasks for agentic coding environments. And it never read an issue, a commit, or a written task record to do it.
The paper behind that number, CodeMidas: Scaling Agentic Coding RL Environments from Code Itself, describes an agentic pipeline that constructs executable reinforcement-learning environments using source code as its only task-specific input. Strip the jargon and here is the direct answer: agentic coding environments are the executable surroundings a coding agent works inside, meaning a real repository, a shell. And tests that decide pass or fail, and they are separate from the agent itself. CodeMidas automates the expensive part of building them, producing 5,545 verifiable tasks across 23 programming languages and 15 technical domains. And ThinkLLM’s digest reports that training on them lifted code repair, program synthesis, and terminal tasks by 8-18%. The scarce resource in coding agents has shifted from model access to verifiable environments. And the raw material is code that already exists, including yours.
The Environment Is the Moat, Not the Agent
Every ranking of AI coding tools argues about agents.
The research spend says the moat sits somewhere else.
The KAT-Coder-V2.5 technical report describes its model as “trained to act autonomously inside real, executable repositories rather than as a single-turn code generator,” and the Hugging Face summary of KAT-Coder-V2 describes the supporting infrastructure, KwaiEnv, as “a modular infrastructure sustaining tens of thousands of concurrent sandbox instances,” with RL training scaled along task complexity, intent alignment, and scaffold generalization.
Nobody stands up tens of thousands of concurrent sandboxes to serve autocomplete.
The configuration research points the same direction.
An arXiv study on setting up agentic coding tools documented eight mechanisms practitioners actually use: Context Files, Skills, Subagents, Commands, Rules, Settings, Hooks, and MCP servers.
Notice what all eight have in common. Not one of them is picking a model. And every one of them shapes the environment the model works in. When an agent wrecks a refactor on a client repo, the fix is nearly always environmental: better context, a real verifier, a sandbox that matches where the code runs. I stopped blaming the model first a while ago, and these papers back that habit up.
Finished Code Beats the Ticket Queue
The detail most coverage will skip is the one that matters most. CodeMidas starts from implemented functionality in open-source codebases, not issues, commits, or written task records, as ArXivSignals characterizes it. That is backwards from how every team I know plans work, and it is correct anyway. An issue is unverified prose. Shipped code is a specification that ran and survived contact with users.
The mechanics make the logic plain. Agents explore implemented functionality, formulate behavioral specifications and task statements, then adapt each codebase into a starting point where the target functionality still has to be implemented. Tests are built from execution of the original code, with execution consistency checks on every candidate task. And agentic compute is allocated to every one of those stages. The ground truth gets captured while the feature still works, then the feature is pulled out and the agent has to rebuild it against tests derived from reality instead of from a ticket’s mood.
FeatureBench makes the same bet for evaluation instead of training: an execution-based protocol and a scalable test-driven method that derives tasks from repositories with minimal human effort. Two independent papers, one conclusion. The richest source of coding tasks is the code you already finished.
The Filtering Is the Real Invention
5,545 tasks is the headline number, and it is the least interesting part of the paper.
The part worth stealing is the post-rollout filtering.
Adversarial rollouts probe candidate tasks for exploitable leakage, solution reviews assess verifier decisions against the stated requirements. And rollout success rates guide which tasks make the final cut.
Every one of those stages exists to kill tasks an agent can pass without doing the work.
That is the actual failure mode of reinforcement learning on code, and of agentic coding generally. An RL agent optimizes the reward you implemented, not the reward you intended. Hand it a weak verifier and it learns to satisfy the verifier, which is a different skill from writing correct code. Anyone with an API key can generate fifty thousand tasks with a prompt loop. So the defensible asset is the adversarial check that throws most of them away.
One honesty note, because I would want it said to me: the 8-18% gain comes from ThinkLLM’s summary, and it is the only secondary source carrying that number. I trust the direction more than the exact band. Read the paper’s own tables before you put the range in a client deck.
What a One-Person Shop Does With This
You are not training a foundation model this quarter, and neither am I. The pattern transfers anyway, and it transfers this week.
– Separate read-only from executable. Tools like DeepWiki, Google Code Wiki, Davia, deepwiki-rs, OpenDeepWiki, and PocketFlow Tutorial Codebase Knowledge generate wiki-style pages, diagrams, and knowledge graphs from a repository snapshot, per one comparison piece. That is exploration, not environment. An environment executes code and grades the result.
– Write the verifier before the prompt. CodeMidas runs spec-from-working-code, then tests, then the agent. That order works for a client refactor too: capture behavior while the code runs, then let the agent loose against it.
– Treat your repo as curriculum. The same pipeline that produced 5,545 tasks from strangers’ codebases would produce a smaller, sharper set from yours, since yours matches your actual domain.
– Swap models freely, keep the environment. Sourcegraph Cody supports Anthropic Claude, OpenAI GPT, Google Gemini, and Mistral AI models, which tells you where the durable asset sits. The model is a line item; the environment is the factory.
– Keep the human on the verifier. That same comparison piece calls these tools “cognitive amplifiers for code comprehension, not as replacements for human judgment or engineering expertise.” Correct, and it applies double when the agent holds edit access.
If you do want to train rather than configure, Orchard is an open-source framework for scalable agentic modeling with specialized recipes for coding, GUI navigation, and personal assistance tasks.
Start there before building scaffolding from scratch.
Your Codebase Is the Training Set
The uncomfortable version of this trend: the labs no longer need your issues, your commits, or your docs to teach an agent your kind of work. They need your code, and open source already handed over 3,185 codebases’ worth. The comfortable version: the technique cuts both ways.
Verified tasks can be manufactured from any code you own.
And a small operator who builds one honest verifier for one messy repo ends up with something a generic model does not have.
Start small this week.
Pick one repo, capture its behavior in tests while it still runs, and point your agent at a rebuild. If you want a second operator’s opinion on turning a client codebase into a verified agent environment, that is the work my shop does. Bring the repo, and expect the first hour to go to the tests, not the model.
