Follow coding-agent corrections until the problem is resolved

In one sentence

The authors ask whether they can predict which generation order will work better for one model before using each order to produce an output. Imagine filling blank spaces in a sentence: filling neighbouring spaces together may miss clues they could have given each other, while filling spaces farther apart may preserve more of those clues. The authors describe each possible order as a path through a set of partly completed states, which they call a corruption lattice. They define a dependence cost for clues lost when positions are filled in parallel, then estimate that cost from pairs of positions using a trained model. They compare the predicted rankings with measured results for text, image and video models.

Key concepts

• Generation schedule: the order in which a model fills positions, and how many it fills at each step.
• Corruption lattice: the authors’ shared map of states in which each text or image position can be blank, partly corrupted or complete.
• Dependence cost: a mathematical measure of what independent, simultaneous updates miss when the positions being updated depend on one another.
• Pairwise kernel: an estimate, taken from trained model weights, of how strongly two positions depend on each other under a given partly completed state.
• Treedepth: for data meeting the paper’s graph assumptions, a property of the dependency pattern that sets the fewest steps possible without this dependence cost.

A concrete example

Hypothetical illustration, not a study result: when filling several gaps in a draft sentence, a system might first choose gaps spread across the sentence rather than adjacent gaps. The idea is to avoid settling neighbouring words independently before either can provide context for the other.

What the researchers measured

On the released MAR-B image model, using its released evaluator on ImageNet-256, the authors report an FID-50K of 13.02 for the model’s random reveal order at 8 steps and 9.44 for spread order at 8 steps. FID-50K is an image-distribution comparison over the evaluation samples; lower is better. Only the reveal order changed. For released LLaDA-8B-Base on the GSM8K maths problems, the authors report that a minimum-distance rule raised strict-match accuracy by 9.7 ± 0.8 percentage points relative to plain confidence selection at 8 steps, using greedy decoding. Strict match counts answers that match the expected answer. The authors say most measured schedule rankings followed their predictions, while identifying exceptions.

Why it matters

For the released models studied, the authors changed schedules without changing the model weights. Their framework gives developers a way to investigate an order before paying for every full decode. Its mathematical zero-cost result applies under stated assumptions about the data’s dependency graph, not automatically to real text or images.

Where it might help

A possible application is comparing candidate generation schedules for an existing model at a fixed step budget. The paper tests changes to decoding order; it does not establish performance in a workplace deployment.

Impact across sectors

• Possible, not tested in deployment — writing tools: developers could compare which blank-text positions to fill together when limiting generation steps.
• Possible, not tested in deployment — image production: developers could examine whether a different order of image-token updates changes output quality at the same step budget.
• Possible, not tested in deployment — video production: developers could examine the trade-off between how many earlier frames a segment uses and how many decoding steps it needs.

Where the evidence stops

The authors say the dependence cost leaves out errors in a trained model’s predictions, which can determine rankings when predicted costs are close. Their pairwise bound is exact only for a specified first-order Markov reference model. On MAR-B, the nested and low-discrepancy orders did not follow the ordering of their estimated floors. Real text also retains dependence across a revealed character, unlike the graph-based zero-cost example. The MAR-B table generally reports one seed per cell, with stated exceptions; the video comparison uses three prompts. The authors also identify teacher-forced text tests, in which true characters are revealed between steps, separately from free generation. General PtoP note: this arXiv source is a preprint, not evidence of peer review, and benchmark results are not a tested deployment.

In one sentence

A useful correction needs a follow-up: did the coding agent fix the problem, or merely follow the advice? Imagine an agent repeatedly searching for a file with a command that does not work. Opera, the authors’ verbal critic, can point out the failed search, keep that issue in a note, and check later whether the agent found the right code. It reviews the agent’s visible work at intervals or after events such as repeated actions, errors and a claim of completion. It may also stay silent. Before sending advice, an audit checks whether the visible evidence supports it. Opera then tracks whether the agent attempted the advice separately from whether the original issue was resolved.

Key concepts

• A coding agent explores files, edits code and runs tools to carry out a software task.
• A verbal critic gives the agent a written correction during its work rather than editing the code itself.
• A persistent note keeps one diagnosed issue and its resolution criterion in view across later reviews.
• An audit screens a proposed correction before delivery and checks the evidence before closing its note.

A concrete example

Hypothetical illustration: an agent changes the wrong error-handling branch. A critic’s note identifies the branch and asks for a check using an input that should reach it. Changing the code shows the agent attempted the advice; seeing the required behavior in the check would address the note’s original issue. This illustration is not an additional measured result.

What the researchers measured

The authors report resolve rate—the share of benchmark tasks completed successfully. With Qwen3.8-27B as the coding agent and GPT-5.6-Sol as Opera’s critic, Opera’s mean resolve rate on Terminal-Bench 2.1 was 73.8%, versus 65.9% without a critic, across the benchmark’s 89 tasks. In a separate training experiment, Qwen3.5-9B fine-tuned on Opera-guided attempts achieved 38.9% on held-out SWE-Bench Pro repositories without a critic at evaluation time; the authors report a 10.2-percentage-point gain over the base model. These figures describe different models and settings. The authors also report that Qwen3.5-9B resolved no DeepSWE v1.1 task in the evaluated runs with or without the critic.

Why it matters

The distinction between attempting a fix and resolving an issue matters when an agent can change code yet leave the failing behavior intact. The authors’ tests examine both feedback during a task and training from the resulting agent work.

Where it might help

A possible use is to give an existing coding agent timely, evidence-linked feedback while it works. The paper also studies using successful critic-guided work as training data so a model can later work without the critic. Neither use should be read as a demonstrated deployment outside the reported evaluations.

Impact across sectors

• Possible, not proven: software maintenance teams could use the note-and-follow-up idea when agents investigate repository bugs.
• Possible, not proven: developer-tool makers could consider checks before delivering automated advice to a coding agent.
• Possible, not proven: teams training coding models could examine corrections made during a model’s own attempts as a source of examples.

Where the evidence stops

The SWE-Bench Pro test-time evaluation used a corrected 100-task subset, not the full benchmark. For the training comparison, the authors selected training tasks on which both the critic-guided student and the stronger teacher had a successful attempt; the held-out evaluation used different repositories. The authors say the training results cover supervised fine-tuning of a single model and one out-of-domain benchmark. They also report that feedback sometimes caused regressions on tasks an agent would otherwise solve. Opera’s critic sees a bounded record of the agent’s visible work, not files or tests it runs independently. General PtoP note: this source is an arXiv preprint, not a claim of peer review; benchmark results are not a deployment test.

In one sentence

The authors ask how to find the sources from which coding-agent skills spread, and whether reviewing those sources could limit the spread of risky capabilities. Their answer is to trace copies through time, rather than judge a source by its popularity. A coding agent is software that helps carry out coding tasks; an agent skill is a folder of instructions, sometimes with scripts, that the agent can follow with its user’s permissions. The authors studied the recorded change histories of SKILL.md files—the instruction files for these skills—in public GitHub repositories, or shared project folders. They dated when each repository first acquired a skill and linked batches of copies to an earlier repository that held them. They call the resulting copy history for one skill a constellation. They then fitted a source-choice model, a calculation that estimates which earlier repository a future copier is likely to choose, and tested review priorities against a later, held-out period of data.

Key concepts

• A copy event is a commit that adds ten or more skill files when one earlier repository already held at least half of their existing skill lineages. A lineage links versions of one skill that are connected by edits.
• A transmission is a skill lineage within a copy event that the identified source held first. This traces distribution; it does not establish who authored the skill.
• A high-risk flag marks capabilities such as bundled executables or instructions for risky actions. The authors say the flag indicates capability, not malicious intent.
• A held-out period is later data excluded from the period used to choose review targets. Here, the authors used it to evaluate what their proposed audits would prevent under their stated rules.

A concrete example

Hypothetical example: a team copies a folder of agent skills from another project. Later, the original project changes one skill, but the team’s copy remains as it was. Looking only at today’s folders would show who has a skill; looking at the copy history could help identify the earlier project to review and the copies that may need attention. This scenario illustrates the method, not a separately measured case.

What the researchers measured

For a split at 1 April 2026, the authors report that reviewing the 100 repositories ranked highest by their source-choice model prevented 14.9% of later adoptions of high-risk skills in the held-out evaluation. Reviewing the 100 most starred prevented 0.5%. These percentages count later skill adoptions that the evaluation’s audit rule would prevent by removing flagged skills at reviewed repositories and their downstream copies; they are not observed outcomes of real-world audits. The authors also report that skill copies rarely follow later edits at their source.

Why it matters

A copied instruction file can be followed with a user’s permissions, yet a copy has no automatic link to later changes at its source. The authors’ history-based view distinguishes repositories that distribute skills from repositories that merely hold them. It also shows why a source fix should not be assumed to reach existing copies.

Where it might help

A possible use is to prioritize reviews of repositories from which others copy skills, then check existing copies separately when a source changes. The paper evaluates a review rule on historical data and in a simulation; it does not report a deployed review programme.

Impact across sectors

• Possible impact for security teams: use dated copying patterns, rather than stars alone, to propose a shortlist of repositories for review. This is an application, not a tested deployment.
• Possible impact for coding-agent platforms: consider versioned references to skills so users can identify a source and its updates. This is the authors’ proposed direction, not a measured platform change.
• Possible impact for software teams: keep track of copied skills and check them when a source changes. This is practical guidance inferred from the findings, not a measured workplace outcome.

Where the evidence stops

The authors say their data cover the format’s first ten months on public GitHub. Their risk flags mark capabilities rather than malice, and their risk counts are lower bounds because the flags miss some relevant skills. In a check against skill-installer records, their source rule often identified a distributing repository rather than the recorded source; source-level measures therefore describe distribution, not authorship. The review findings come from a held-out historical evaluation and a calibrated simulation, not a deployed audit. PtoP note: this arXiv source is a preprint; the supplied text does not establish peer review.

In one sentence

Most messages in these conversations did not need a distant memory, but some did. The authors ask whether an AI companion can tell the difference, find the relevant exchange and avoid claiming to know more about a person than they shared. They release conversations from ten consenting participants, together with records of stated facts, interpretations of each person, and test items tied to supporting messages. For chat tests, they sample messages the participants sent and record what an appropriate reply could draw on. They then test ways to find earlier messages, compare replies given different context, and ask three agent systems to reconstruct portraits of the participants.

Key concepts

• Memory demand means a reply needs something beyond the current exchange. The authors estimate how often this occurs using only a sample drawn without selecting for memory-dependent messages.
• A profile records claims a participant stated. A persona records interpretations of how that person thinks, feels or decides; the paper treats these as different kinds of claims.
• Retrieval means finding an earlier message that a reply needs. Checking only the newest messages can look effective when many test items concern the current exchange, yet miss distant references.
• Restraint means not bringing up the past when it is not needed. The authors test this separately from whether a system can find and use a relevant past message.

A concrete example

Hypothetical illustration, not a paper result: Someone tells an assistant in spring that they are unsure about joining a gardening class. Months later, they say, “I went back.” A useful reply might first establish whether “back” refers to that class. On an unrelated new topic, bringing up the class could be a mistake.

What the researchers measured

In the authors’ proportional sample of scoreable chat probes, 3.4% needed something outside the current thread under their recorded reading. Under their stricter reading, which removes references already visible on screen, the figure was 1.3%. For memory-bearing probes, the furthest required message lay a median of 2,157 messages back. In the authors’ distant-item retrieval test, taking the five most recent messages found a required message on 0.022 of items; this score is the share of items with at least one required message among those five. The authors also report that, in their matched context comparison, a “memories” heading made the generator bring up the past 10 to 14 percentage points more often than a neutral heading, including when the past was not needed. In the full-history persona reconstruction test, Antigravity running Gemini 3.8 Flash scored F1 0.701 against the released persona fields; F1 combines how many scored fields it recovered with how many of its filled fields agreed. The authors report that the three tested systems also added interpretations beyond what the released personas supported.

Why it matters

A test built around questions that require an old fact does not ask the assistant to decide whether remembering was necessary. The authors’ sampled chat messages let them study that decision separately from finding the right message and writing a reply.

Where it might help

The authors present the release as a benchmark for testing when to remember, what to find and how to use it in a reply. It can also support tests of how systems build a profile or persona from a conversation. These are research uses, not demonstrated product features.

Impact across sectors

• Possibility, not a tested deployment: teams building companion apps could test whether an assistant reaches into old chats only when warranted.
• Possibility, not a tested deployment: teams building customer-support assistants with continuing conversations could examine whether an old exchange helps with a current request or merely intrudes.

Where the evidence stops

This is an arXiv preprint, not a peer-reviewed publication. The authors say the ten participants chose one companion product and are not a sample of any population. The two heaviest relationships contain 73% of the messages. Released messages were rewritten for anonymity, and the profiles, personas and chat labels were derived with models and reviewed or audited; a citation to a message does not by itself establish that the message supports a claim. Some sampled items were withdrawn or were not scoreable. The authors report both recorded and stricter readings because some purportedly distant references were visible on screen. They also say a comparison between recorded context and recent messages changes the heading as well as the messages, so its difference cannot be assigned entirely to memory. Empathy, prediction of future behavior and a person’s values are outside this benchmark’s scope.

In one sentence

A model’s choice between ordinary words can carry information about training it received for an unrelated task. The authors tested this by starting with a public language model, then separately adapting a copy for a task such as coding. They used the public model to find prompts where it was almost equally likely to choose either of two words. They asked the adapted model for just one word per prompt and trained another copy of the public model on those prompt–word pairs. Finally, they tested that student on the task it had never seen during this training. The authors call the changes visible in unrelated choices a behavioral shadow and their procedure Active Taskless Distillation.

Key concepts

• A shared ancestor is the public model from which both the task-trained teacher and the student start. The method uses that known starting point to select prompts.
• A near-tie is a prompt where the public model has almost equal preference for two offered words. A small change in preference can then change which word is chosen.
• A carrier is an ordinary prompt and its one-word answer. It carries a training observation without containing an example of the target task.
• A matched control is a student trained in the same way with the word choices reassigned to prompts. In the primary coding comparison, the exact nuisance-matched control also preserves the collection of chosen words and matches other properties of the choices.

A concrete example

Illustrative example from the authors’ carrier data: a prompt about a lobster and a heart offers “jacket” and “tie” as possible next words. The public model slightly prefers “tie,” while the coding-trained teacher chooses “jacket.” The student learns that prompt–word pairing, not a coding solution. This example illustrates the training input; by itself, it is not a measured coding result.

What the researchers measured

In the primary Qwen2.5-1.5B coding experiment, the authors trained a student on 5,664 one-word responses from a coding-trained teacher. On HumanEval+, which checks whether generated code passes tests, the student’s pass@1 score was 51.22% versus 45.88% for the exact nuisance-matched control. Pass@1 counts tasks solved by a single generated solution. The authors report a +5.34 percentage-point difference, with a 95% confidence interval of [1.22, 9.60] percentage points. In separate task-specific Qwen2.5-1.5B settings, students also scored above their own shuffled-label controls on the six reported multiple-choice benchmarks. For three additional model settings, the authors report positive mean coding gains, but the paired 95% confidence intervals include zero.

Why it matters

The authors report that, in their tested settings, information about a task-specific update appeared in choices whose visible words were unrelated to that task. This matters for understanding what can be learned from access to an adapted model’s outputs; it does not establish that arbitrary prompts or models will transfer a capability.

Where it might help

Possible uses, not tested deployments, include studying whether an adapted model’s changes appear outside its training task and designing controlled comparisons of model adaptations. The reported method needs a known public ancestor and carefully selected prompts.

Impact across sectors

• Software development — possibility, not a deployment result: teams studying code-adapted models could use the finding to ask whether unrelated outputs contain information about an update.
• Education — possibility, not a deployment result: researchers studying models adapted for science questions could investigate similar off-task choices, while still testing learning outcomes separately.

Where the evidence stops

The authors say the approach relies on actively constructed prompts and a known public ancestor, and does not yield reliable transfer in every tested setting. A secondary coding test, MBPP+, showed little recovery: the primary acquisition’s signal-minus-control result was +0.33 percentage points, with an interval that included zero. The authors also report no transfer from two teachers that were not compatible descendants of the students, but say those pairs differed in other ways too. The code teacher’s source data was filtered for overlap with the coding benchmarks; that filter applied to the teacher’s data, not to the student, which received only carriers. General PtoP note: this source is an arXiv preprint, not a claim of peer review, and benchmark results are not deployment results.

In one sentence

An AI agent can carry out steps toward a broad improvement goal, but the authors find that completing those steps rarely produces an improvement that survives their independent checks. They built Aspire to study a problem that starts before training: deciding what a goal such as “improve mathematical reasoning” means in practice. The agent chooses data, an update method and its own ways to check progress. The researchers keep the evaluation tasks hidden, then check the resulting model or the supporting system around it. They study changes to model weights—the settings changed by training—and, separately, changes to an agent harness—the instructions, tool rules and workflow that surround a model.

Key concepts

• Goal operationalization means turning a broad aim into specific things to practise and ways to check progress. A person told to “write better reports” faces the same first step: deciding which kinds of reports and which weaknesses matter.
• A proxy is a stand-in test chosen during improvement. Doing better on that test need not mean doing better on the wider goal.
• A hidden evaluation set is a collection of tasks the agent cannot inspect while choosing its updates. Aspire uses it to check whether progress carries beyond the agent’s own practice material.
• A checkpoint is a saved version of a model after training. In the weight-update experiments, the controller could keep an eligible checkpoint that beat the starting model or roll back to that starting model.

A concrete example

Hypothetical illustration, not a paper result: an assistant asked to become better at workplace research might practise summarising articles and score itself on tidy summaries. A separate check might instead ask it to identify an unsupported claim. The practice score alone would not answer whether it improved at that task.

What the researchers measured

In the final-only weight-update setting, Qwen3.5-9B Self exceeded its starting score on 1/6 model–goal pair means; each mean combined two separately run final checkpoints, and the agent received no intermediate hidden-evaluation score. Qwen3.5-4B Self had no pair mean above its starting score. In the separate adaptive-feedback setting, 28/30 configuration–goal cells produced an evaluated checkpoint, but only the Terra-directed Qwen3.5-4B mathematics cell retained one that scored above its starting model under the study’s selection rule. These counts describe outcomes, not how many training steps succeeded. In the harness study, all valid successor harnesses ran with fixed Qwen3.5-4B model weights and had lower reported means than the original Qwen-Agent reference on the academic and scientific writing goal. The authors also report lower overall scores for vague-goal runs than for explicit-task references in their separate PostTrainBench comparison; they do not treat that comparison as a prompt-only causal effect.

Why it matters

The authors distinguish doing the improvement work from showing that it helped. An agent may gather data, train, save a checkpoint and see a rising score on its own checks, yet still finish below its starting model on the goal-specific hidden tasks. For a working team, that distinction matters whenever it must decide whether to keep an update.

Where it might help

A possible use of the paper’s approach is to separate an AI team’s chosen practice checks from an independent check of the capability it wants to improve. The paper tests this in a research environment, not in a workplace deployment.

Impact across sectors

• Possible, not a tested deployment — education: a team improving a study assistant could distinguish success on self-chosen exercises from performance on independently prepared questions.
• Possible, not a tested deployment — healthcare research: developers of an assistant for medical reasoning could check whether practice on selected material carries over to separate expert-authored tasks; this paper does not establish clinical performance.
• Possible, not a tested deployment — software teams: people revising an agent’s instructions and tool workflow could test the revised system against a fixed reference rather than rely only on its own checklist.

Where the evidence stops

The authors bound their conclusions to the goals and coverage of their expert-authored tasks. The adaptive-feedback study has one run per configuration–goal cell, and its retained Terra mathematics checkpoint was selected using repeated aggregate feedback on the same evaluation items, without a separate confirmation set. The final-only and adaptive-feedback settings have different feedback and selection rules, so they are not repeat trials. The harness study tests one-step edits for one writing goal; its repeated executions measure variation in a frozen harness, not repeated creation attempts. The evaluations focus on particular goals and do not establish that unrelated abilities were preserved. The authors checked registered training data for overlap with hidden tasks, but say this does not establish that those tasks are absent from a model’s earlier training material. Some content-level trace evidence is available only under controlled access. General PtoP note: this arXiv source is a preprint, not evidence of peer review, and benchmark results are not a deployment test.