Three studies. Three ways to lose track of the world.

In one sentence

An agent can use real search results, terminal output or tests and still reason from an assumption that the evidence has already challenged. The authors call this task-state contamination: an unsupported hypothesis or outdated plan persists in the agent’s history and shapes later moves.

How it works, step by step

An ordinary agent proposes a line of reasoning and its next action. AEWM sees the task, the history of observations and that proposal before the action runs. This matters: it does not pretend to know what the next tool will return.

Action Judge gives the proposed decision one of three labels. Critical is a directly necessary step, such as a decisive test. Exploratory reduces a real uncertainty. Noisy is a detour, repetition or move built on an assumption that the evidence no longer supports.

When a proposal is noisy, State Revision generates replacement reasoning and an action from the same observed history. The EditAct loop executes the selected move in the real tool and records the actual result. That changes the history the agent will use next time. The paper also tests AEWM-RFT: training agents on verified, corrected trajectories so they can later work without the online supervisor.

A concrete example

Imagine an assistant investigating why an app will not start. It spots an authentication error and immediately blames the API key. A later test shows the key works, yet the assistant still wants to regenerate it. A useful revision at that point would be: “The key has been checked. Let’s read the failing service’s log.”

The same action can mean different things in different contexts. Re-running a test after changing code may be critical; re-running it unchanged, without new information, may be noisy. That is why AEWM considers the history and the agent’s reasoning, not just the name of the tool.

What the researchers measured

The authors build a benchmark of 3,000 decisions, with 1,000 each from search, terminal and software engineering. AEWM scores 70.5% macro-F1 on classification, 10.6 points above the strongest model compared. Macro-F1 gives weight to all three labels, not just the most common one.

When placed in the agent loop, EditAct raises average scores across six benchmarks by 3.2–6.7 points against the strongest comparison, depending on the agent’s size. Training on verified EditAct trajectories beats Self-RFT by 2.2–2.6 points across the three reported domains. These are the authors’ measured tasks, not a promise that every agent will succeed.

Where it might help

The approach could help with long research tasks, debugging and automations in which each step depends on earlier ones. A practical design might focus the check before costly or hard-to-reverse actions, or when new evidence contradicts the plan. That is an application of the principle, not a product validated by this paper.

Where the evidence stops

The judge can be wrong too: it may block a valuable exploratory move or replace it with a worse one. The benchmarks and some annotations are made by the authors, so generalisation to other settings remains open. An online supervisor also adds operational cost. The contribution is a method with promising results in the tested domains, not a final solution to agent reliability.

In one sentence

An audio model may recognise a “doorbell”, “footsteps” or “someone calling” without knowing whether the source is to the left, behind a door or far away. For an agent that must navigate, naming the sound is not enough. OmniEcho combines spatial audio, vision and language to answer questions about a scene and decide where to go.

How it works, step by step

The paper introduces OmniEchoBench, combining audiovisual perception questions with navigation trials. It contains six tasks, 197 real audiovisual scenes, 2,972 question–answer pairs and 900 navigation episodes in 30 real environments. The navigation data include first-order ambisonics, or FOA: four audio channels that preserve directional cues.

To obtain larger training sets, the authors also build a synthetic pipeline. They specify the sound sources’ positions, camera and agent movement, then render audio that agrees with that geometry. Consistency matters: a model cannot learn “the sound comes from the right” from training data where the image and audio disagree.

The model adds a spatial FOA encoder alongside an audio pathway that already handles semantic content. It can therefore combine “what do I hear?”, “where is it?” and “what do I see?”. Navigation is not a single answer: the agent observes, picks a direction, moves and then has to decide again.

A concrete example

Imagine a home robot in a corridor with two doors. A person says “come here”. The words do not name the room; choosing a door requires the sound’s direction and the corridor’s appearance. After moving, the sound may change in loudness and direction, so the robot needs to update its hypothesis.

Or imagine an alarm hidden behind furniture. Video alone might not show its source; audio adds a clue. Walls, reverberation and competing sounds can also distort that clue.

What the researchers measured

The authors report that OmniEcho outperforms the systems compared on several spatial perception tasks in their benchmark. The most revealing figures concern navigation: in 52.89% of the 900 episodes, the agent enters the area within one metre of the source at least once, yet it successfully stops in only 16.22%. “I got close” and “I completed the mission” are different outcomes.

In the failure analysis, the mean absolute error in distance estimation is 2.20 metres. The authors also find that adaptation with real data lifts the average score on a held-out subset from 28.5 to 34.9, evidence of a gap between synthetic training and real scenes.

Where it might help

The principle could help service robots, mobile assistants and systems that need to locate off-screen sound events. A practical evaluation should measure recognition, direction, distance, route and final stop separately. The paper does not establish that OmniEcho is reliable for emergencies, public spaces or unsupervised assistance.

Where the evidence stops

Distance estimation remains weak and transfer from simulation to reality is difficult. The large gap between “enters the right area” and “stops correctly” has several possible causes: proximity estimates, path choices and how often decisions are made. The authors themselves caution against attributing it solely to the stop decision. The benchmark is useful, but covers defined environments and protocols.

In one sentence

A video can look realistic frame by frame and still be impossible. A hidden ball continues to exist: that is object permanence. A ball does not pass through a solid wall: that is solidity. WROP tests whether video models maintain these constraints when they generate the continuation of a scene.

How it works, step by step

The authors design 150 3D scene generators in Blender. Each changes lighting, camera angle and speed while preserving the same underlying physical question. The tests fall into six families: moving occlusion, static-scene occlusion, objects inside containers; then obstruction, support removal and collision.

The corpus contains 1.5 million training examples and a fixed 300-question exam. A model sees the first part of a video and must produce the second, where the decisive event occurs. The authors compare 14 models and further train a 16-billion-parameter continuation model called PWM-WROP. They do not introduce a new base architecture; they change its training signal.

The main evaluation uses blind comparisons between videos by people. This is a methodological choice: a pixel-similarity metric could reward a video that preserves colours and background while missing the physical event the test was about. Automated metrics therefore complement, rather than replace, the human judgement.

A concrete example

A small ball rolls behind a panel and should emerge on the other side along a plausible path. A generator might erase it, duplicate it or bring it back in a position incompatible with its trajectory. The result can still look cinematic at first glance.

In another test, a sphere falls onto a rigid surface. If it passes through as if the surface were air, the failure concerns solidity. These examples are close to the benchmark families; they do not prove that the model has a general theory of physics.

What the researchers measured

In the authors’ human comparison, PWM-WROP ranks first among true video-continuation models and third among all 14 systems. That suggests targeted training can improve performance on these tests. The primary evaluation consists of 361 pairwise judgements from 20 people. Confidence intervals for some ranks overlap, so an exact placing is not always a certain difference.

The systems also use different interfaces: some continue a video, others generate a new video from a reference, and others edit scenes. The distinction matters. A system free to regenerate a scene may have an advantage on some tests, but may preserve the identity of already shown objects less faithfully.

Where it might help

WROP offers a test bed for video generators and research models that predict how physical scenes evolve. A team assessing synthetic video could use similar tests to separate visual quality from event consistency. Extending this to open-ended simulations or real robots remains a research possibility, not a result demonstrated here.

Where the evidence stops

The scenes are synthetic and controlled; the real world contains more objects, materials, forces and ambiguity. Model classes are not perfectly comparable. PWM-WROP is also trained on the same structure of tasks that the benchmark tests, so its result does not establish general physical understanding outside these families. The paper’s value is to make a failure that can hide behind beautiful video easier to measure.