Nobody Told the Agent to Use the Camera. It Worked That Out on Its Own.

Two robot arms on a white table. Red and green cubes on the left, a bucket toward the top right, and three USB cameras: one overhead, one on each gripper. The two gripper cameras are the same model, mounted the same way, plugged into the same machine. To the operating system they are just /dev/video0, /dev/video2 and /dev/video4.

The instruction given to the AI coding agent, as the team describes it, was simple: use the top camera and the left gripper camera. Nobody told it which device was which.

This account comes from RebelDot, written up by AI engineer Andrei Voic, describing frames the agent captured on 6 June 2026. It is a first party engineering write up rather than an independently verified report, and I am treating it that way throughout.

What the agent actually did

According to the write up, the agent inspected the available video devices, recognised that the odd numbered entries were metadata nodes, and identified three real camera feeds. It captured reference frames from each feed and copied the images over SSH so it could look at them. When it tried to open all three streams at once, video4 timed out, which the team attributes to a real USB 2.0 bandwidth limit from three uncompressed streams sharing one bus, so the frames were read one at a time.

The overhead view was easy. /dev/video0 showed the whole table, both arms, the cubes on the left, the bucket top right. The gripper cameras were the hard part: two near identical close ups of an orange gripper above the same white table, with metadata that gave nothing away.

So the agent used a landmark. The overhead frame established that the cubes sat on the left side of the table. One gripper camera could see those cubes at the edge of its frame. The other was looking at an emptier stretch of table nearer the bucket. From that, the agent concluded that /dev/video4 was the left gripper camera and /dev/video2 was the right one, which it could ignore for this task.

The team is explicit about what it did not supply. No instruction to take reference photos. No suggestion that the cubes could work as a landmark. No camera map, no spatial description. The agent assembled that investigation itself.

Separating four things that are easy to blur

It is worth being precise here, because the whole point of the story lives in the gaps between these categories.

• What the human requested. Run the data collection process using the top camera and the left gripper camera.
• What the agent decided by itself. Enumerate video devices, distinguish real feeds from metadata nodes, capture frames, move them over SSH, look at them, work around a bandwidth timeout by reading one at a time, and reason visually about which physical arm each feed belonged to.
• What the environment allowed. Shell access on a machine with the cameras attached, plus SSH between that box and wherever the agent was reading the images. That is what made the whole sequence possible, and it existed before the instruction did.
• What required confirmation. The mapping conclusion, before the agent moved on to the data collection itself.

Notice how much of the middle category is ordinary. Listing devices, copying files, opening images. Every one of those actions is unremarkable in isolation. Together they are a small autonomous investigation that nobody specified and nobody would have thought to authorize individually.

Capability expanded. Authority did not have to.

This is the sentence I keep coming back to. Autonomous reasoning can expand. Authority does not have to expand with it.

The instruction was an objective, not an execution plan. The agent hit a gap between the objective and reality, and instead of failing or asking immediately, it generated a path. That is the behaviour everyone building agents actually wants. It is also the behaviour that makes it impossible to enumerate, in advance, every action the agent will take.

If you cannot enumerate the actions beforehand, then approving the objective cannot be the same as approving the actions. Which raises the question this story is really about: when the agent invents the next step itself, what determines whether that step is allowed to execute?

The part I cannot verify, and it matters

The confirmation step is the most encouraging detail in the whole account, and it is also the one I have the least evidence about.

The write up records that the agent reported its reasoning and asked before continuing. What it does not establish is why it asked. Was that a system enforced gate that the agent could not have passed on its own? Was it a policy in the prompt? Or was it simply how the model behaved on that run, on that day, with that phrasing?

I am not going to guess, because the difference between those answers is the difference between a control and a habit. A control holds when the model changes, when the task is phrased differently, when the run is the thousandth rather than the first. A habit is a property of a sample size of one. Both look identical in a blog post. Only one of them is something you can rely on.

That is not a criticism of the team. They were documenting an interesting engineering moment, not publishing a control audit. But if you are reading this story for reassurance, read that distinction carefully first.

Sensors and actuators are not the same risk

Here is where the physical setting changes the maths.

Everything the agent did in this account was observation. Reading device lists, capturing frames, copying files, looking at pictures. A camera is a sensor. The worst case of a wrong inference at that stage is a wrong conclusion.

A robot arm is an actuator. It changes the world, and the world does not have an undo. The account describes the agent working toward a data collection process involving arms that move physical objects on a physical table. I have no evidence about what arm control the agent did or did not have, and I am not going to imply any. Nothing in the published material describes the agent taking unauthorized control of the arms, and I want to say that plainly because this story would be very easy to sensationalise in that direction.

But the structure is worth naming even when nothing went wrong. An agent that can independently expand its own action set from observation is an agent whose action set could, in a differently configured environment, expand across the line from observing the environment to changing it. The mechanism that decides whether it does is not the model's judgment. It is whatever sits between the decision and the execution.

Why I think this is a better story than a failure would have been

Most of what we cover at Belay Intelligence is what happened after a boundary was missing. An agent reached a production database. An agent followed a path nobody intended. Those stories are useful, but they all argue the same way: something broke, therefore a control was needed.

This one argues forward instead. Nothing broke. The agent was resourceful, the inference was correct, the human was asked, the task proceeded. And the authority question is still there, completely intact, because the agent's competence is exactly what created it. A less capable agent would have stopped at the ambiguous device names and asked immediately. It is the capable one that goes exploring.

Capability is improving quickly. Planning, tool discovery, multimodal reasoning, adaptation to messy environments. Every improvement widens the set of actions an agent can reach without being told. None of it widens, by itself, the set of actions it should be permitted to execute.

Questions I would ask about my own setup

• Can I list the actions my agent took last week, or only the tasks it completed?
• Which of those actions did I specify, and which did it generate?
• What is technically reachable from the credentials and shell access it already holds, independent of what I asked for?
• When it pauses for approval, is that a gate or a behaviour? Would it still pause if the model were swapped tomorrow?
• Where exactly is the line between reading state and changing state, and is anything enforcing it?
• If an inference were wrong, which of the next actions would be reversible?

That last one is the one physical systems answer differently. In software you can usually restore. A gripper that closed on the wrong object has already closed.

The thing worth keeping

An objective is not authorization for every action discovered while pursuing it. That gap used to be small, because agents could not do much beyond what you spelled out. It is getting larger in exactly the direction we were hoping for, which is the awkward part.

The agent in this account did good work. It also demonstrated, in a completely benign way, that the set of things it will decide to do is no longer something a human enumerates in advance. When the next step is invented by the agent, the only reliable place to answer "is this allowed" is at the moment of execution.

Your agent found a clever way around a problem this week. Do you know what it did?