Computer-use agent
A computer-use agent is an AI agent that works in software the way a person does, by reading the screen and using a mouse and keyboard instead of calling APIs. It can open an application, find the right field, type in it and click through to the next step. Any system with a screen becomes reachable, including ones that were never built to be automated.
The appeal is reach. Back offices are full of software with no usable API, like old billing consoles, carrier portals and internal admin tools. A computer-use agent can work those screens directly, so the work behind a customer request no longer has to wait on a custom integration.
How a computer-use agent works
The agent runs a perceive-and-act loop. The surrounding application captures the current screen and passes it to the model. OpenAI's Computer-Using Agent works from raw pixels, while other systems also read a page's structure to find buttons and fields more precisely.
The model then picks the next action based on the screen, the task and what has already happened. In Anthropic's computer use tool, targets are given as coordinates on the screenshot. The application, not the model, carries out the click, keystroke or scroll.
After each action, the agent checks the new screen to confirm the step did what it was meant to. The cycle repeats until the task is complete or the agent hands off to a person.
Open-ended versus recorded computer use
There are two broad ways to put computer use to work. In the open-ended approach, the model decides every click on its own, working out how to navigate an application it may never have seen. That flexibility is impressive, and it's where much of the research attention goes.
Enterprises usually want something more predictable. In the recorded approach, a person shows the agent how a task is done once, such as processing a refund or updating a customer record. AI turns that demonstration into a defined flow the agent can run the same way every time.
The agent still brings judgment to the work. It decides when a flow is needed, what inputs to pass in and what to do with the result. Decagon's Browser Actions takes this approach inside web applications, running recorded flows as one step in a larger conversation.
Computer use versus RPA and tool calling
Recorded computer use can look a lot like robotic process automation, since both follow set steps through an interface. The difference is what sits around those steps. A classic RPA bot runs a script from start to finish, and every exception has to be scripted in advance. A computer-use agent is guided by an agent SOP that uses the conversation and business context to decide what happens before and after each action.
That also makes flows easier to keep running. Small changes to a screen, like a moved button, can be handled without rebuilding the flow. When something does fail, the agent can hand off with a clear note on what went wrong.
Tool calling is still the right fit when a system offers an API for the action you need. Computer use fills the gaps between those integrations, so an agent can finish the whole task rather than stopping at the edge of what's been connected.
Keeping computer-use agents secure
A computer-use agent reads everything on the screen, and anything it reads can try to give it orders. Anthropic notes that a model may follow instructions found in webpages or images even when they conflict with the user's. That's prompt injection arriving through the interface.
Good controls keep that risk contained. The agent should log in with an account scoped to the tasks it handles, run in an isolated environment, and stay limited to the systems it needs. Recorded flows help here too, since the agent follows a known path instead of improvising across the screen.
Actions with real consequences, such as refunds or account changes, can also require a person to approve them. Every step should be logged so teams can review exactly what the agent did.
What computer-use agents mean for customer service
Resolving a customer request often means more than answering a question. It can mean updating a record in one system, checking a status in a partner portal and filing a form in a third. Computer use lets an AI agent carry out that work in the same tools employees use every day.
A typical setup looks like this. The agent looks up an order through an API, runs a recorded flow to update a field in a legacy console, then confirms the change before replying to the customer. The customer gets a finished resolution in one conversation, without waiting on a manual follow-up.
For support leaders, the question shifts from which systems have integrations to which tasks are worth automating. That opens up more of the work behind each request, without waiting months on engineering to connect every system.
For a deeper dive, download Decagon's guide to agentic AI for customer experience.

