**Building Browser-Using Agents for Support Workflows: A Hands‑On Guide**
Many high‑value workflows still live behind web interfaces: support consoles, dashboards, and CRM systems. For large language model (LLM) agents to truly assist teams, they need to work directly in those browsers—clicking buttons, filling forms, reading data, and taking action. This article shows how to build a browser‑using agent with the OpenAI Agents SDK and Playwright MCP, using a real‑world support ticket as a concrete example.
—
### 1. The Mental Model
At a high level, a browser‑using agent is an LLM placed in an interaction loop with a browser:
1. The agent receives a task and the current browser state.
2. It interprets the state, decides what to do next, and sends an action back to the browser.
3. The browser produces a new state, which becomes the input for the next decision.
4. The loop continues until the agent signals task completion.
To make this loop work, we need two reliable channels:
– Observation channel: how the agent sees the browser (e.g., screenshots, accessibility trees, or structured DOM information).
– Action channel: how the agent influences the browser (e.g., mouse, keyboard, or element‑targeted commands).
Common pairings include:
– Screenshot + coordinate‑based mouse/keyboard actions (general computer use).
– Structured page state + element‑targeted browser actions (browser‑specific use).
This article focuses on structured observations combined with element‑targeted actions, which are well‑suited for support consoles and similar web apps.
—
### 2. Case Study: Resolving a Customer Support Request
We’ll build an agent that resolves a customer support case entirely inside a web‑based support console.
#### 2.1 Preparing the Support Console
I created a small static support console using HTML, CSS, and JavaScript—no backend or database required. You can serve it locally:
“`powershell
cd toy_app
python -m http.server 8000 –bind 127.0.0.1
“`
The console includes:
– An inbox where you can select a case.
– Panels showing order details, customer context, and resolution policies.
– A way to add an internal note and record a resolution.
– An audit log that tracks updates.
The agent’s task: investigate one incoming support case and carry it through to resolution, using only the console interface.
#### 2.2 Defining the Browser‑Using Agent
Our agent stack:
– **OpenAI Agents SDK** for agent runtime.
– **Playwright MCP** to connect the agent to a real browser.
Here’s the final agent definition:
“`python
from agents import Agent, ModelSettings
from openai.types.shared import Reasoning
agent = Agent(
name=”Support Console Browser Agent”,
model=”gpt-5.4″,
model_settings=ModelSettings(
reasoning=Reasoning(effort=”medium”),
),
instructions=AGENT_INSTRUCTIONS,
mcp_servers=[playwright_server],
)
“`
Three components need explanation:
1. **LLM client**: connects to Azure OpenAI (or your provider) using the Responses API.
2. **Agent instructions**: kept minimal to focus on browser interaction.
3. **Browser tools**: provided by Playwright MCP.
##### Connecting the LLM Client
“`python
import os
from openai import AsyncAzureOpenAI
from agents import set_default_openai_api, set_default_openai_client
azure_client = AsyncAzureOpenAI(
api_key=os.environ[“OPENAI_API_KEY”],
api_version=os.environ[“OPENAI_API_VERSION”],
azure_endpoint=os.environ[“OPENAI_API_BASE”],
)
set_default_openai_client(azure_client)
set_default_openai_api(“responses”)
“`
##### Agent Instructions
“`python
AGENT_INSTRUCTIONS = “””
You are an agent that can interact with a web browser.
“””.strip()
“`
##### Browser Tools with Playwright MCP
Playwright MCP exposes page structure as accessibility snapshots and lets the agent target specific elements. It runs via Node.js:
– Windows: `winget install OpenJS.NodeJS.LTS`
– macOS: `brew install node`
– Linux: `curl -o- | bash`
Install and launch the MCP server like this:
“`python
from agents.mcp import MCPServerStdio
playwright_server = MCPServerStdio(
name=”Playwright MCP”,
params={
“command”: “npx”,
“args”: [
“-y”,
“@playwright/mcp@latest”,
“–browser”,
“chrome”,
],
},
)
“`
When the agent first calls a browser tool, Chrome opens visibly, and the agent can observe and act on the page.
—
### 3. Running the Agent
Define the task and launch the agent:
“`python
APP_URL = “http://localhost:8000″
TASK = f”””
Open {APP_URL} and resolve the support case for order ORD-1042.
The customer says they received the wrong item. Use the information available in the
application to determine and apply the appropriate resolution. Add a concise internal
note and make sure the resolution was successfully recorded.
Report what you did when the task is complete.
“””.strip()
from agents import Runner
async with playwright_server:
result = await Runner.run(
agent,
TASK,
max_turns=20,
)
print(result.final_output)
“`
During execution, you can watch Chrome automate the console. In my run, the agent:
– Located the case for **ORD-1042**.
– Inspected order details, customer request, inventory status, and resolution policies.
– Determined a **replacement** was appropriate.
– Added an internal note and submitted the resolution.
– Confirmed the update in the audit log.
Final summary:
“`
Resolved CASE-4107 for order ORD-1042 with Replacement.
The case now shows Resolved, the recorded action is Replacement,
and the audit log contains the corresponding resolution entry.
“`
You can also inspect tool calls and their outputs:
“`python
for item in result.new_items:
print(type(item).__name__, item)
“`
—
### FAQ
**Q: What do I need to run this example?**
A: You need Python, Node.js, the OpenAI Agents SDK, and Playwright MCP. Install dependencies with `pip install openai-agents` and ensure `npx` can find `@playwright/mcp`.
**Q: Can I use a different browser?**
A: Yes. Change the `–browser` argument to `firefox` or `webkit` in the MCP server configuration.
**Q: What if my site requires authentication?**
A: You can extend the agent instructions to include login steps, or modify the MCP configuration to handle authentication cookies before starting the task.
**Q: Is coordinate‑based mouse/keyboard needed for this use case?**
A: Not here. Structured accessibility snapshots and element‑targeted actions were sufficient, which keeps the interaction robust and portable.
**Q: How many agent turns are reasonable?**
A: Set a `max_turns` limit (e.g., 20) to prevent infinite loops. Complex workflows may need more turns, but well‑designed instructions keep usage low.
—
### Conclusion
Browser‑using agents unlock powerful automation for support, operations, and sales workflows that still live in web interfaces. By combining structured page observations with element‑targeted actions, agents can reliably navigate, investigate, and complete tasks—just as we saw with the support console case study.
The same observe–decide–act loop generalizes to computer‑level automation by swapping structured observations for screenshots and element targets for coordinate‑based mouse and keyboard input. With the OpenAI Agents SDK and Playwright MCP, you have a flexible foundation for building both browser‑specific and broader computer‑use agents.
