Computer Use (Browser) in Microsoft Agent Framework
AI in C# (Rasmus Wulff Jensen)
0:00 / 0:00
Computer Use (Browser) in Microsoft Agent Framework
266 просмотров · 10 ч назад
AI in C# (Rasmus Wulff Jensen)
4,04 тыс. подписчиков
266 просмотров · 10 ч назад
Give an AI agent browser tools with Playwright and Microsoft Agent Framework in C#. This walkthrough builds a browser automation sample, demonstrates form filling and Google navigation, and compares ARIA snapshots with raw HTML for token usage.
Source code: https://github.com/rwjdk/agent-framew...
You'll learn how to:
Initialize Playwright and expose browser actions as agent tools.
Read ARIA snapshots, discover labeled controls, and verify results after actions.
Fill fields, click buttons, navigate pages, and submit searches with keyboard input.
Handle cookie dialogs and report CAPTCHA blockers without claiming success.
Compare the token usage of accessibility snapshots and full HTML.
Run Chromium headlessly and understand this sample's practical limitations.
The local newsletter demo succeeds; Google's unusual-traffic verification blocks the search-result check. This is a starting sample for light browser automation, with discussion of why a fuller browser implementation may be preferable.
Full course — AI in C# using the Microsoft Agent Framework:
https://www.udemy.com/course/ai-in-c-...
Microsoft Agent Framework playlist:
• Microsoft Agent Framework for C#: All videos
Chapters
00:00 Browser computer use with Playwright
00:27 Project setup and Chromium initialization
01:46 Local newsletter page and agent cursor
02:57 Agent setup, instructions, and safety rules
04:18 Browser tools and ARIA snapshots
05:32 Navigate, fill fields, click, and press keys
08:19 Define the agent's two browser tasks
09:25 Run the demo and encounter Google's CAPTCHA
10:41 Debug the tool calls and page snapshots
14:50 Token usage: ARIA snapshots versus raw HTML
17:01 Limitations and extending the tools
18:00 Run the browser headlessly
18:34 When to use a fuller browser implementation
GitHub: https://github.com/rwjdk
#MicrosoftAgentFramework #Playwright #CSharp