Перейти к содержимому

Computer Use (Browser) in Microsoft Agent Framework

AI in C# (Rasmus Wulff Jensen)

0:00 / 0:00

Computer Use (Browser) in Microsoft Agent Framework

266 просмотров · 10 ч назад
AI in C# (Rasmus Wulff Jensen)
4,04 тыс. подписчиков
266 просмотров · 10 ч назад
Give an AI agent browser tools with Playwright and Microsoft Agent Framework in C#. This walkthrough builds a browser automation sample, demonstrates form filling and Google navigation, and compares ARIA snapshots with raw HTML for token usage. Source code: https://github.com/rwjdk/agent-framew... You'll learn how to: Initialize Playwright and expose browser actions as agent tools. Read ARIA snapshots, discover labeled controls, and verify results after actions. Fill fields, click buttons, navigate pages, and submit searches with keyboard input. Handle cookie dialogs and report CAPTCHA blockers without claiming success. Compare the token usage of accessibility snapshots and full HTML. Run Chromium headlessly and understand this sample's practical limitations. The local newsletter demo succeeds; Google's unusual-traffic verification blocks the search-result check. This is a starting sample for light browser automation, with discussion of why a fuller browser implementation may be preferable. Full course — AI in C# using the Microsoft Agent Framework: https://www.udemy.com/course/ai-in-c-... Microsoft Agent Framework playlist:    • Microsoft Agent Framework for C#: All videos   Chapters 00:00 Browser computer use with Playwright 00:27 Project setup and Chromium initialization 01:46 Local newsletter page and agent cursor 02:57 Agent setup, instructions, and safety rules 04:18 Browser tools and ARIA snapshots 05:32 Navigate, fill fields, click, and press keys 08:19 Define the agent's two browser tasks 09:25 Run the demo and encounter Google's CAPTCHA 10:41 Debug the tool calls and page snapshots 14:50 Token usage: ARIA snapshots versus raw HTML 17:01 Limitations and extending the tools 18:00 Run the browser headlessly 18:34 When to use a fuller browser implementation GitHub: https://github.com/rwjdk #MicrosoftAgentFramework #Playwright #CSharp