SIH26171 – Vielagent: Privacy-Preserving Vision Agent for the Browser

Vielagent is a privacy-preserving browser-based vision agent designed to assist users with complex web workflows while minimizing the exposure of sensitive information. The system combines client-side visual processing with server-side AI reasoning. The browser extension analyzes the current webpage and screen context, detects sensitive information such as PII, passwords, and faces, and locally redacts or anonymizes this information before any network request is made. Only the sanitized context is transmitted to the server-side AI model. The AI interprets the anonymized screen state and generates actionable commands, such as clicking buttons, navigating pages, filling fields, or scrolling, which are then executed by the browser agent. MVP Pipeline: Browser Screen → Local Vision Processing → Privacy/PII Redaction → Sanitized Context → AI Reasoning → Browser Action The prototype demonstrates an end-to-end approach to privacy-preserving agentic AI, while considering the key requirements of visual accuracy, PII detection, redaction precision, client-side resource utilization, and end-to-end latency. Key Features: Client-side browser extension Local visual/screen analysis Automatic sensitive-data detection and redaction Privacy-preserving communication with the AI server Server-side AI/VLM reasoning AI-generated browser actions Automated execution of web tasks End-to-end agentic workflow demonstration