OSWorld-Pro: Grading each step a computer-use agent takes, not just its final result

The paper introduces a benchmark that splits long desktop tasks into subgoals and grades each step an AI computer-use agent takes, giving partial credit and locating where failures begin. Even the best agent completes every subgoal on only 77.7 percent of tasks, and a language-model judge matches human labels 94.1 percent of the time on subgoal completion. #ComputerUseAgents #AIAgents #Benchmarking #LanguageModels #AgentEvaluation #ArXivSignals