Seeing What Matters, Learning How to Find Out: Toward Visual General Intelligence by Kohsuke Ide

Which visual differences should change a model's answer, and which should not? I will discuss recent work on learning relations between 3D objects and on color-dependent judgments in vision-language models. These studies examine the importance of fine-grained geometry and the effects of changing how the same text is presented. Using these concrete examples, I will connect the task-dependent use of visual information to questions in Visual General Intelligence: A White Paper about acquiring knowledge from visual experience and applying it to new tasks. I will close with a research perspective on a further challenge: when current observations are insufficient, can a system learn how to obtain useful evidence through observation and interaction, and can those ways of finding out transfer to unfamiliar situations?