26. Mai 2026 |

Teaching AI to see like humans

Cyber Valley Research Fund Project draws on how the brain processes visual stimuli to improve computer vision

Fabian Sinz’s research group has successfully completed the project “Mechanisms of representation transfer” at the Institute for Bioinformatics and Medical Informatics at the University of Tübingen. The project drew on brain activity data to improve how AI models understand images. This could improve the reliability and adaptability of computer vision systems, which are used in fields as diverse as autonomous vehicles, medical imaging, and robotics.

The project was funded by the Cyber Valley Research Fund and was carried out between 2019 and 2024. It addressed one of the main challenges in computer vision research: While current AI systems are proficient in classifying objects and segmenting images, they are less effective when images are distorted. Humans, by contrast, are highly adept at processing visual information from vastly different contexts. Our brains can process and understand both familiar landscapes and entirely new environments; we can interpret both photographs and abstract paintings, even though they have drastically different characteristics.

Because we do not yet fully understand what exact features humans extract when processing visual stimuli, directly incorporating human-like visual processing strategies into artificial intelligence systems has proven challenging. Consequently, this research project took an alternative approach to improving how their AI model interprets visual data: Instead of teaching it to replicate how humans process visual stimuli, they developed a method that used data from how the brain reacts to images to guide the model’s behavior. In other words, they taught their model how to process visual data more like humans by learning from how the brain responds to images, rather than trying to copy how the brain works.

The researchers found that models trained with brain activity data alongside standard computer vision tasks demonstrated an improved ability to process visual distortions, even though they were not explicitly trained for this. These models tended to focus on parts of the images that humans find important, indicating that they were learning to process visual stimuli more like humans. Although this project was primarily concerned with basic research, its findings have potential applications in numerous fields where reliable computer vision is essential, such as autonomous driving, robotics, and medical imaging, demonstrating how AI and robotics research has the potential to help create a more sustainable future for all.

This project produced the following peer-reviewed publications:

Nix, A., Shrinivasan, S., Walker, E. Y., and Sinz, F. H. (2022). Can Functional Transfer Methods Capture Simple Inductive Biases? In International Conference on Artificial Intelligence and Statistics (AISTATS).

Pierzchlewicz, P. A., Willeke, K. F., Nix, A. F., Elumalai, P., Restivo, K., Shinn, T., Nealley, C., Rodriguez, G., Patel, S., Franke, K., Tolias, A. S., and Sinz, F. H. (2023). Energy Guided Diffusion for Generating Neurally Exciting Images. In Advances in Neural Information Processing Systems (NeurIPS).

Safarani, S., Nix, A., Willeke, K., Cadena, S. A., Restivo, K., Denfield, G., Tolias, A. S., and Sinz, F. H. (2021). Towards robust vision by multi-task learning on monkey visual cortex. In Advances in Neural Information Processing Systems (NeurIPS).

Nix, A., Burg, M., Sinz F., Leading by example: Guiding knowledge transfer with adversarial data augmentation. NeurIPS 2022 Workshop SyntheticData4ML.

About the Cyber Valley Research Fund

The Cyber Valley Research Fund was established to support Cyber Valley research groups undertake basic research in the fields of artificial intelligence and robotics. The fund totaled five million euros, including contributions from six of Cyber Valley’s founding corporate partners: Amazon, BMW, Bosch, IAV, Mercedes-Benz, Porsche, and ZF. It supported 20 research projects, the first of which began in 2020, and the final of which will conclude in 2026.