Research Talk: Semantic Probabilistic Layers for Neuro-Symbolic Learning

Extending current deep learning architectures to correctly conform to prior knowledge and capture symbolic constraints while guaranteeing their satisfaction is a big open problem. Doing so would benefit many structured-output prediction (SOP) tasks in machine learning such as predicting rankings, hierarchies and other structured objects. In this talk, I will discuss how we can design a predictive layer for SOP that guarantees its predictions to be consistent with a set of predefined symbolic constraints. This Semantic Probabilistic Layer (SPL) can act as a drop-in replacement of existing predictive layers while still being amenable to end-to-end learning via exact maximum likelihood. SPLs combine exact probabilistic inference with logical reasoning in a clean and modular way via probabilistic circuits. We empirically demonstrate that SPLs outperform other neuro-symbolic approaches in terms of accuracy on challenging SOP tasks including hierarchical multi-label classification, pathfinding and preference learning, while retaining perfect constraint satisfaction.

Antonio Vergari is a Lecturer (Assistant Professor) in Machine Learning at the University of Edinburgh. His research focuses on efficient and reliable machine learning in the wild; tractable probabilistic modeling and combining learning with complex reasoning. He recently was awarded an ERC Starting Grant on automating probabilistic reasoning for trustworthy ML. Previously he was postdoc in the StarAI Lab lead by Guy Van den Broeck at UCLA. Before that he did a postdoc at the Max Planck Institute for Intelligent Systems in Tuebingen in the Empirical Inference Department of Bernhard Schoelkopf. He obtained a PhD in Computer Science and Mathematics at the University of Bari, Italy. He likes to tease and challenge the probabilistic ML community at large on how we desperately need reliable ML an AI models nowadays. To this extent, he organized a series of tutorials, workshops, seminars and events at top ML and AI venues such as UAI, ICML, AAAI, IJCAI and NeurIPS and last year a Dagstuhl Seminar.