As AI systems continue to shape various industries, from healthcare to finance and beyond, ensuring their reliability, efficiency, and sustainability has never been more critical. While deep neural networks have driven groundbreaking advancements, their success has largely been fueled by heuristic techniques, ever-growing datasets, and increasing computational resources. However, this approach raises concerns about scalability, energy consumption, and our ability to truly control and predict AI behavior. Linara’s research aims to address these pressing issues by developing a more principled understanding of deep learning generalization, bridging the gap between theoretical insights and real-world applications.

Deep neural networks are the most trendy class of machine learning models and are believed to be the foundation for creating general artificial intelligence. The basis of the current successful trajectory of development is heuristic techniques for improving existing neural network architectures and the investment of more significant resources, both in data and computational power. Further progress requires constant effort to design and test heuristics and unsustainably more data and energy. To develop deep neural networks more reliably, make them sustainable, and guarantee their performance, we need to understand why and when they work. In my thesis, I improve the understanding of deep neural networks used in practice. The challenge here is twofold: (i) to find a way to describe an aspect of deep learning generalization ability mathematically precisely and formulate a conjecture that can be verified formally, and (ii) to use this conjecture in real-world deep neural networks to intuitively explain phenomena occurring during training or inference. I have selected two approaches to develop this understanding: the information-theoretical perspective and loss surface geometry. Additionally, I investigated federated deep learning as a widely used practical solution for training models in real-world scenarios.