Machine Learning is taking the world by storm and its use in critical applications requires trustwothy methods. Almost all theory and methods, however, are inherently flawed in such a basic way that it prevents them from being used in practice. Unlike what most papers assume, in many applications (e.g., autonomous driving, industrial machines, or healthcare) it is impossible or hugely impractical to gather all data into one place. Besides the sheer size of data, the main factor preventing its centralization is data privacy and security. Federated learning offers a solution: models are trained only locally and combined to create a well-performing joint model – without sharing data. This, however, compromises model quality and could still infringe data privacy. Ensuring data privacy and giving a guarantee on the model quality and training efficiency is a huge undertaking. In this talk, I will explain federated learniing and the challenge of trustwortyness, and explore challenges and open problems.