Machine learning with generalised symmetries

This thesis explores and expands the rapidly growing field of geometric deep learning, with a particular focus on neural networks that are equivariant to spaces with group actions. Our work is divided into two parts.
In Part I, we extend existing approaches by introducing new groups, actions, and architectures. We begin by developing neural networks designed for meshes, which are equivariant to the symmetry of rotations of the bases of the tangent plane. This is followed by the introduction of more scalable equivariant architectures for 3D data, leveraging the power of transformers and geometric algebras. Additionally, we present more efficient samplers for quantum systems that are invariant to symmetries, further broadening the applicability of geometric deep learning techniques. In all cases, we find compelling evidence that the incorporation of symmetries into neural networks improves their performance.
Part II of this thesis broadens the scope of geometric deep learning from groups to groupoids. We first apply this concept to causal representation learning, where we identify a groupoid of equivalent models, facilitating the identification of model classes from data. Furthermore, we generalize group equivariance to natural transformations on groupoids, proposing a novel framework we term natural deep learning. We theoretically analyze the space of natural transformations in general, and explore applications to graph-structured data. Finally, we combine our groupoid-based approach with the powerful method of message passing, allowing us to subsume and formalize many previous methods while also inspiring new ones.