Math4ML Seminar - Structures in deep neural networks by Professor Ding-Xuan Zhou

This seminar launches Math4ML's online seminar series, which aims to bring together researchers and students interested in the rich connections between mathematics and machine learning. Abstract: Deep learning has been widely applied and brought breakthroughs in speech recognition, computer vision, natural language processing, and many other domains. The involved deep neural network architectures and computational issues have been well studied in machine learning. But not much is known about the modelling, approximation or generalization abilities of deep learning models with network architectures and structures, due to some differences between the classical fully-connected neural networks and structured ones used in deep learning. An important family of structured deep neural networks are deep convolutional neural networks (CNNs) induced by convolutions. It is believed that local shift-invariance of natural images and speeches plays an essential role in the efficiency of CNNs. Another family of structured deep neural networks are transformers based on attention structures where variable dependence is believed to be crucial in processing natural language data. This talk describes approximation and generalization analysis of deep CNNs, transformers, and related structured deep neural networks.