Training, Evaluating, and Understanding Evolutionary Models for Protein Sequences

PhD Dissertation Talk. Covers work on using methods from natural language processing, specifically large scale language models, for learning representations of protein sequences. Papers covered: Evaluating Protein Transfer Learning with TAPE (https://www.biorxiv.org/content/10.11...) Transformer protein language models are unsupervised structure learners (https://www.biorxiv.org/content/10.11...) MSA Transformer (https://www.biorxiv.org/content/10.11...) Language models enable zero-shot prediction of the effects of mutations on protein function (https://www.biorxiv.org/content/10.11...) Timestamps: 0:00 - Intro 0:56 - Evolutionary Models 11:06 - Neural Evolutionary Models 13:25 - Evaluating Protein Transfer Learning with TAPE 15:58 - Transformer protein language models are unsupervised structure learners 26:18 - MSA Transformer 33:48 - Language models enable zero-shot prediction of the effects of mutations on protein function 42:30 - Future Work 46:42 - Conclusion 47:15 - Thank yous 50:21 - Q&A Thank you very much to my advisors, John Canny and Pieter Abbeel, and to everyone who helped along the way!