Overfitting and Generalization in Linear Regression: Analysis of Coefficients

In this video we study the coefficients of linear regression models. Using real data, we show that these coefficients recover meaningful linear structure when the number of data is large, but can be very noisy otherwise. We then provide a theoretical analysis and simulated-data experiments (where the response is a noisy linear function of the features) showing that (1) ordinary-least-squares estimation is unbiased, (2) the coefficient error decreases as the number of data increase, and (3) there is large coefficient variance in directions of low feature variance. Probability and statistics book: https://a.co/d/7k259eb Website with free preprint, videos, slides and solutions to exercises: https://www.ps4ds.net Slides for this video: https://github.com/cfgranda/ps4ds/blo... Code: https://github.com/cfgranda/ps4ds/blo... https://github.com/cfgranda/ps4ds/blo...