KPL2: Model Mechanics for Tree-Based Methods

This is Key-Point Lecture 2 in a series of lectures prepared for a two-week introductory course in Machine Learning at the University of Cape Town, South Africa. The course is aimed at students with some background in statistical modelling, computing, and linear algebra. KPL2 outlines the key components in the recursive partitioning procedure behind tree-based classification and regression models. This work by Etienne A.D. Pienaar is licensed under CC BY-NC-ND 4.0. 0:45 Partitioning the Feature Space: Insights From Linear Models 06:00 Non-linear decision bounds? 08:15 Partitioning 12:04 Iteration (Recursive Partitioning) 14:43 Tree Plot (Dendrogram) 16:15 Stopping Criteria 17:03 Classification 21:28 Partitioning Multiple Features As always, the relevant academic literature cited herein or for purposes of additional reading, are given: Yaser S Abu-Mostafa, Malik Magdon-Ismail, and Hsuan-Tien Lin. Learning From Data, volume 4. AMLBook New York, NY, USA:, 2012. https://work.caltech.edu/telecourse Jerome Friedman, Trevor Hastie, and Robert Tibshirani. The elements of statistical learning, volume 1. Springer series in statistics New York, NY, USA:, 2001. https://web.stanford.edu/~hastie/Elem... Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani. An introduction to statistical learning, volume 112. Springer, 2013. http://faculty.marshall.usc.edu/garet... Michael A Nielsen. Neural networks and deep learning. Determination Press, 2015. http://neuralnetworksanddeeplearning....