Identifying Bottlenecks (slow code parts) in R using Profiling / profvis

“Premature optimization is the root of all evil (or at least most of it) in programming.” Donald Knuth, Computer Programming as an Art (1974) When R code runs too slowly, it is often only a few lines of code that really matter. Rather than trying to write optimized code from scratch, it is recommended to focus on a working, readable example first before worrying about optimization. In our example here we create scatterplots and smoothing lines, trying different variations which are likely to differ in runtimes: a linear model, several loess variants (locally weighted scatterplot smoother) with different span parameter settings, and a gam model (generalized additive model), a powerful nonlinear approach. By means of the profvis package, which is conveniently embedded in RStudio, we identify the bottleneck - which turns out to be in a different part of our code than we first expected. The profiling approach here is quicker to use than setting up several benchmarks using dedicated packages such as microbenchmark (which I used to work with) or bench (which is newer - I now prefer it). Video that contains a benchmarking example: Eliminating duplicated rows in R: Speeding up unique()    • Eliminating duplicated rows in R: Speeding...   Contact me, e. g. to discuss (online) R workshops / trainings / webinars: LinkedIn:   / wolfriepl   Twitter:   / statistikindd   Xing: https://www.xing.com/profile/Wolf_Riepl Facebook:   / statistikdresden   https://statistik-dresden.de/kontakt R Workshops: https://statistik-dresden.de/r-schulu... Blog (German, translate option): https://statistik-dresden.de/statisti... Playlist: Music chart history    • Music Chart History