Understanding Machine Learning: The Free eBook
Time to get back to basics. This week we have a look at a book on foundational machine learning concepts, Understanding Machine Learning: From Theory to Algorithms.
Welcome to another installment of these weekly KDnuggets free eBook overviews.
While we have detoured into specialized topics over the past several weeks, including some which are more advanced in nature, we felt it was time to bring it back to basics, and have a look at a book on foundational machine learning concepts. This week we introduce Understanding Machine Learning: From Theory to Algorithms, by Shai ShalevShwartz and Shai BenDavid.
Directly from the book's website:
The aim of this textbook is to introduce machine learning, and the algorithmic paradigms it offers, in a principled way. The book provides a theoretical account of the fundamentals underlying machine learning and the mathematical derivations that transform these principles into practical algorithms. Following a presentation of the basics, the book covers a wide array of central topics unaddressed by previous textbooks.
[...]
Designed for advanced undergraduates or beginning graduates, the text makes the fundamentals and algorithms of machine learning accessible to students and nonexpert readers in statistics, computer science, mathematics and engineering.
This book is explanatory in nature, and focuses on the theory of a variety of machine learning concepts. There is no code to see here; you aren't writing algorithms from scratch, nor are you using existing libraries to implement any. This is strictly for learning theory.
The book is split into four distinct parts, as outlined below:
Part I: Foundations
 A gentle start
 A formal learning model
 Learning via uniform convergence
 The biascomplexity tradeoff
 The VCdimension
 Nonuniform learnability
 The runtime of learning
Part II: From Theory to Algorithms
 Linear predictors
 Boosting
 Model selection and validation
 Convex learning problems
 Regularization and stability
 Stochastic gradient descent
 Support vector machines
 Kernel methods
 Multiclass, ranking, and complex prediction problems
 Decision trees
 Nearest neighbor
 Neural networks
Part III: Additional Learning Models
 Online learning
 Clustering
 Dimensionality reduction
 Generative models
 Feature selection and generation
Part IV: Advanced Theory
 Rademacher complexities
 Covering numbers
 Proof of the fundamental theorem of learning theory
 Multiclass learnability
 Compression bounds
 PACBayes
In its introduction, the book sets out the following pair of goals:
The first goal of this book is to provide a rigorous, yet easy to follow, introduction to the main concepts underlying machine learning: What is learning? How can a machine learn? How do we quantify the resources needed to learn a given concept? Is learning always possible? Can we know if the learning process succeeded or failed?
The second goal of this book is to present several key machine learning algorithms. We chose to present algorithms that on one hand are successfully used in practice and on the other hand give a wide spectrum of different learning techniques.
During the course of explaining its concepts, the book relies heavily on maths in doing so. In reality, thoroughly understanding the theoretical underpinnings of these machine learning concepts is not possible without mathematics, but this is often a surprise and can be overwhelming to some, so consider yourself warned.
Once the possible shock of mathheavy theory wears off, you will find thorough treatments of topics from biasvariance tradeoff to linear regression to model validation strategies to model boosting to kernel methods to prediction problems and beyond. And the benefit of such a thorough treatment is that your understanding will go deeper than just grasping the abstract intuition.
You can directly download a PDF of the book here.
If you are in the market for a rigorous deep dive into learning concepts and the theory of deep learning, be sure to add Understanding Machine Learning: From Theory to Algorithms to your short list.
Related:
 The Elements of Statistical Learning: The Free eBook
 Mathematics for Machine Learning: The Free eBook
 A Concise Course in Statistical Inference: The Free eBook
Top Stories Past 30 Days

