LIONbook Chapter 7: Ranking and selecting features
The LIONbook on machine learning and optimization, written by co-founders of LionSolver software, is provided free on a chapter by chapter basis for personal and non-profit usage. Chapter 7 examines the process of feature selection, a key step in getting more accurate and understandable models.
Here is the latest chapter from LIONbook, a new book dedicated to “LION” combination of Machine Learning and Intelligent Optimization, written by the developers of LionSolver software, Roberto Battiti and Mauro Brunato.
This book will available for free from the web, chapter after chapter.
Here are previous chapters:
- Chapters 1-2: Introduction and nearest neighbors.
- Chapter 3: Learning requires a method
- Chapter 4: Linear models
- Chapter 5: Mastering generalized linear least-squares
- Chapter 6: Rules, decision trees, and forests
The latest chapter is
Heuristically, one aims at a small subset of features, possibly close to the smallest possible, which contains sufficient information to predict the output, with redundancy eliminated. In this way not only memory usage is reduced but generalization can be improved because irrelevant features and irrelevant parameters are eliminated. Last but not least, your human understanding of the model becomes easier for smaller models.