LionBook Chapter 4: Linear Models
The deep reason why linear models work well is the smoothness underlying many physical phenomena. So when confronted with a difficult problem, try linear equations first and you are likely to either solve it or at least come up with a workable approximation.
Here is the latest chapter from LIONbook, a new book dedicated to "LION" combination of Machine Learning and Intelligent Optimization, written by the developers of LionSolver software, Roberto Battiti and Mauro Brunato.
This book will available for free from the web, chapter after chapter.
Here are previous chapters:
and the latest Chapter 4: Linear models
Here is an excerpt:
Just below the mighty power of optimization lies the awesome power of linear algebra. Do you remember your teacher at school: "Study linear algebra, you will need it in life"?
Well, with more years on your shoulders you know he was right. Linear algebra is a "math survival kit," when confronted with a difficult problem, try with linear equations first. In many cases you will either solve it or at least come up with a workable approximation.
... The deep reason why linear models are so popular is the smoothness underlying many if not most of the physical phenomena ("Nature does not make jumps"). An example is in Fig. 4.5, the average stature of kids grows gradually, without jumps, to slowly reach a saturating stature after adolescence.
After reading Chapter 4, do an exercise on linear regression and classification (handwritten digits).