LIONbook Chapter 9: Neural networks, shallow and deep
The LIONbook on machine learning and optimization, written by co-founders of LionSolver software, is provided free for personal and non-profit usage. Chapter 9 looks at Neural networks, shallow and deep.
Here is the latest chapter from LIONbook, a new book dedicated to "LION" combination of Machine Learning and Intelligent Optimization, written by the developers of LionSolver software, Roberto Battiti and Mauro Brunato.
This book will available for free from the web, chapter after chapter.
Here are previous chapters:
- Chapters 1-2: Introduction and nearest neighbors.
- Chapter 3: Learning requires a method
- Chapter 4: Linear models
- Chapter 5: Mastering generalized linear least-squares
- Chapter 6: Rules, decision trees, and forests
- Chapter 7: Ranking and selecting features
- Chapter 8: Specific nonlinear models
You can also download the entire book here.
The latest chapter is Chapter 9: Neural networks, shallow and deep.
... Our wet neural system, composed of about 100 billion (100,000,000,000) computing units and about 1015 (1,000,000,000,000,000) connections is capable of surprisingly intelligent behaviors. ...
The main mystery to be solved is how a system composed of many simple interconnected units can give rise to such incredible activities as recognizing objects, speaking and understanding, drinking a cup of coffee, fighting for your career. ...
Emergence is the way in which complex systems arise out of a multiplicity of relatively simple interactions. Similar emergent properties are observed in nature, think about snowflakes forming complex symmetrical patterns starting from simple water molecules. ...
Given that the function of a single neuron is rather simple, it subdivides the input space into two regions by a hyperplane, the complexity must come from having more layers of neurons involved in a complex action (like recognizing your grandmother in all possible situations). The "squashing" functions introduce critical nonlinearities in the system, without their presence multiple layers would still create linear functions. Organized layers are very visible in the human cerebral cortex, the part of our brain which plays a key role in memory, attention, perceptual awareness, thought, language, and consciousness. ...