Deep Feelings On Deep Learning
A thoughtful opinion piece on deep learning and its role in Strong AI. A pragmatic view of deep learning and its comparison to competing learning strategies is presented.
By Carlos Argueta, Soul Hacker Labs.
So I want to build Artificial Emotional Intelligence (AEI), and I already wrote about a possible application to treat mental health problems. Even the big guns like Apple Inc. are trying to build AEI (for some obscure reason). So the obvious step when you want to build something is to study and to do research.
Bogbot, Thomas Eykemans / Monocol.
As much as I tried not to fall for the hype recently gained by Deep Learning, I could not really resist to explore their promises. Let me quickly explain. In order to build real AEI I wanted to start by the component that can understand our words. This belongs to the fields called Natural Language Processing (NLP), and Computational Linguistics (CL). Building powerful and useful NLP/CL systems is extremely challenging. It took me nearly 3 years to build a system that can guess your emotions from what you write, and the accuracy is far from perfect. The reason is that such systems are traditionally built using manually defined rules, features, and algorithms tailored for specific tasks.
Deep Learning, on the other hand, promises to replace handcrafted features with efficient algorithms able to “learn” the features automatically from some input data, saving you all the hard work. So yeah! When you think about this it makes sense to want to give it a try. And so I did. First I studied the basics of Artificial Neural Networks using the awesome Coursera Machine Learning Course. Then, to complement that knowledge I read this great online book, and checked these fantastic video tutorials. All that taught me to play with toy Deep Networks on code fully written by me. When I was ready I jumped to TensorFlow, a full-fledged Deep Learning software library and followed their tutorials to train Deep Networks to classify handwritten characters. My reaction? A rush of elation followed by a bit of disappointment.
Don’t take me wrong, Deep Learning is awesome. There is mathematical proof that in theory they can solve any problem. The handwritten characters classification tutorial, although simple, hints to that. Yet there is something about Deep Learning that leaves a sour taste in the mouth. During my previous research project, I always felt I was in control, and in most cases I could justify why things worked. With Deep Learning, it all felt like magic. Except for the valid mathematical intuition, you can’t really understand what’s going on inside the black box that is the constructed Networks. Moreover, even the state-of-the-art systems were constructed in an empirical way, by testing different network architectures until finding the best performer, with little clue of why it performs better.
This per se is not a problem, and it becomes more of a philosophical matter. To understand my mixed feelings, let me ask a fundamental question related to my task of creating AEI: Do I want to explore and see the possibility of creating machines that can feel real empathy, or do I just want to automate the process of detecting positive/negative sentences with high accuracy? Deep Learning can already outperform humans (in certain situations) for the latter. Recently all kinds of single-task emotion-related algorithms are being developed to target us with better ads, to detect if we like what we see in store displays, etc. All this is exciting, but Deep Learning has no clue about real empathy.
To build real empathy, it is necessary to build artificial general intelligence (Strong AI is one of the terms in vogue) that goes beyond achieving high accuracy at small predefined tasks, and into machines that exhibit behavior at least as skilful and flexible as humans do. Although achieving such level of intelligence in a machine might be impossible, even dangerous, I believe it is absolutely necessary to try, if we really want to tackle important problems like treating depression with machines. The ability of Deep Learning to deliver on their promises of cracking small automated tasks, plus the potential economic benefits, are luring more and more talented people in the other direction, leaving just a few “crazy ones” with the thirst for deep understanding, and that is a bit disappointing.
So yes, Deep Learning can solve complex problems, yes it can save time and effort, yes it can make you rich if you automate a mundane task, but without clear understanding of what is going on inside, it might lead to many frustrations in the process of creating a system able to interact naturally with humans. At the end of the day, it’s all a matter of asking yourself what your ultimate goal is. I will keep trying to understand the capabilities and limitations of this great tool, and as soon as I move past the tutorials and into developing the first part of my AEI, I will post more about my feelings towards Deep Learning.
Bio: Carlos Argueta is a Honduran entrepreneur residing in Taiwan. He holds a Ph.D. in Information Systems and Applications and is the co-founder of Veryfast Inc. and Soul Hackers Labs.
Adapted by Carlos Argueta from his original.
Related:
So I want to build Artificial Emotional Intelligence (AEI), and I already wrote about a possible application to treat mental health problems. Even the big guns like Apple Inc. are trying to build AEI (for some obscure reason). So the obvious step when you want to build something is to study and to do research.
Bogbot, Thomas Eykemans / Monocol.
As much as I tried not to fall for the hype recently gained by Deep Learning, I could not really resist to explore their promises. Let me quickly explain. In order to build real AEI I wanted to start by the component that can understand our words. This belongs to the fields called Natural Language Processing (NLP), and Computational Linguistics (CL). Building powerful and useful NLP/CL systems is extremely challenging. It took me nearly 3 years to build a system that can guess your emotions from what you write, and the accuracy is far from perfect. The reason is that such systems are traditionally built using manually defined rules, features, and algorithms tailored for specific tasks.
Deep Learning, on the other hand, promises to replace handcrafted features with efficient algorithms able to “learn” the features automatically from some input data, saving you all the hard work. So yeah! When you think about this it makes sense to want to give it a try. And so I did. First I studied the basics of Artificial Neural Networks using the awesome Coursera Machine Learning Course. Then, to complement that knowledge I read this great online book, and checked these fantastic video tutorials. All that taught me to play with toy Deep Networks on code fully written by me. When I was ready I jumped to TensorFlow, a full-fledged Deep Learning software library and followed their tutorials to train Deep Networks to classify handwritten characters. My reaction? A rush of elation followed by a bit of disappointment.
Don’t take me wrong, Deep Learning is awesome. There is mathematical proof that in theory they can solve any problem. The handwritten characters classification tutorial, although simple, hints to that. Yet there is something about Deep Learning that leaves a sour taste in the mouth. During my previous research project, I always felt I was in control, and in most cases I could justify why things worked. With Deep Learning, it all felt like magic. Except for the valid mathematical intuition, you can’t really understand what’s going on inside the black box that is the constructed Networks. Moreover, even the state-of-the-art systems were constructed in an empirical way, by testing different network architectures until finding the best performer, with little clue of why it performs better.
This per se is not a problem, and it becomes more of a philosophical matter. To understand my mixed feelings, let me ask a fundamental question related to my task of creating AEI: Do I want to explore and see the possibility of creating machines that can feel real empathy, or do I just want to automate the process of detecting positive/negative sentences with high accuracy? Deep Learning can already outperform humans (in certain situations) for the latter. Recently all kinds of single-task emotion-related algorithms are being developed to target us with better ads, to detect if we like what we see in store displays, etc. All this is exciting, but Deep Learning has no clue about real empathy.
To build real empathy, it is necessary to build artificial general intelligence (Strong AI is one of the terms in vogue) that goes beyond achieving high accuracy at small predefined tasks, and into machines that exhibit behavior at least as skilful and flexible as humans do. Although achieving such level of intelligence in a machine might be impossible, even dangerous, I believe it is absolutely necessary to try, if we really want to tackle important problems like treating depression with machines. The ability of Deep Learning to deliver on their promises of cracking small automated tasks, plus the potential economic benefits, are luring more and more talented people in the other direction, leaving just a few “crazy ones” with the thirst for deep understanding, and that is a bit disappointing.
So yes, Deep Learning can solve complex problems, yes it can save time and effort, yes it can make you rich if you automate a mundane task, but without clear understanding of what is going on inside, it might lead to many frustrations in the process of creating a system able to interact naturally with humans. At the end of the day, it’s all a matter of asking yourself what your ultimate goal is. I will keep trying to understand the capabilities and limitations of this great tool, and as soon as I move past the tutorials and into developing the first part of my AEI, I will post more about my feelings towards Deep Learning.
Bio: Carlos Argueta is a Honduran entrepreneur residing in Taiwan. He holds a Ph.D. in Information Systems and Applications and is the co-founder of Veryfast Inc. and Soul Hackers Labs.
Adapted by Carlos Argueta from his original.
Related:
- Beyond the Fence, and the Advent of the Creative Machines
- The Unreasonable Reputation of Neural Networks
- What Is Machine Intelligence Vs. Machine Learning Vs. Deep Learning Vs. Artificial Intelligence (AI)?
