Deep Learning Finds What Makes a Good #selfie
Ever wonder how a convolutional neural network would rate your selfies? Well, wonder no more!
By Matthew Mayo.
Andrej Karpathy has a new deep learning blog post. And it's a good one.
Andrej starts out by reminding us that, yeah, convolutional neural nets can do some useful stuff, but since we're all mostly busy taking pics of ourselves with our phones he's gone ahead and trained one to classify good and bad selfies. What a time to be alive.
They do actual work, too.
He goes on to cover the basics of convolutional neural nets (LeCun through ImageNet, more or less), and provides us with his personal recollection of the astonishment surrounding the ImageNet 2012 results. Under the hood, convolutional neural networks represent images with pixel-by-pixel arrays, and cover these image representations with convolutions, or filters. These filters then respond in different ways to different image features, and when applied in succession result in a new "image" encapsulating the various filter responses. He then reminds us that convolutional neural networks are simple, fast, and, importantly, they work. For further information on convolutional neural networks, see Karpathy's notes for everything you need to get started.
Andrej then goes on to explain how he grabbed ~5 million images labeled #selfie, weeded out those without easily-identifiable faces, and automatically scored the remaining ~2 million as good or bad based on a clever popularity algorithm (the more likes or the more views the higher the score). What was left were a million good and a million bad selfies, which were used to train the neural network.
The top 100 selfies are shown, and most folks would probably intuitively agree that these are decent selfies, ignoring personal preferences of physical appearance.
Andrej provides his advice for taking good selfies, based on generalizations of the top 100:
Some of the top 100 "good" selfies.
For comparison, bad selfies seemed to have the following in common:
#BadSelfies :(
The trained neural net was then tested on celebrities, just for fun. All sorts of comments could be made here. Inappropriate comments. Instead I'll note that there were an awful lot of duck faces in the bottom quartile. Just sayin'...
He then ran the images through the t-SNE algorithm for some interesting visual clustering, and explores some comparisons of cropped selfie shots... with some odd results. You can see yourself how the network performs by tweeting a selfie to its Twitter bot @deepselfie.
Andrej concludes the light but informative post by listing some additional deep learning resources. We won't list them here, but the bottom of the post in question has a very concise collection indeed.
Hey, it could be worse...
After trying to contact the bot a few times, it finally responded after I begged. I blame my score on the lighting. And my prominent forehead. And the fact that I'm a dude.
Andrej has written another interesting article that captures the attention, teaches a few things, and, most importantly, shows us that we can have a bit of fun while increasing our understanding of machine learning.
Bio: Matthew Mayo is a computer science graduate student currently working on his thesis parallelizing machine learning algorithms. He is also a student of data mining, a data enthusiast, and an aspiring machine learning scientist.
Related:
Andrej Karpathy has a new deep learning blog post. And it's a good one.
Andrej starts out by reminding us that, yeah, convolutional neural nets can do some useful stuff, but since we're all mostly busy taking pics of ourselves with our phones he's gone ahead and trained one to classify good and bad selfies. What a time to be alive.
They do actual work, too.
He goes on to cover the basics of convolutional neural nets (LeCun through ImageNet, more or less), and provides us with his personal recollection of the astonishment surrounding the ImageNet 2012 results. Under the hood, convolutional neural networks represent images with pixel-by-pixel arrays, and cover these image representations with convolutions, or filters. These filters then respond in different ways to different image features, and when applied in succession result in a new "image" encapsulating the various filter responses. He then reminds us that convolutional neural networks are simple, fast, and, importantly, they work. For further information on convolutional neural networks, see Karpathy's notes for everything you need to get started.
Andrej then goes on to explain how he grabbed ~5 million images labeled #selfie, weeded out those without easily-identifiable faces, and automatically scored the remaining ~2 million as good or bad based on a clever popularity algorithm (the more likes or the more views the higher the score). What was left were a million good and a million bad selfies, which were used to train the neural network.
The top 100 selfies are shown, and most folks would probably intuitively agree that these are decent selfies, ignoring personal preferences of physical appearance.
Andrej provides his advice for taking good selfies, based on generalizations of the top 100:
- Be female!
- Face = 1/3 of image
- Don't show entire forehead (???)
- Show long hair
- Oversaturate the face (lighting leads to uniform look to face)
- Put a filter on it (B&W or decreased contrast)
- Add a border
Some of the top 100 "good" selfies.
For comparison, bad selfies seemed to have the following in common:
- Low lighting
- Disproportionately large head
- Group shots
#BadSelfies :(
The trained neural net was then tested on celebrities, just for fun. All sorts of comments could be made here. Inappropriate comments. Instead I'll note that there were an awful lot of duck faces in the bottom quartile. Just sayin'...
He then ran the images through the t-SNE algorithm for some interesting visual clustering, and explores some comparisons of cropped selfie shots... with some odd results. You can see yourself how the network performs by tweeting a selfie to its Twitter bot @deepselfie.
Andrej concludes the light but informative post by listing some additional deep learning resources. We won't list them here, but the bottom of the post in question has a very concise collection indeed.
Hey, it could be worse...
After trying to contact the bot a few times, it finally responded after I begged. I blame my score on the lighting. And my prominent forehead. And the fact that I'm a dude.
Andrej has written another interesting article that captures the attention, teaches a few things, and, most importantly, shows us that we can have a bit of fun while increasing our understanding of machine learning.
Bio: Matthew Mayo is a computer science graduate student currently working on his thesis parallelizing machine learning algorithms. He is also a student of data mining, a data enthusiast, and an aspiring machine learning scientist.
Related:
