Does Machine Learning Have a Future Role in Cyber Security?
In the past, ML learning hasn't had as much success in cyber security as in other fields. Many early attempts struggled with problems such as generating too many false positives, which resulted mixed attitudes towards it.
By Paul O'Neill, Data Analyst at Black Duck Software.
According to Google Trends, machine learning has shown a steady (almost threefold) increase in interest since 2015. Coursera and Udacity machine learning courses are both in the top ten related topics. It appears that many people want to learn more about it.
If you have ever used Google, Netflix, Amazon, Gmail, then you have interacted with machine learning (ML). It has become an important component in online retail, recommendation systems, fraud detection and others. Open source machine learning and data science tools such as Python’s Scikit-learn package are freely available, very powerful and often used to build these tools.
Early Attempts in Machine Learning
In the past, ML learning hasn't had as much success in cyber security as in other fields. Many early attempts struggled with problems such as generating too many false positives, which resulted mixed attitudes towards it. For example Simon Crosby, CTO of Bromium, has argued that while machine learning is very good at finding similarities, it is less successful at finding anomalies and therefore not suited to Cyber Security. Former Symantec CTO Amit Mital has claimed that cybersecurity is “basically broken” and machine learning is one of the few ‘beacons of hope.’ I disagree with the first opinion, it is factually incorrect. Unsupervised machine learning is capable of distinguishing outliers from normal activity.
Algorithms in Play
There are many machine learning algorithms, typically grouped as supervised or unsupervised learning. Supervised learning requires labelled training datasets and is less suited to cyber security. Unsupervised learning doesn't require labelled training data and is better suited to finding suspicious activity, including the ability to detect attacks that have never been seen before.
Amit Mital's opinion is more interesting. Is cyber security broken? I’m not a security expert, so I can’t comment on the technical aspects of security, but I can look at the facts. Every day major companies and organizations undergo attacks, and some of these attacks are successful. Cyber security is a ‘zero-tolerance field,’ by which I mean that one successful attack is a failure of the security. It seems to me that if there is even a chance that ML can help then we should investigate it.
Big Data Capabilities
In recent months some major companies have acquired machine learning capabilities. For example, Sophos acquired Invincia, Radware bought Seculert and Hewlett Packard bought Niara. This may be a sign that at least some major organisations see ML and big data capability as an important asset for the future.
It's also possible that ML will soon no longer be optional in cyber security. The internet of things will severely challenge existing cyber security methodology. Once all those potentially insecure ‘smart’ toilets and toaster ovens and so on start coming online, it will significantly increase the amount of data being produced and distributed. This may significantly increase the number of weaknesses that can be exploited, a challenge that machine learning can perhaps help with.
Machine learning provides 24/7 monitoring and handles larger data loads than a human can deal with. It still requires human intervention. ML is not really a ‘plug and play’ technology; it requires tuning to help filter real attacks from what appear suspicious but are actually benign activity, and this tuning requires human experts. I don't think even the most pro-machine people would claim that it can replace firewalls, antivirus or human security experts, but it will complement these more traditional defenses to create a more multi-layered defense.
Recent Developments
Some recent developments and improvements in cyber security machine learning include a joint effort by MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) and a ML startup called PatternEx. The result of this effort is AI2, a system that delivers a detection rate of 85%, with a five fold decrease in false positives.
Deep Learning is also something to watch for the future. This form of AI is different from ML because it's based on algorithms that mimic how neurons in the brain behave. The goal with deep learning is similar to the goal for unsupervised learning: to be able to recognize known and unknown attack types. Deep learning, however, may deal with incomplete, messy and complex data better — which is the kind of data we see in the real world.
Bio: Paul O'Neill (LinkedIn) has a background in Physics, specifically Astrophysics. These days he is a data analyst. Not just his job title, Paul is fascinated with data. His interests include data visualisation, natural language processing, machine learning, the technical aspects of working with data. But Paul's main interest is in the impact of the data revolution on humanity.
Original. Reposted with permission.
Related:
- Machine Learning and Cyber Security Resources
- The Artificial ‘Artificial Intelligence’ Bubble and the Future of Cybersecurity
- Using Machine Learning to Detect Malicious URLs