Glimpses & Impressions: Strata Silicon Valley AI + ML Review – Part Two
Read some impressions from onsite visits to 2 companies during Strata Silicon Valley in March: Novumind and Numenta.
By Zhang Xiatian, Chief Data Scientist, TalkingData.
Editor's note: Before continuing, you may first want to read part 1 of this in-depth overview here.
II. AI Companies: NovuMind and Numenta
2.1. NovuMind
NovuMind and Numenta were the only AI companies we visited during this Silicon Valley trip, but even though we were only there for a limited time, we were still able to gain some insights on the developmental trends of AI and deep learning.
After Dr. Wu departed from Baidu, he founded the startup NovuMind to focus on deep learning and AI technologies. I accompanied TalkingData's CEO Leo Cui, Mr. Che Pingjue, and Beijing GZT Network Technology's Vice Chairman, Mr. Li Yangshong on their visit to NovuMind. Leo is an excellent go player (ranked at amateur 5-dan). In fact, it was only a twist of fate that prevented him from becoming a professional go player—though if he had, there would be no TalkingData today. At NovuMind, Leo was tempted into having a match with Dr. Wu's NovuGo. After a hard-fought and evenly matched two-hour game, Leo was able to defeat NovuGo in the end. Despite NovuGo's loss, it should be pointed out that the program was ran from a notebook with the computing power that was approximately 1/30000 of AlphaGo's. Leo finds it remarkable that NovuGo, running on a single notebook, can achieve the results of an amateur 3-dan or 4-dan. According to Dr. Wu, NovuGo is essentially built with the technology that he had previously used for image processing. Even though its search methods are not as fine-tuned as AlphaGo's, it has several percentages over AlphaGo in terms of policy network accuracy. We believe that when NovuGo refines and enhances its computing performance, it will be just as good as AlphaGo.
NovuGo is not, in fact, Dr. Wu's day job. After Dr. Wu left Baidu, many financial companies courted him to do quantitative investment to improve investment returns by using deep learning. Wishing to do something that wasn't only about making money, Dr. Wu founded NovuMind on his own in order to further develop AI technologies. Currently NovuMind has two main goals: (1) to push AI technology to new heights and (2) to popularize the application of deep learning.
As deep learning needs a tremendous amount of computing resources, Dr. Wu believes that in order for it to be more effective, we must extract the maximum amount of possible computing power from the machines. NovuMind is working on a type of all-in-one computer that combines deep learning with depth-first search, thereby maximizing algorithm efficiency. Dr. Wu is also planning to design NovuMind’s own deep learning chip that will have better compatibility with systems and algorithms. I very much share his vision, and I feel that all big data professionals should share it as well. Big data is getting even bigger at a rate that far exceeds computing power's growth rate, and we need to fully utilize all the computing resources at hand. For people who are working on machine learning, in order to making model training more effective, we need to adapt the system to the algorithm—but also adapt the algorithm to the system. We can only achieve maximum efficiency by attacking at the problem from both ends.
Deep learning has achieved resounding victories in many fields. It has broken through long-standing bottlenecks in fields such as image processing and speech recognition, its application growing wider and wider still. Thanks to deep learning, AlphaGo was even able to breach the bastion of human intelligence. There is a growing belief that deep learning is capable of anything and everything. Dr. Wu, however, believes that even though deep learning has achieved great success, its gains have mainly been in the area of perception, and it still has a long way to go when compared to human intelligence. Its powerful power of perception is growing omnipotent, and in order to use it more effectively, we need to make it omnipresent. NovuMind is working to combine deep learning's power of perception with terminal equipment, transforming the Internet of Things' emphasis on connectivity into perception. This line of thought is very similar to what Mr. Yu Kai is trying to accomplish at Horizon Robotics, and it's not surprising that two brilliant minds share the same vision.
2.2. Numenta
One of Numenta's founders is Jeffery Hawkins, one of the world's top machine learning specialists. Numenta was founded in 2005, and more than a decade later, it's still being run by a small team of a few dozen people.
Numenta is focused on developing and popularizing cortical learning technologies, and it has developed the model known as HTM (Hierarchical Temporal Memory). Although it’s still relatively unknown in China, at its heartcortical learning isa type of artificial neural network. However, when compared with the currently in-vogue deep neural network, the only real similarity between them is that they both have neurons and neuron networks. Deep neural network's training and prediction processes are, in fact, very different from the way human brains work. Deep neural network's training is completed through backpropagation, which is fundamentally about computing gradients. This is clearly not how the human brain learns. When a deep neural network is engaged in learning or prediction, all of its neurons participate in the process. In the case of human brains, usually only 1/1000th of total neurons are active at any given time. HTM, on the other hand, tries to imitate the cerebral cortex's structure and rules of operation. HTM has set up certain rules about neuron firing, including rules regarding the connection and disconnection between neurons. It learns by having a great quantity of neurons implementing these simple rules, achieving recognition and enhancement, predicting patterns in time series data. Given this article's limited length, I will not discuss HTM in details here.
In its application, Numenta mainly uses HTM for anomalies detection. During the Strata conference, Numenta employees demonstrated how it functioned for us. For a set of server operating status time series data, it took HTM less than 10 seconds to capture the abnormal patterns. As HTM is unsupervised, it closely resembles human intelligence, and this is very useful for anomalies detection. Anomalies are by definition outliers, thus it is difficult to obtain enough learning examples, and it's difficult to discover new anomalies in time. HTM's unsupervised self-learning feature both solves the problem of obtaining samples and automatically discovers new anomalies. Currently all the examples of Numenta application are used for anomalies detection for fields such as securities market, IT operation analytics, malware, and tracking anomalies in geographic positioning.
NovuMind and Numenta represent two different paths for AI development. Deep neural network has already achieved great success in both academia and industry, and NovuMind is trying to take it to the next level. Numenta, on the other hand, is forging ahead with a less-travelled path in its pursuit of building an AI that resembles the human brain.
Conclusion
Even though our Silicon Valley visit was a short one, we were still able to identify certain new technological trends. We also saw that doing big data—and IT in general—in the U.S. is quite different from doing it in China. In the U.S. there are many small start-ups, usually comprised of upwards to a few dozen employees, which are focused solely on developing new technologies. It's difficult to imagine that such startups can survive in China. None of the employees whom we interacted with talked about business models or how to make money. All they talked about were new technologies, new ideas, and how to find solutions for their clients. From the founding CEOs to the newest engineers, all of them exuded an unmistakable enthusiasm. They have passions and ideals, and perhaps these are the reasons why the U.S. remains a cradle for trend-setting new technologies while China is stuck in the role as a micro-innovator. This realization has stirred something deep within me. After nine years on the job, I find my idealism gradually giving way to practical considerations, and I have become increasingly preoccupied with the question of "worth" in everything that I do. In this respect, we should learn from the Silicon's Valleys entrepreneurs and tech professionals.
Bio: Zhang Xiatian has long engaged in data mining and machine learning research and has dozens of research papers in publication and sufficient patents. Xiatian is now fully responsible for mobile big data mining, ML algorithm research and implementation in TalkingData. He used to work for IBM China Research Institute, Tencent data platform, Huawei Noah's Ark Lab.
Related:
