Intel’s Investments in Cognitive Tech: Impact and New Opportunities

An overview of Intel's recent investments in cognitive technology, the impact of these investments on technology and research, and the new opportunities these investments present.

By Peter Zhegin, Flint Capital.

Intel logoIntel is the company which is enormously contributing to development of cognitive technologies. Being the leader of Top500 supercomputer sites, the firm shapes the future of high performance computing (HPC) and the wider ecosystem (Table 1). Analysis investment activity of Intel may help to spot emerging trends in cognitive tech.

Intel top 500 supercomputers

To date Intel has finalized at least 20 transactions aiming to acquire or invest in assets directly or indirectly related to various areas of cognitive tech, including natural language processing, speech recognition, robotics and others. The largest bets were placed on assets that at first glance have no clear connection with cognitive technologies (e.g. interconnects). However, exactly these assets may shape the future of the cognitive tech (Table 2).

Machine learning-related assets

The Intel’s CEO Brian Krzanich has recently outlined the future of the company. Machine learning looks to be an integral part of his statement, even though it was mentioned only twice. Machine learning links some other pillars, mentioned by the CEO, namely cloud/data center, connected things and connectivity[1]. Data centers are places where machine learning takes place. The cloud is a medium to share outcomes of machine learning with connected things that may become active learners themselves. That is a story Intel may bring to life.

Intel’s investments and acquisitions are aligned with the recent CEO’s statement. For example, the company invests in infrastructure that makes data centers more cognitive tech friendly, e.g. field-programmable gate arrays (FPGAs) and interconnects. Intel also backs technologies that have a potential to bring new computing capabilities to connected things, including computer vision, speech recognition, advanced analytics.

A push towards machine learning by a chip giant opens new opportunities for the cognitive tech community. Better, and presumably cheaper, hardware for data centers leads to a faster adoption of cognitive technologies by corporates. Technological changes in hardware, for instance diversification of FPGAs’ in new verticals, create new challenges startups may find relevant to seek a solution for.

New Workloads in Data Centers and in the Cloud

In 2015, Intel made its largest acquisition by closing a transaction with Altera, a FPGAs manufacturer, valuing it around $17B. The reasoning behind the transaction and its timing are to be explored and understood by taking into account that back in 2010, Christopher Danely, the analyst of JP Morgan Securities, highlighted Intel’s considerations to acquire an FPGAs vendor[2]. The question is what motivated Intel to make this kind of a bold move in 2015.

A possible incentive to jump into the FPGA segment is that it is a new emerging market, when tech companies’ data centers are experiencing a new type of a workload. Workloads related to cognitive technologies, e.g. machine learning, speech and images recognition are becoming more notable, but not yet large though.

New types of workloads currently represent a marginal share of data centers’ work, however, it is expected to grow. For instance, the number of servers used for running deep learning at Microsoft ‘…is at most in the single digits percentage of all workload’[3]. In 2014, ~10% of search queries at Baidu were done with a voice. Voice queries are expected to achieve 50% threshold by 2020[4].

Some of the cognitive tech workloads are mature enough to be allocated to different types of servers. For example, powerful graphic processing units (GPUs) assist with neural networks training, and traditional central processors execute production workloads.

At the same time, a larger part of the deep learning network design is still ‘empirical’, demands much experimentation, larger-than-optimal datasets, and therefore more power. The question of process acceleration is still relevant.

Important thing to highlight is that the cloud is critical for companies that deal with new cognitive tech workloads. As current endpoints can hardly provide required processing power for such workloads as machine learning, delivering it through the cloud is deemed significant.

Unlike in the situation with traditional workloads, where the cloud is sometimes a ‘nice to have option’, the cloud is a ‘must have option’ for cognitive tech. For instance, one potentially may run a CRM on a desktop, without going to Salesforce for its cloud products. At the same time, training of a neural network is hardly possible without relying on powerful servers of a provider. Intel estimates that a third of cloud providers will be using FPGA accelerated server nodes by 2020[5].

As new types of workloads proliferate, customization of infrastructure also follows.

As Hamant Dhulla, the vice president of the data center division at Xilinx, a FPGAs maker, puts it, ‘[we] are seeing that these data centers are separated into ‘pods’ or multiple racks of servers for specific workloads. For instance, some have pods set aside to do things like image resizing, as an example…’[6]

Intel’s Push into FPGAs and Challenges Associated with it

Required level of specialization of data center hardware may be achieved via FPGAs. Eric Chung at Microsoft Research explained to Thenextplatform the rationale behind using FPGAs, its power profile and flexibility in particular.

According to Mr. Chung, FPGAs represent the balance ‘between something that is general purpose and specialized hardware’ even if its peak performance is lower than for GPUs, and there are difficulties to program it[7].

Microsoft for example uses FPGAs to speed up Bing[8]. Baidu was also experimenting with FPGAs in traditional search, image and speech recognition workloads[9].

One may expect that with Intel’s help Altera’s FPGA will diversify into new verticals and functions. For instance, FPGAs approach natural language processing, medical imaging or network packet processing. In some cases FPGAs are employed on a storage end, while in others they get some computing workloads[10].

Intel’s push into FPGA does not seem to be a reactive answer to the market demand, but a logical move. During the last five years, Intel has established a strong position in Top500 list not only as a provider of CPUs, but as a provider of co-processors accelerating them. Intel’s Xeon Phi co-processors accelerated 27 out of 500 of the most powerful computers in November 2015 (Table 3). Having an understanding of the importance of hybrid computing, Intel has made a move into FPGAs.

Accelerators by Intel and Nvidia in top 500 supercomputers

However, there are concerns about using FGPAs for cognitive tech workloads. As of November 2015, there were no systems among Top500 the most powerful computers that were using FPGA acceleration.

Urs Hölzle, the senior vice president of the technical infrastructure team at Google, mentioned that FPGAs are more a ‘niche thing’ as they ‘… are much harder to program than a CPU… you use them in a place where you don’t have a choice’[11]. FPGA programming languages, Verilog and VHD, are not the easiest ones to learn and alternative solutions like OpenCL for Altera’s FPGAs and C for Xilinx’s FPGAs do not look optimal[12].

Moreover, in addition to challenges with programming, data movement and memory are the other concerns, when CPUs are accelerated by GPUs or FPGUs[13]. For example, a bottleneck appears at a PCI-express bus. NVidia explains: ‘…GPUs are connected to x86-based CPUs through the PCI Express (PCIe) interface, which limits the GPU’s ability to access the CPU memory system and is four- to five-times slower than typical CPU memory systems[14]. Connecting of FPGAs with CPUs is also challenging[15].

Both challenges, programming of FPGAs and data movement/memory, are on Intel’s radar and agenda.

For tackling the programming challenge, Intel develops a suite of FPGA libraries, including standard acceleration for cloud, networking and traditional enterprise. Machine learning applications are also considered as interesting for the company. Intel expects to bolster programmability and cloud use of FPGAs via the suite [16].

Technologies acquired with Interconnect Technology Business of Cray and InfiniBand Business of QLogic may be helpful in integrating CPUs with FGPAs and solving data flows/memory issues[17].In the future, Intel expects to locate FPGA and CPU on one single die. In addition, Intel seems to have capabilities required to develop clusters with FPGAs based on a remote direct memory access (RDMA) via InfiniBand.

Improving capability in data center business is not the only aim of Intel’s investment activity. Intel has acquired Xtremeinsights, a consulting company, in order to promote Intel’s distribution of Apache Hadoop used in advanced analytics and machine learning[18].

Moreover, the company acquired assets in navigation, computer vision and natural language processing/speech recognition. These acquisitions are important for executing Intel’s strategy in connected things.