Pivotal – new Big Data spinoff from EMC, VMware

Pivotal unit, set to launch April 1, will integrate an unusual and powerful set of software components into a single big data platform.

InformationWeek, Charles Babcock, March 27, 2013

On April 1, EMC and VMware will spin out Pivotal Inc., a potential giant of the big data analytics field. They hope the company will grow to $1 billion in revenue within five years.

The budding company, which brings together a set of software components that were not originally designed to work together, will produce a data analysis platform that can capture large amounts of data in one system, address it with SQL-like queries, come up with answers in near real time and store the data in a multi-petabyte, scale-out storage system. In other words, it will comprise the best of what heretofore have been different worlds and usher VMware customers into a new era of data management. Pivotal will give VMware users a highly automated tool with which to compete with challengers such as Amazon.

To ensure that industry-neutral approach, Pivotal will be broken out of EMC and VMware as an independent company, with its own board of directors and 500 employees drawn from VMware and EMC. It will not be required to stick to a VMware-only game plan. EMC Chairman Joe Tucci is following the pattern he set with VMware, which remained independent after EMC acquired it. VMware maximized the opportunity without marching in lockstep with EMC; Pivotal will be expected to do the same.

EMC and VMware are using an assortment of components to build the new platform. One central piece is the Greenplum data warehouse system, built on the open source PostgreSQL relational database. PostgreSQL is somewhat under-productized as open source code due to the popularity of MySQL, a simpler system suited to many read-only uses. But PostgreSQL is a full ANSI-standard relational database system and Greenplum has used it to build a data warehouse system with parallel processing characteristics. That experience, it turns out, is very useful when trying to figure out how to address a large parallel system like Hadoop.

Another component is analytics startup Cetas, acquired last April, which performs fast analysis of Hadoop data. VMware will also contribute Gemfire, an in-memory data management system, to serve as a caching layer for a Pivotal analytics platform. The company gets its name from VMware acquisition Pivotal Labs, a 250-employee "extreme programming shop" in the Silicon Valley, which turned out applications by forming joint agile development teams for startups.

Read more.