Fortune, Michal Lev-Ram, September 6, 2011
Data scientists have been a fixture at online companies like Google (GOOG) and Amazon (AMZN) for years. But these days organizations as diverse as Wal-Mart (WMT) and Foursquare are hiring computer science experts who can analyze all their data and provide intelligence that leads to better business decisions or new products. At Bitly, the URL-shortening service, for example, chief scientist Hilary Mason is helping the company package some of its massive volume of data into a measurement tool that Bitly customers can use to track how their content is faring online.
Data science has become such a hot field that EMC convened the first-ever data scientist summit in Las Vegas in May (300 people attended). The profession has its own blogs, including Dataists.com, founded by Bitly's Mason. And Stanford University's course on data mining is packed: More than 120 students registered last year; when it was first offered five years ago, just 20 signed up.
"That shows you the growth and interest in large-scale data mining," says course instructor Anand Rajaraman, who also runs @WalmartLabs, a division of Wal-Mart that is looking at ways to use e-commerce data to add mobile and social shopping features at its retail locations. "Companies want these people, and they become more attractive if they learn the skills."
No one currently tracks exactly how many data scientists there are, or how many will be needed, but by all accounts demand will be high. A recent report from the McKinsey Global Institute says that by 2018 the U.S. could face a shortage of up to 190,000 workers with analytical skills. "Data engineers are already harder to find than search engineers, and that's a sign of the times," says Deep Nishar, head of product at LinkedIn (LNKD). There's certainly plenty for data scientists to work with: IDC estimates consumers and companies will create 1.8 zettabytes (equal to a trillion gigabytes) of digital information by the end of the year. And that's a data point too big to ignore.