Does Your Company Need a Data Scientist?
Your company needs a data scientist... doesn't it? It very well may not, but you need to know either way. Read on to determine whether or not your company could benefit from the skills of an on-board data scientist.
By Bilal Mahmood, Bolt.
Data Science is growing.
It’s been called the “sexiest job of the 21st century”, and is attracting a flood of new entrants.
Recent reports indicate that there are 11,400 data scientists who have held 60,200 data-related roles. And the overall count has grown 200% over the last 4 years, across Internet, Education, Financial Services, and Marketing industries.
And yet amidst a field growing so fast, you can observe a bit of confused exuberance. It’s not uncommon for a company to hire a data scientist just after product launch, or after Series A. To some, data science has become the magic bullet for achieving scale or their next inflection point.
But what does a data scientist do? And does your company actually need one?
Data Scientists and Analysts
At its core, data science helps your company make decisions on product and operating metrics. It does this via data products and decision science – improving product performance, building prediction models, affinity maps, and cluster analysis.
But data science is just one tool. Business intelligence and analyst functions can also help with operating metrics, albeit with more basic toolsets of SQL and Excel. Whether you use one or the other depends on your company’s data infrastructure and event volume – and hiring data scientists too early can be like trying to crack a nut with sledgehammer. Except that this particular sledgehammer will feel understimulated, underappreciated, and probably end up quitting.
Not recognizing the distinction can lead to premature adoption of a data science team at high resource costs, hurting your business and limiting your data scientists. I’ve worked on several teams that started, grew, and eventually churned out their data science org – all for what seemed like good reasons, but nevertheless leading to unfortunate outcomes.
It stands to reason that not every company needs a data scientist.
Companies of a certain data maturity are best positioned to leverage data science teams, while others can fulfill their data needs with BI and Analyst functions. The criteria below outline scenarios where a data science team may not be necessary, and assist individual data scientists in identifying companies that may not have the right infrastructure to support them.
Low Event Volume
Not enough data is every data scientist’s nightmare.
Data scientists thrive when they are able to work with larger data sets and event volumes. But more importantly, the specialized skillsets that data scientists employ – linear regressions, bayesian modeling, etcetera, simply don’t work on smaller data sets.
Low event volumes can affect the statistical and explanatory power of your dataset. If you have only 100K users doing a couple actions a day, you likely have a lot statistical power but little explanatory power. By contrast if you have 1000 users doing 1000s of events per day, you have little statistical power but lots of explanatory power.
Small or sparse data sets can make drawing statistically meaningful conclusions via correlations and propensity scores impossible. And defining cohorts or segmenting your users only reduces the sample sets within smaller event volumes. Machine learning techniques used by data scientists simply wouldn’t be possible in such situations.
If your company has a low volume of events, traditional business intelligence and analyst roles may be more resource and cost-effective solutions to a majority of your analytics needs.