Data Science Career: 7 Expectations vs Reality
Let’s get into some of the expectations of data scientists – and the reality they face.
There’s a huge mismatch between the expectations of people who are about to start a data science career path, and the reality of what a data scientist does, actually. There are some expectations that are met, but there are many new data scientists who find that their jobs are nothing like what they expected.
Part of this is because data science is such a rapidly moving field. Today’s buzzword is tomorrow’s obsolete tech. Another is because it’s a broad field – data scientists find they must wear many hats at their job. Plus, there’s often some misrepresentation at the interview stage: employers are so desperate for employees that they’ll really gloss up the job. And yet another part is because data science is such a broad field. It encompasses data cleaning, wrangling, machine learning, fundamental statistics, and data viz.
Source: What’s The Big Data WordPress Blog
Let’s get into some of the expectations of data scientists – and the reality they face.
Expectation #1: Data scientists make a lot of money.
The generally accepted wisdom is that data scientists are in high demand, so they take home the big bucks. In reality, your data science salary is heavily dependent on where you live, who you work for, where in your career you are, and what industry you’re in.
So how much do data scientists make? Salaries get up into the six figures, but just as common are salaries in the $60k-70k range. Some junior roles make less than $50k on average. That being said, most of the rates are very competitive and come with plenty of perks.
Expectation #2: You’ll impact your business with key insights.
Many people will tell you that being a data scientist comes down to more than just your skill with numbers and Python. Being a good data scientist means that you have business insights. You know what the numbers mean. And you'll be relating those important insights to key members of your company's leadership team.
The reality is that like in any other role, you have to bow to politics. Sometimes the work you do will have a real business impact. You'll develop an algorithm that really affects the company's bottom line. But just as often, if not more so, you'll make a spreadsheet, data visualization, or just crunch some numbers for an exec who needs them for a presentation tomorrow. “[Y]ou have to constantly do ad hoc work such as getting numbers from a database to give to the right people at the right time, doing simple projects just so that the right people have the right perception of you,” writes Deliveroo data scientist Jonny Brooks-Bartlett.
Expectation #3: It’s all ML, AI, and algorithms.
Right now, for example, AI is a very popular buzzword in data science. A new data scientist might naturally believe that they’ll be building cutting-edge algorithms, and working on machine learning.
However, as mentioned above, a lot of it is just ad hoc data-adjacent work. It’s a lot of ETL. It’s a lot of data cleaning. While some of your time will certainly be spent on exciting concepts and technology, a lot of it will not.
Furthermore, your employer will often expect you to start building those AI models ASAP using extremely limited, clunky data. “[New data science hires] expected to put some effort into gathering and cleaning data — but they didn’t expect it to be so complex and messy, with so much missing or hard to access,” writes Monica Rogati in her guide for how not to hire a data scientist, perfectly encapsulating what it’s like to be a new data science hire.
Expectation #4: You can learn everything you need online.
There’s this sense that boot camps or Coursera courses will teach you everything you need to know about how to become a data scientist. The reality, like in just about every other job, is that there's a lot of on-the-job learning you need to do. There are some communication skills and leadership instincts that you can't get from a Coursera course.
It’s also true, more for data scientists than other roles, that the job is what you make of it. Many startups hire data scientists because their investors are asking for them. Tech-illiterate founders have no idea what you can do, so it’s up to you to figure out and define the role. (And get ready for a lot of misunderstandings along the way.)
Expectation #5: It’s all numbers.
As a data scientist, you expect your job to be straight math. You collect data, run analyses, create data products, and crunch numbers. But remember that your employers are not as neutral as you are. Just as data science can be used for bad purposes by accident, so too can data scientists.
I’m sure we’re all familiar with Amazon’s infamous anti-woman AI hiring tool. Some data scientists built that model. An even more morally gray example is Uber’s Greyball project that dodged law enforcement to help Uber drivers avoid being ticketed. Again, that product didn’t spring into being fully formed from Travis Kalanick’s forehead like Athena. Real people built it.
As a data scientist, some of what you’re asked to do can be questionable. Companies aren’t necessarily forces for good; they’re mostly forces for profit. Especially if you’re building products that could affect many people, it’s worth keeping an analytical mind about the tasks you’re asked to complete.
Expectation #6: You’ll be valued, understood, and supported.
As many data scientists know, the reality of the job is that there’s no guidance, little infrastructure, and poor understanding of what you actually do. “The data scientist likely came in to write smart machine learning algorithms to drive insight but can’t do this because their first job is to sort out the data infrastructure and/or create analytic reports,” writes Brooks-Bartlett. Especially if you’re working for a startup, you’ll often have to do a lot of pre-data science work to get your tools, architecture, and technology up to par. And the IT team is not always cooperative.
Furthermore, colleagues will think you can handle any request that has to do with “data,” whatever the sexy data term is at that time. Six or seven years ago, data scientists would have been asked to store and process data with Hadoop. A C-suite-level executive might ask you to do something with “deep learning.” Simultaneously, you’ll get asked to send over an Excel sheet with a simple VLOOKUP formula.
“At the end of the day, you’re no longer building complex algorithms and predictive models like you imagined. You now spend all your time brushing up on SQL and data preparation skills to pull data out of the system into different formats, and present this data to stakeholders so they can use it to make business decisions,” writes Natassha Selvaraj on why so many data scientists are leaving their jobs.
While data scientists are definitely sought-after, there’s so much confusion about the role that there’s not always the support or infrastructure you need to do your job.
Expectation #7: You’ll love your job.
Lots of data scientists are leaving as Selvaraj points out, but many still enjoy their lives. Glassdoor lists it as the #3 best job in America, with a relatively high job satisfaction level (4.1 out of 5). One KDNuggets survey found that the majority of data scientists are still happy with their jobs.
Like any job, being a data scientist can be tedious. It can come with unexpected tasks. It can be challenging. But it can also be a lot of fun. As a data scientist, you’ll be asked to push your limits and you will have the chance to make a real impact on your company (even if you also have to do menial jobs too).
And once you’re past the growing pains of landing in your new role, you’ll have the power to create a job you love. “I'm in a position where I just get to do data science,” writes BullCityPicker on Reddit, “A minimal number of stupid meetings, lots of don't-mess-with-me time, and my pick of interesting projects. I get paid really well for it. I have lots of autonomy, get to constantly learn new stuff, and have loads of autonomy. Two thumbs up from me.”
What’s the reality of being a data scientist?
To summarize, as a data scientist, you:
- Have the chance to earn lots of money – but it’s not guaranteed.
- Can impact the bottom line – but you need to play nice with others if you want people to listen to you.
- Will work on cutting-edge tech – and do a lot of dull SQL queries.
- Can be well-prepared just with online learning – but you’ll have to learn certain skills by being thrown in the deep end.
- Create data-based products that really impact people – but those impacts can be negative as well as positive.
- Are valued by your coworkers and company – but you’ll also be deeply misunderstood and often undersupported.
- Will love your job – if it’s the right one for you.
In other words, being a data scientist is a lot like any other job. Unlike other jobs, data science has also been subject to the marketing behind people calling it the “sexiest job of the 21st century.” This causes a lot of misunderstandings and mismatched expectations for you and your employer.
If you know what to expect and you’re still interested, it can be a highly rewarding career path to pursue.
Nate Rosidi is a data scientist and in product strategy. He's also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Connect with him on Twitter: StrataScratch or LinkedIn.