The Third Wave Data Scientist
An extensive look at what skills are needed to make up the portfolio of the third wave of data scientists.
By Dominik Haitz, IONOS.
Drew Conway’s visualization of the data science skill set is an often cited classic. Different opinions and the versatility of the role have spawned numerous variations:
Various data science Venn diagrams. Image courtesy of Google Images. Source: https://sinews.siam.org/Details-Page/a-timely-focus-on-data-science
There seems to be no consensus on the data science skill set. Additionally, as the field evolves, shortcomings become obvious and new challenges arise. How can we describe this evolution?
The first wave of data scientists happend before data became big and before data science was actually a thing (pre-2010s): Statisticians and analysts who had always been around, doing a lot of what modern data scientists are doing, but accompanied by less hype.
Second wave: Large-scale data collection created a demand for smart minds who can work magic and turn all this big data into big money. Companies were still figuring out what kind of people to employ and often turned to science graduates. While the second wave data scientists did a lot right, their carefully crafted models often ended up as PoCs and failed to bring about actual change.
Now, at the end of the 2010s, amidst the hype around deep learning and AI, enter the third wave of data scientists: Experimenting and innovating, efficiently seeking out business value und bridging the deployment gap to create great data products. What skills are required here?
The skill portfolio of the third wave data scientist.
1. Business Mindset
The business mindset is the centerpiece of the data science skill set, as it sets goals and applies the other skills to reach them. Patrick McKenzie states in this blog post:
Engineers are hired to create business value, not to program things: Businesses do things for irrational and political reasons all the time […], but in the main they converge on doing things which increase revenue or reduce costs.
Likewise, data scientists are hired to create business value, not just to build models. Ask yourself: How will the outcome of my work influence company decisions? What do I have to do to maximize this effect? With this entrepreneurial spirit, the third wave data scientist does not only produce actionable insights, but also seeks that they bring about real change.
Look where the money flows in your organization — the divisions with the largest cost or revenue will likely offer the highest financial leverage. However, business value is a fuzzy concept: It goes beyond cost and revenue of the current fiscal year. Experimenting and creating an innovative data culture will increase a company’s long-term competitiveness.
Prioritizing your work and knowing when to stop is the key to efficiency. Think of diminishing returns: Is it worth spending weeks to tweak a model for another 0.2% of precision? Quite often, good enough is the real perfect.
Domain expertise, which makes up a third of Conway’s skill set, is by no means to be neglected — however, you’ll almost everywhere have to learn it on the job. This includes knowledge about your industry as well as all the company processes, naming schemes and peculiarities. This knowledge does not only set the frame conditions for your work, but it is often indispensable to understand and interpret your data.
Keep it simple, stupid
Half of the time when companies say they need "AI" what they really need is a SELECT clause with GROUP BY.
— Mat Velloso (@matvelloso) May 30, 2018
Look out for the low hanging fruit and quick wins. A simple SQL query on existing data warehouse might yield valuable insights unbeknownst to product managers or executives. Don’t fall into the trap of doing “buzzword-driven data science”, focusing on state-of-the-art deep learning where a beautifully simple regression model would be sufficient — and much less work to build, implement and maintain. Know the complicated things, but do not overcomplicate things.
2. Software Engineering Craftsmanship
The notion of (second wave) data scientists needing only “hacking skills” instead of proper software engineering has been repeatedly critizised. Lack of readability, modularity or versioning hinders collaboration, reproducibility and productionizing.
Instead, learn the craft from proper software engineers. Test your code and use version control. Follow an established coding style (e.g. PEP8) and learn how to use an IDE (e.g. PyCharm). Try pair programming. Modularize and document your code, use meaningful variable names and refactor, refactor, refactor.
Bridge the deployment gap for agile prototyping of data products: Learn to use tools for logging and monitoring. Know how to build a REST API (e.g. using Flask) to provide your results to others. Learn how to ship your work inside a Docker container or deploy it to a platform like Heroku. Instead of letting your models rot on your laptop, wrap them into data-driven services that fit snugly into your company’s IT landscape.
3. Statistics and Algorithms Toolbox
Data scientists have to thoroughly understand the basic concepts in statistics and particularly in machine learning (A STEM university education is probably the best way to acquire this foundation). There are tons of resources on what’s important, so I’m not gonna delve further into this here. You will often have to explain algorithms or concepts like statistical uncertainty to your clients, or red-flag an insight because of a confusion between correlation and causation.
4. Soft Skills
As people skills are as important to productivity as technical skills, the third wave data scientist makes conscious efforts to improve in these areas.
Work well with others
Consult your peers — most people are happy to help or give advice. Treat others on par: You might have a fancy degree and an understanding of sophisticated algorithms, but others have experience that you don’t (This sounds like basic social advice, but who hasn’t met an arrogant IT professional?).
Understand your client
Ask the right follow-up questions. If a client or your boss wants you to calculate some key figures or create some chart, ask “Why? What for? What do you want to achieve? What actions will you take, depending on the result?” to better understand the core of the problem. Then figure out how to get there together — is there another, better way to reach that goal than the one proposed?
Navigate company politics
Network, not because you expect others to benefit you in your career, but because you are an approachable person. Connect with people with similar work topics. If there are no platforms for this in your company, create them.Identify key stakeholders and figure out how to help them with their problems. Invite others early on and make them part of the change process. Keep in mind: A company is not a rational entity, but an assembly of often irrational human beings.
Communicate your results
Ramp up your visualization and presentation skills. Communicate from a client perspective: How can I precisely answer their question? Learn to communicate at different levels and sum up the details of your work. People are easily intrigued by fancy multidimensional plots, but often a simple bar chart conveys a message more efficiently. Showcase your results: When people see what you’re doing, and see that you’re doing great work, they will trust you.
Communicate your goals and problems and actively seek out advice. Find role models inside and outside the data science community and learn from them.
Original. Reposted with permission.
- On-line and web-based: Analytics, Data Mining, Data Science, Machine Learning education
- Software for Analytics, Data Science, Data Mining, and Machine Learning
- The most desired skill in data science
- Who is a typical Data Scientist in 2019?
- The Difference Between Data Scientists and Data Engineers