Data Scientists, Corporate Fortune Tellers
I realized that from a corporate perspective, “fortune teller” was not entirely off from the role of a “data scientist”.
Afew years ago, while driving in my father’s car, he started to ask me questions about my education, future, life, goals, etc. He asked me: “How big is the job market for your university major?” At the time, I was completing my degree in Artificial Intelligence. I replied: “It’s very big.” He nodded and continued: “What is this Artificial Intelligence exactly?” I started to explain it in a few easy-to-understand phrases, speaking to machine’s learning capabilities, decision making, etc. But he did not stop there, and continued: “Is it like robots?”, to which I replied: “Yes, exactly. A robot who can learn.” He pondered for a second before asking: “Will you be working with robots then?” That was the tricky one I answered “Meh… Not exactly.” Rightfully unsatisfied, he asked “So what then?” It was then that I realized that from my father’s perspective, my definition of AI is not exactly aligned with how I would apply it as a data scientist. Suddenly, I became creative and said: “I’ll be a fortune teller — but in a scientific way.” I am sure that I added more confusion to my father’s thoughts instead of clearing them.
Many years passed, and I continued consulting, teaching, and developing AI solutions in industry and academia. My long-time self appointed title of “fortune teller” for a data scientist was not comprehensive at all, as they are responsible for many things beyond future predictions. However, by that time, I realized that from a corporate perspective, “fortune teller” was not entirely off from the role of a “data scientist”.
Back to the concept of fortune tellers, this conjures up the image of an oracle with an all-seeing glass ball used to tell fortunes and announce prophecies.
“An oracle is a person or agency considered to provide wise and insightful counsel or prophetic predictions or precognition of the future, inspired by the gods. As such it is a form of divination.” — Wikipedia
Data scientists also use an oracle, which is data. Data science heavily relies on data, either past historical data or current experimental data. And the end goal is proper decision making which may affect the future.
“Data knows everything, you just need to speak her language.” — Myself
Artificial Intelligence and Data Science
In general, AI is the intelligence demonstrated by machines, which mimics human cognitive functions such as learning and decision making. Nowadays, AI is extended into so many domains, that there is almost no area left, to which AI does not contribute. AI is capable of various tasks such as classifying, clustering, predicting (regression), optimizing, reinforcement learning, etc., which ultimately help humans and machines make better decisions.
AI is a widely used term with applications ranging from image analysis to robotics. Machine learning is considered a subset of AI, targeting a narrower range of activities which in fact is applied in real-world problems. Meanwhile, Data Science uses machine learning to analyze data and make predictions about the future. It combines machine learning with other disciplines such as big data analytics, research analytics, domain knowledge, among others. Data Science is a combination of domain knowledge, computer programming skills, math, and statistics (machine learning) that is being exercised in companies for today’s world problems.
The Data Science Market
Today, Data Science has become one of the hottest jobs in the industry, since it is a relatively new job with higher than average salary ranges, and prestige. And usually, the minimum educational requirement for this job is a master’s degree. Companies, on the other hand, are trying to hire more and more data scientists, as they realized that they can use machine learning and AI to provide more insights from their data. Isn’t it great? Well, it sounds so at first, but to become a data scientist who is well prepared can be a big hassle. Data scientists must be skilled in various fields including mathematics, machine learning, computer programming, statistical modeling, data engineering, visualization, pattern recognition, uncertainty modeling, data warehousing, cloud computing, and often big data. As their job market is volatile and based on modern technologies and methods, data scientists need to update their knowledge regularly. Furthermore, they need to have some domain knowledge of the field of business they are working in. During the recruitment process, it is extremely common to have questions concerning operating the most recent technologies. Requirements vary completely from one domain to another. The questions are not only technical, but also domain knowledge related issues, and conceptual analytics.
Many make the correlation that Data Science is all about data, and data means revenue. Nowadays, companies are excited about hiring data scientists and AI engineers to make data-driven decisions. However, the majority of them miss an important issue: Data Quality.
AI can only extract valuable insights from data if the data in question contains such insights. Many companies do have data, but they are not very useful for decision making, or are filled with untraceable noise.
There is a general conception that data scientists do generate money by predicting and optimizing the future, which indeed is correct but only if the required resources are in place.
Usually, data scientists have strong scientific and domain knowledge paired with outstanding technical and research skills. This combination of skill sets can bring value to the company. Not only can they use the company’s data to provide valuable insights, but they can also help move the company towards a data-driven structure for better and more accurate fortune-telling capabilities.
Want to work with us? Click here to see all open positions at SSENSE!
Original. Reposted with permission.
- Recommender System Metrics: Comparing Apples, Oranges and Bananas
- Scaling Your Data Strategy
- Schema Evolution in Data Lakes