What’s missing from self-serve BI and what we can do about it
The notion of self-service BI tools caught an expectation that they could provide a magic formula for easily helping everyone understand all the data. But, such an end-result isn't occurring in practice. To identify a better approach, we need to take a step back and determine what problem is actually trying to be solved.
By Benn Stancil, Co-founder, President, and Chief Analytics Officer of Mode.
At most growing companies, data teams are simply too small to answer the number of questions that people ask. To address this problem, most teams turn to self-serve BI tools to enable people to find their own answers.
The ambition of self-serve BI is an understandable one: these tools aspire to open up vast troves of “insight” by making data accessible to everyone at a company. But the reality is often messier. While self-serve tools make it possible for anyone to extract and visualize data, they don’t imbue their users with the skills required to make sense of data and all its complications. And without those skills, data is not that useful.
But saying self-serve comes up short isn’t enough; data analysts still get asked more questions than they have time to answer. Rather than enumerating the ways in which self-serve BI has failed us, the real question we should be asking ourselves is, “What can we do about it?”
What problem are we trying to solve?
Analyzing data is more than just a technical skill. An analysis isn’t a matter of writing the write query to extract data; it’s a matter of understanding what to do with it. Interpreting data correctly requires a number of specific and nuanced skills, including the ability to reason in a certain way. Self-serve BI tools can help people who don’t know SQL pull data, but they don’t magically help people know what to do with it or now to make sense of the messy details within it.
Fortunately, this doesn’t mean that all self-serve is useless; it just means we need to be thoughtful about what self-serve provides.
We tend to think of self-serve BI as a code-free version of what analysts do. More than a decade ago, Forrester said the aim of self-serve is to help everyone “explore rich analytic information sets from all possible angles,” just as an analyst would. “In the self-service scenario, the core development issue becomes one of user creativity.” This hasn’t changed over the last ten years. We still imagine that, with the right self-serve tool, people are only constrained by the questions they can ask.
I believe this framing is wrong.
Analysts and non-analysts use data in structurally different ways. By conceptualizing self-serve BI as a simplified means for doing an analyst’s job, we’re not only making the self-serve problem too hard, but we’re solving the wrong thing altogether.
Instead, we should build self-serve tools to help people with what they really want: a means for extracting metrics. Rather than asking complex questions that require manipulating data in novel ways, most business users just want to choose from a list of understood KPIs, apply them to a filtered set of records, and aggregate them by a particular dimension. They don’t want to write a novel; they want to fill in analytical Mad Libs.
Say, for example, that a sales leader wants to assess their team’s performance for the quarter. To do this, they wouldn’t conduct analysis like an analyst; they’d survey core metrics like bookings, close rates, and deal velocity by segment, region, and team. They would compare these numbers to prior quarters and the current targets. Then, weighing this body of evidence, they would make an overall assessment of the quarter.
When we think about building self-serve tools, we should focus on this sort of metric extraction. This would directly address the needs of the people who use self-serve tools—much more so than seemingly flexible capabilities like “code-free analysis.”
What can we do now?
Reorienting self-serve around metric extraction will take time. In the meantime, how do we escape between the rock of having too few analysts to answer these questions and the hard place of analytical thinking being hard? Here are a few steps we can take in order to improve the current self-serve experience:
- Define self-serve better. Up to this point, the term “self-serve” has been used rather nebulously as a vague catch-all referring to any tool that helps people answer questions by themselves. We should recognize that serve-self doesn’t necessarily mean the same thing to all people, and each company should define to best suit their needs.
- Develop data proficiency across the organization. The coming decade will bring huge advancements in data technology. Though not everyone will be an analyst, we’ll all need to be comfortable working with data. That means everyone will need to develop a certain degree of data literacy.
- Hire more analysts. If you want to build a great brand, you invest in great marketers. If you want to build great technology, you invest in great engineers. And if you want to make great decisions with data, you... buy great self-serve tools? No. If you want to be great at interpreting data, you need to hire great analytical thinkers. This isn’t a free lunch.
If, as a data professional, you find yourself in a situation where you feel that self-serve BI tools are failing you, take a step back and determine what problem you are actually trying to solve. Who is asking for the data, and what do they actually need to do with it? The better we understand the problem, the greater the likelihood that we can build something to solve it.
Bio: Benn Stancil is an accomplished data analyst with deep expertise in collaborative Business Intelligence and Interactive Data Science. Stancil is Co-founder, President, and Chief Analytics Officer of Mode, an award-winning SaaS company that combines the best elements of Business Intelligence (ABI), Data Science (DS) and Machine Learning (ML) to empower data teams to answer impactful questions and collaborate on analysis across a range of business functions. Under Stancil’s leadership, the Mode platform has evolved to enable data teams to explore, visualize, analyze and share data in a powerful end-to-end workflow. Prior to founding Mode, Stancil served in senior Analytics positions at Microsoft and Yammer, and worked as a researcher for the International Economics Program at the Carnegie Endowment for International Peace. Stancil also served as an Undergraduate Research Fellow at Wake Forest University, where he received his B.S. in Mathematics and Economics. He believes in fostering a shared sense of humility and gratitude.
Related: