3 Ways to Manage Human Bias in the Analytics Process
Managing human bias is an important part of the analytics process. Learn about three areas to watch out for to ensure your models are as unbiased as possible.
As businesses turn to machine learning to automate processes, questions have been raised about the ethical implications of computers making decisions. How do we address the potential for bias? What’s less talked about, but equally important, is the topic of human bias as it relates to analytics and business decision making. Human bias can enter the analytics process every step of the way. As business decision-makers start looking to predictive analytics for specific advice on what action to take next, it’s important that data and methods are leveraged as objectively as possible. The responsibility of monitoring human bias in analytics is a great one, and it all starts with the people building models from the ground up: data scientists.
As a data scientist navigating human bias in analytics every day, here are three areas I focus on to manage bias throughout the process—from evaluating the initial request and collecting information to building the model and mining for insights.
Evaluate the ask: What is the business decision-maker looking for?
In some cases, bias enters the analytics process from the start, coming directly from the business user who requests information in the first place. For instance, a model might be requested and evaluated with confirmation bias, from someone looking to validate a pre-existing belief. If a CMO believes their company should invest in PR, for example, it’s problematic to task their analytics team with building a model that demonstrates the need for PR. Instead, I approach the modeling process from the perspective of considering what the best of many options could be.
Like anyone, a data scientist wants to please the boss—we always want to deliver information that will satisfy the business decision-maker’s inquiry. That said, it’s important not to seek and assess the findings to fit what the business decision-maker wants. To achieve the most unbiased results at the end, avoid setting expectations from the start. The process should be a collaborative one—you may need to educate the business decision-maker on the most ethical and accurate way to go about answering their business question, and that’s OK. It’s tricky, but it is possible to address the ask while also avoiding being influenced by what the business decision-maker wants.
Carefully select & evaluate information that feeds the model
The next place human bias can enter is when selecting data. Consider what of the required data you have available and where you will collect it from. Ask questions like: How much data do I have compared to the total relevant population? How is that sample of data created?
When addressing quality, look for consistency in information, and evaluate whether it captures sufficient variables. Ensure nothing vital is missing and if it is, make sure to call attention to it and identify the likely repercussions that will have on the model.
Objectively choose the best analytics method
Each method and model has its assumptions—it’s very important to know which best fits your problem. Different modeling choices can sometimes lead to very different outcomes. The complexity and the nature of the ask and the availability of data will direct you to the appropriate method. Be mindful of the outcomes, test results for stability, and compare model results to your a priori expectations. Is the direction of the effect logical? Is the magnitude of the effect logical? Is the function of the effect logical? Is the fit expected? All those are questions that should be addressed to have confidence in the model.
If you’re testing data in various algorithms, be careful that you don’t choose a specific algorithm because it spits out the desired output. Be mindful of all insights that the model gives.
Ultimately, the best way to avoid biased analytics is by implementing a process that includes checks and balances. All assumptions should be peer-reviewed and checked. The greater diversity in people, perspectives, and information throughout the analytics process, the greater the chance for a balanced, unbiased outcome.
- Types of Bias in Machine Learning
- Is Bias in Machine Learning all Bad?
- The Algorithms Aren’t Biased, We Are