Bringing Business Clarity To CRISP-DM

comments

This is the second of three articles. The first outlined Four Problems in Using CRISP-DM and How To Fix Them. Two of these problems relate to a lack of clarity about the business problem that undermines the business value of any analytic developed. Analytic teams need to model business decision-making to connect their analytics to the business goals as this will bring clarity to the business problem and enable analytic results to be evaluated in business terms. In CRISP-DM solving these problems involves changes to the Business Understanding and Evaluation phases as shown in Figure 1.

Figure 1: Framing activities in CRISP-DM.

The first phase of CRISP-DM, Business Understanding, is all about bringing clarity to the business problem and so providing a business focus for the project. The need for this clarity is persistently under-called by analytic teams and overlooked in a rush to find data, analyze it and try new techniques. At best, analytic teams identify the business metrics they want to improve and then dive straight in.

But analytics cannot improve metrics directly. Take customer churn, for instance. If you need to reduce customer churn you might develop various analytics – churn predictions, propensity to accept retention offer predictions, life time value predictions etc. But none of these analytics will, by themselves, improve your customer churn rate. Unless you change your behavior – unless you do something different – your churn rate is not going to improve. The analytics might give you clues as to what change you need to make and how you might need to change but unless you actually change behavior you will not get a different outcome.

So what is it you need to change? Decision-making is the key – only if people and systems make different choices, different decisions, will results change. If that decision-making can be improved with analytics, then results will improve. For instance, simply having a churn model will not reduce churn but deciding to focus valuable retention offers on those customers most likely to churn will.

To ensure their analytic results will improve business results, analytic teams need to connect the metrics they are focused on to specific decision-making that can be improved with analytics. And they need to do this explicitly in the Business Understanding step of CRISP-DM. They need to be able to capture which decisions must change, how these decisions are made, and by whom. This is where a decision model comes in.

Decision modeling begins by understanding the business measures of success and finding the business decisions that impact those measures – especially the day to day, repeatable, decisions that do so. One or more of these decisions will have to be made differently – more analytically – if the project is going to make any business difference. A decision model clarifies this decision-making in terms of the question that must be answered to make the decision as well as the possible answers. It then breaks down the decision-making and represents it using a simple but powerful notation like Decision Model and Notation (DMN) standard to clarity the decision-making approach.

Figure 2 shows an example decision model. It shows the overall business decision to be improved (the rectangle at the top) so the project’s business focus is clear. It breaks this decision-making down into its component sub-decisions (more rectangles), both to make it clear how the decision is made and to identify the role of the specific decision that should be made analytically. It shows the regulations, policies and relevant experience involved in making the decision (as Knowledge Sources, the document shapes), proposed analytic models and currently used data (as Input Data, the flattened ovals). Such a model meets all the requirements defined in CRISP-DM for Business Understanding.

Figure 2: A decision model expressing business understanding

The roles of different groups in the decision-making can be defined by reference to the model while the systems, processes and business environment that will be impacted by any changes can be identified. Because these models are simple and visual they can be collaboratively developed and shared between analytic, business and IT teams.

The end result is clarity about:

• The business problem
  We don’t make this decision accurately enough and this is the measure that tells us this
• The use of the analytic
  An analytic that predicted this kind of thing would be helpful at this point in the decision-making
• A solution approach that is likely to work
  These people will need to believe the analytic and it will need to be delivered in this specific business process to be useful

Successful analytic teams spend more time understanding the business problem and less time wading through lakes of data.

With a decision model in hand, analytic teams also have something to evaluate their models against. The analytic measures – how accurate is this model, what’s its lift – can be used to see how well it would help make the specific analytic decision. Changes in the analytical decision can be put into the broader business context to see how that will impact overall decision-making. This will show whether the analytic approach is a match for the original business intent. Knowing who is making the various decisions involved and in what context can also help make sure the style of analytic selected will work. For instance, not using a black box analytic for a decision that involves regulators or a visual analytic for a largely automated decision.

If models don’t evaluate successfully against the decision model, then the project is back to square one. It might only need to look at more data or analyze the data in a different way, but more likely the team will need to sit down with the business and discuss how else the decision could be made. Many projects lose their way at this point, continuing to add new data or new refinements to an analytic without checking that they are still working toward a valid business objective. The data may not support the kind of analytic that would improve the decision as currently made but perhaps it will have revealed a way to improve a different decision or that the metric is not the right one. A return to Business Understanding and an updated decision model may be required to re-focus and give the project the purpose it needs to move forward.

Successful analytic teams minimize the white space between analytic success and business success, evaluating their models against business drivers as well as analytic ones.

Bringing clarity to an analytic project using decision modeling ensures teams have the business understanding they need to build a successful analytic and gives them a business benchmark to assess their analytic results against. With a decision model in hand, analytic projects can avoid the first two problems in CRISP-DM and build an analytic that will solve a real business problem and make a real business difference.

To see how one global leader in information technology is using decision modeling, check out this Leading Practices Brief from the International Institute for Analytics.

In the final article we’ll consider how to ensure analytic results get deployed and managed for long term value.

Related:

• Standards-based Deployment of Predictive Analytics
• The Data Science Process
• Data Science Automation: Debunking Misconceptions