FICO Lessons in Developing, Applying Decision Modelling Methods
Analytically sophisticated businesses combine predictive analytics and decision models with optimization to solve complex problems and achieve good results. Top FICO expert explains.
Guest blog by Neill Crossley, FICO Labs, Dec 20, 2013.
This article is based on a presentation at the 2013 University of Edinburgh Credit Scoring & Credit Control XIII Conference.
One of the hallmarks of the analytically sophisticated business is its use of optimization to solve complex problems. Optimization can be especially rewarding when combined with decision models. A decision model is a framework of models and calculations used to understand the interactions between criteria used in a decision - defining potential decision options, inputs and outcomes, and the objective(s) in making that decision.
If a predictive model measures the probability of a single outcome, then a decision model measures the probable outcomes of multiple potential decisions so you can choose the best decision to meet your goals.
Decision models are often represented graphically by an Influence Diagram - with the constructed decision model itself using mathematical formulae for the relationships shown.
Clarity through Project Design
Decision models are very helpful in explaining complexity. They provide a standard framework, allowing the key criteria of decisions and their relationships to be defined and understood by business users and analysts while also guiding model development.
My team at FICO primarily uses decision models to formulate decision-focused predictions and optimization solutions. Clients often find the initial design process itself very rewarding, as it guides the identification of key inputs, outputs and relationships, with a primary focus on profit. This can help identify business issues, as well as bring different business areas together to focus on a common goal.
Action Effect Models Are Key
These causal models are crucial components in successful decision modelling, particularly when used for decision optimization. They evaluate a customer's reactions to different actions so you can identify those decisions that provide the desired result.
Action Effect Models are difficult to develop, as you never have complete data for all actions by all customers, with the observed data often heavily biased by historical targeting or correlation effects.
I've found the best models result from an iterative development process that allows analysts to easily apply statistical factors to observed data, but importantly also overlay business knowledge. I often recommend using simple model formulations such as look-up tables that make it easier to visualise, compare, and adjust predicted outcomes.
Power of Scenario Analysis
Decision models drive a wide range of scenario analysis, including optimizations, simulations, stress tests and comparisons with current decision strategies. Each scenario is constructed using different constraints, which allow users to focus on various business objectives.
The scenario output provides huge insight into more profitable decisions and the impact of different business constraints. Efficient Frontier graphs are often used to compare scenarios (right), identifying profit improvements where key business constraints are also met.
However, it's also important to understand strengths and weaknesses -which areas are based on sound analysis and which are more "guestimates."
From Scenarios to Decision Strategies
Once a scenario has been chosen, there are usually two deployment options: 1) customer decisions taken directly from the scenario output; 2) conversion of the scenario into a decision tree or business rules, which is often required in highly regulated credit markets.
This latter requirement brings a choice between palatability and power. A larger, more complex decision tree may give a finer segmentation, and provide a better outcome, while a smaller, simpler tree may be easier to manage.
To help with this trade-off, it is useful to have a software solution that allows the user to define the size and shape of the decision tree as part of the optimization process. This ensures the output decision tree has the best balance between power and palatability.
When executed properly, decision modelling is a powerful technique for understanding complex scenarios and optimizing decisions to give an organization the highest probability of achieving its desired outcome. The key is understanding the best modelling approach to use for a given problem, and applying sufficient domain expertise to the modelling so the scenarios can be set up properly and the models can yield the most realistic and predictive results.
For more discussion of analytic technology, see the FICO Labs Blog at ficolabsblog.fico.com.
Neill Crossley has over 29 years Financial Services experience, the majority focused on the development and use of predictive and prescriptive analytics to solve business problems across the customer lifecycle.
He has been with FICO for the last 10 years and is a member of FICO Labs, a cross functional team that focuses on new innovations. He brings together a rare combination of retail banking and technical expertise and has a proven track record of delivering innovative solutions that result in multi-million dollar benefits.