A reasoned rule approach is a good first step to managing decisions that fall into common patterns or cases. You identify six to eight variables that are distinct and obviously impact the outcome of the case and normalize them into standard scores that can then be added or averaged to create a summary score.
Daniel Kahneman’s Reasoned Rule Approach
to Reducing Case Management Errors
In “Noise: How to Overcome the High, Hidden Cost of Inconsistent Decision Making” (HBR Oct-2016) Daniel Kahneman et. al. offer a “reasoned rule” model for reducing case management decision errors. It’s a great first step in improving the accuracy of your team’s decision making capabilities where the decisions are re-occurring and fall into common categories or cases that need to be managed consistently by multiple people on your team. Kahnman suggests that an adequate algorithm can be developed without any outcome data at all using “commonsense reasoning” and input information on only a small number of cases. He calls a heuristic for decision making built without outcome data a “reasoned rule.”
How To Build a Reasoned Rule
You don’t need outcome data to create useful predictive algorithms. For example, you can build a reasoned rule that predicts loan defaults quite effectively without knowing what happened to past loans; all you need is a small set of recent loan applications. Here are the next steps:
- Select six to eight variables that are distinct and obviously related to the predicted outcome. Assets and revenues (weighted positively) and liabilities (weighted negatively) would surely be included, along with a few other features of loan applications.
- Take the data from your set of cases (all the loan applications from the past year) and compute the mean and standard deviation of each variable in that set.
- For every case in the set, compute a “standard score” for each variable: the difference between the value in the case and the mean of the whole set, divided by the standard deviation. With standard scores, all variables are expressed on the same scale and can be compared and averaged.
- Compute a “summary score” for each case, the average of its variables’ standard scores. This is the output of the reasoned rule. The same formula will be used for new cases, using the mean and standard deviation of the original set and updating periodically.
- Order the cases in the set from high to low summary scores, and determine the appropriate actions for different ranges of scores. With loan applications, for instance, the actions might be “the top 10% of applicants will receive a discount” and “the bottom 30% will be turned down.”
You are now ready to apply the rule to new cases. The algorithm will compute a summary score for each new case and generate a decision.
Daniel Kahneman et. al. In “Noise: How to Overcome the High, Hidden Cost of Inconsistent Decision Making” (HBR Oct-2016)
The standard scale he constructs is simply the number of positive or negative standard deviations from the mean for each state variable that you believe has an impact on the outcome. It should normally range between -3 and +3 (plus or minus three standard deviations from the mean should normally cover 99% of the cases); for those variables with a negative effect you invert the sign. Although not called out explicitly in his basic approach a yes/no or binary variable would simply be a +1 or -1 depending upon whether its presence or absence–and whether it was viewed as contributing to a negative or positive outcome.
The key benefit to this approach are:
- It encourages you to identify key state variables that you believe will have an impact on the outcome. Just getting agreement on this and whether they make a positive or negative (risk factor) contribution will help to bring clarity.
- It then has you analyze the range of values for a given population of prior decisions or cases you have faced. By looking at a series of decisions as part of a category instead of one-off you are more likely to improve.
- This is a very basic approach that can be extended by careful analysis of outcomes and potentially by weighting some variables more heavily than others.
I think this has particular merit in bringing a case management approach to lead scoring, opportunity qualification, and forecasting a rough likelihood of a deal closing.
Related Blog Posts
- Video & Slides from “Limits of I’ll Know It When I See It” Talk at SFBay ACM
- Q: How To Speed Up Early Trials, Adoption, and Sales
- Successful Bootstrappers Are Trustworthy Salespeople Committed to Customer Satisfaction
- Information That is Not Written Down
- Knowledge That is Not Written Down, Yet
- Organizing Your Experiment Log
Photo Credit: Emmi Land “This is not going to end well“