I am constantly reading disparaging comments about how the probabilities used in decision tree analysis of litigation are “mere estimates”, and garbage in, garbage out, etc. The probabilities we use are not the same as the classical statistical frequencies which are used, for example, to deal with a tossed die or a flip of a coin. The probabilities we use are our quantified gut feel to express our understanding of how likely an event is or will be. These are almost always one time events. We can’t try the case 100 times and see how many times we would be liable.
The reasons we use numbers to express these judgments about the likelihood of future events are twofold. First, numbers are unambiguous. Any phrase you might use to describe the likelihood of a ruling or trial outcome is ambiguous. To some people “Very likely” means a probability from .8-.9, for others, the same phrase will be interpreted to mean .6-.7. The second reason we use numbers, is that we can used well known and understood rules of probability to combine these judgments to come to an overall conclusion. Any real case has at least several events that must be assessed to come to an overall conclusion. What is the likelihood of having to succeed on three events where the likelihoods of success are given as “Very Likely”, “Somewhat likely”, and “Likely?” However, if I tell you the likelihoods are .8, .4, and .3, the overall probability of success is .096, or .1 for all practical purposes.
When we talk about probabilities, there are two kinds of variables we are trying to describe. The first are the likelihood of discrete events. By discrete events, I mean events that have a finite (usually less than 5, more commonly 2 or 3) number of possible outcomes. The objective is to assign a probability to each of the possible outcomes. This is a rather straightforward process that has been described in detail elsewhere (ref Carl article, mention I can email pdf). Examples of such variables are the motion to dismiss is granted, we are found liable, damages are trebled, pre-judgment interest is included, etc. I find that is most useful to structure the decision tree, to the extent possible, using two branch nodes, only two possible outcomes. There are two reasons for this. First, it is much easier to assess yes/no events, and second, it is straight forward to do sensitivity analysis for such events/nodes in the tree. More complex (i.e. more outcomes) can be modeled by using sequential two branch nodes.
The second type of probabilities relate to issues where the event we are trying to assess is a numerical outcome. We call these continuous variables. These are outcomes such as the amount of damages, a date, an interest rate etc. These outcomes are described by continuous probability distributions. However, in order to include them in the tree, we must approximate them by a discrete number of branches, for example – High, Medium and Low. I have a slideshow (see Probability Assessment: How Do We Get “Good” Numbers For The Analysis? under Slideshows) that discusses this in more detail, but the basic questions that need to be asked are in the form of: “How likely is it that the jury will find damages greater than $15M?” After several of these questions are answered, you can draw a probability distribution and approximate it with a 3 branch node.
When we discussed the coin toss, the 50% probability of heads or tails was obvious (assuming, of course, a fair coin). Suppose now that I flip a coin and cover it up with my hand so that you can’t see it, but I peek at it. What is my probability that the coin is heads? It’s either 0 or 1 (depending on whether the coin landed tails or heads). What is the probability that the coin is heads (you still haven’t seen it!)? 50%! How do we explain the apparent discrepancy between the same coin toss and different probabilities? The probabilities represent a quantification of a state of knowledge and judgment. You and I have different states of knowledge about the coin, therefore we have different probabilities. Here we are using Bayesian statistics, not the classical statistics that most of us have been taught; the latter deal only with the frequency with which the coin will land heads or tails, not its state on any one toss.
Unlike the coin, for which there is an observable frequency that we could measure by tossing it many times and counting the number of heads and tails, each litigation case happens only once, and we can never measure a frequency for the particular trial under consideration. The probabilities represent the best judgment, knowledge, and experience that we can bring to bear on the particular uncertain outcome. There is no correct probability. If we asked someone who could foretell with perfect accuracy the outcome of the trial or of any single issue, “Will we win?” the answer will be a “yes” or “no”, not a probability. This is a very important point. A probability is correct only to the extent that it accurately represents the state of knowledge and judgment of the person being asked.
There are, however, well-known biases in the way we think about probabilities and uncertain outcomes ⚖. The most important of these is the tendency to think we know more than we do. We make our probability distributions too narrow for our true state of knowledge. This has been demonstrated in hundreds of tests in which executives and professionals were asked to encode their own probability distribution on knowable quantities (e.g., the air distance from Moscow to Beijing). They were asked to set outer limits on their distributions so that there would be only a 2% chance that the correct answer would lie outside their limits. In fact, the correct answers fall outside their limits about 50% of the time. The world is much more uncertain than we would like to think it is! Fortunately, techniques have been developed to counteract such biases,⚖ and they are straightforward to use.
The question remains, of course, whether or not a particular person is a good judge of an issue. We all have good intuitive ideas about who the best experts are on particular questions, but by encoding probabilities we can calibrate their judgments quite readily.
What we are trying to do with probabilities is to pick the best decision, the one with the highest expected or average outcome. In the face of uncertainty, that is the best we can do. In fact, when we make a decision without doing a decision tree, we are implicitly assigning probabilities. It makes much more sense to do the best job we can assessing the probabilities of all the critical events that determine the expected outcome and pick the best alternative.
⚖ Please request supplemental articles by using the Contact link and we will email them to you.