The Clinical Drug trial

The Clinical Drug Trial demonstration illustrates some of the underlying ideas of machine learning, based on probability theory. The setting is a simple drug trial scenario in which patients are given either the drug or a placebo (a fake drug with no physiological properties), and we then observe how many patients are cured or not cured in each category (as specified by the user of the demonstration). These observations provide evidence as to whether the drug is effective or not. An important aspect of this demonstration is that our beliefs about the efficacy of the drug and the placebo are represented using probability distributions, thereby capturing the inherent uncertainty involved; as the number of patient observations increase, this uncertainty decreases, represented by narrowing probability distributions.

 

Overview

This demonstration is available as an application that will run in most major web browsers, on Windows as well as Mac OS X. The following picture shows the running demonstration:

 

 

Start Demo
The white-yellow and white-blue medical capsules on the left represent the drug and the placebo, respectively. The columns in the corresponding rows show the number on patients who did (green) and did not (red) get cured. Finally the blue bar on the left shows the probability that the drug is effective, in other words that, it acts differently from the placebo. In this particular instance, the same number of patients were given the drug and the placebo. The majority of those given the drug got cured, whereas for the placebo, the numbers are opposite and accordingly, the probability distributions over the chance of being cured are mirrored around the 50%-line, with patients receiving the drug standing the better chance of being cured. Based on this, the probability of the drug being effective is fairly high, as indicated by the blue bar situated between the medical capsules on the right.

 

The user can interact with the demonstration by picking the number of patients in the two green and the two red boxes. This is done by simply clicking within a box, which will fill the box, from left to right and top to bottom, with figures representing patients; clicking in the top-left corner results in zero patients, while clicking in the bottom-right corner fills the box.

 

Try the demonstration in your browser. If you don't have Silverlight installed already, you will be prompted to install it.

 

Try this

Working only with the number of patients in the group receiving the drug,

  • vary the number of patients in the red and green boxes separately, exploring the entire range of number of patients;

  • vary the number of patients in the red and green boxes, but keep the proportions fixed; e.g. for every patient added to the red box, add two to the green box;

  • swap the number of patients in the red and green boxes.

 

Keep a close look at how the probability of cure changes with the changing number of patients in the red and green boxes.

 

Working with the number of patients in both groups,

  • vary the number of patients in each group separately, using the same number of patients in the red and green boxes within each group;

  • vary only the number of patients in the green box for the placebo group, keeping the number of patients in the remaining three boxes fixed.

 

This time, look at how the belief about the effectiveness of the drug is changing. Note that we do not distinguish between a positive and a negative effect; a drug that lowers the probability of cure clearly has an effect, albeit undesirable.

 

How it works

This demonstration uses Bayes’ theorem, one of the cornerstone equations for probabilistic reasoning, to combine any prior beliefs we might hold about an unknown quantity of interest with evidence from observations, to obtain a revised belief about this quantity.

 

For example, consider the probability of cure when given the drug and denote this quantity α. It is assumed that we have no particular prior beliefs about what values α is likely to take, and so distribution over α is flat initially (when there are no patients within a group); we call this the prior distribution over α. As we observe outcomes for patients receiving the drug (cured or not cured), these are combined with the prior distribution to yield a posterior distribution. It is worth noting that the most likely value for α under this distribution (the mode of the distribution) is the number of patients that got cured divided by the total number of patients receiving the drug, which seems intuitive. It is also worth noting that, while the mode is clearly an informative property of this distribution, there are important aspects that is does not capture. In particular, the mode has the same value regardless of whether we have seen 8 out of 10 patients or 8000 out of 10000 patients being cured, but in the latter case, the distribution is much more tightly peaked (narrower) around the mode. This again agrees with our intuition: the more patient data we see, the more certain we become about the most likely value of α. The same remarks of course also apply to the probability of cure when gien the placebo.

 

To obtain the probability that the drug is effective, we pose two alternative models for the observed data: one where the probability of cure is different between patients receiving the placebo and patients receiving the drug, and one where the probability of cure is the same in the two groups. We then make use of an auxiliary, binary (0/1) random variable that switches between these two models. We can infer the probability of that variable being 1 (or 0), corresponding to the probability that there is (not) a difference between the drug and the placebo, again making use of Bayes’ theorem. This inference will need to take into account that the probabilities of cure when given the drug or the placebo are both unknown.

 

Additional Remarks

This demonstration, just like the Movie Recommender, has been developed using the probabilistic programming framework Infer.NET, developed at Microsoft Research Cambridge.