Confidence limits of the probability of success in animal experiments and clinical studies: A Bayesian approach

Introduction 

Clinical studies and animal experiments where all outcomes of a treatment can be classified just as positive (success) or negative (failure) outcomes are subject to binomial statistics. In such studies, the unknown parameter is the probability of the positive/negative outcome, p. It is the determination of its most probable value together with its mean, variance and confidence limits which is of primary interest in these studies. The importance of reporting confidence intervals for this type of studies was emphasized by Shakespeare and Holecek [1] The idea of confidence limits and of fiducial intervals has been introduced in the early thirties of last century by several prominent statisticians [2], [3], [4], [5], [6]. The first work concerning the “confidence or fiducial limits” in the binomial case was published by Clopper and Pearson [3]. An approach based on the idea of the fiducial limits is discussed in Ref. [7]. It is worth mentioning that the fiducial approach was criticized by Neyman [5], [6]. Indeed, this approach to the problem as described in Collet [8] experiences several difficulties. For instance, if k is the number of positive/negative events and n is the number of trials, the standard formula [8], [9] for the variance (uncertainty) of p is , where is the estimated (most probable) value of p. It is obvious, that these formulae should not be used in marginal cases, namely when and are smaller than around 5, i.e. when the number of trials is rather small and/or the number of successes or the number of failures is around zero.

Due to the fact that the random variable p is defined in a closed interval, it is more likely that its probability distribution function is asymmetric. This makes the report of confidence intervals, rather than the variance, a more adequate issue in this case. According to the traditional definition of the lower and upper limits of the 100 (1−α)% confidence interval for p given in the orthodox statistical books (see Refs. [8], [9] for example) the following equations must be solved:

(1)

Therefore, we advocate here a different, Bayesian approach to the problem (Bayes [10], Laplace [11]). This approach has been reintroduced (Jaynes [12], Jeffreys [12], [13], [14]) and applied to the treatment of astrophysical and physical data [15], [16]. In the described experiments the known parameter is the observed number of successes/failures and the unknown stochastic variable is the probability of success, p.

Method and results 

If the probability distribution for p was known, on its basis the upper and lower limits of a given confidence interval could be defined and determined in the usual way in which confidence limits of a random variable with a known distribution are found. The problem thus could be more precisely formulated as a problem of constructing the probability distribution of p as a function of the observed number of successes k and the number of trials, n.

What Laplace did more than 200-years-ago (Laplace [11], see also Jaynes' Probability theory: the logic of science [14]) was to propose the following probability density distribution of p:

(2)

By differentiating the distribution function (Eq. (2)) with respect to p one finds the most probable value of p, , that will give the observed number of successes k in n trials:

(3)

Based on Eq. (2) the mean value of p, and its variance can be calculated (Laplace [11]) giving:

(4)

The lower limit pL of the 100(1−α)% confidence interval can be determined through solving the following integral equation for pL:

(5)

The upper limit pU of the 100(1−α)% confidence interval is determined through solving a similar equation for pU:

(6)

The constants A1 and A2 should satisfy the following relation, A2−A1=1−α, so that combining Eqs. (5), (6) one gets

(7)

There exist different ways of constructing a second equation for A1 and A2. The following two are the most commonly used approaches.

(A)For symmetry reasons one may choose: and .

(B)Alternatively, one may impose the requirement that .

Definition (B) determines the shortest possible 100(1−α)% confidence interval. The equations for the upper and lower confidence limits can be more easily solved using the realization that the integrals are incomplete β functions. MATLAB codes for calculating the confidence limits according to the above definitions are presented in Fig. 1a,b.

View Large Image Download to PowerPoint

Figure 1 MATLAB codes for calculating the confidence limits according to (a) definition A and (b) definition B.

However, none of the above described ways of determining pL and pU can be applied in the case of k=0 or k=n. Indeed, in the case of k=0 when the most probable value is , it is obvious that pL should be set to 0 because otherwise the most probable values of p will be excluded from the 100(1−α)% confidence interval. The upper limit is determined through solving the equation wherefrom . Thus, in case of k=0 the true value of p lies between zero and with 100(1−α)% probability. Analogously, in case of k=n, pU is chosen equal to 1 and , so that the true value of p is between and 1 with 100(1−α)% probability.

Discussion and conclusion 

It is worth mentioning the fact that due to the complexity in solving the definition equation (Eq. (1)), for pL and pU, there exist a number of papers [17], [18], [19] in which different approximations to the confidence limits are discussed, others [20], [21], that include just tables of values of confidence limits and even internet addresses (see Table 1) for their calculations, while the proposed method provides a simple and exact procedure of calculating the confidence limits for different confidence intervals and different values of n and k as shown in the Matlab codes provided here. These codes could be easily incorporated in any Matlab code developed by the researcher for clinical or animal experiments data treatment.

Table 1. List of web-sites

	Web sites of computers of binomial confidence limits
	http://statpages.org/confint.html#Binomial
	http://www.causascientia.org/math_stat/ProportionCI.html
	http://www.swogstat.org/stat/public/binomial_conf.htm
	http://www.measuringusability.com/wald.htm
	http://www.graphpad.com/quickcalcs/ConfInterval2.cfm

In conclusion, one may say that the main advantage of the proposed method is its inner consistency and logic. It can be shown that there exists a nearly perfect coincidence between the values of pL and pU calculated according to this method and the values of pL and pU calculated according to the normal approximation to the binomial distribution in the range of its validity, that is for large n and . Moreover, it also has the advantage that it provides exact solutions for the confidence limits for any values of n (large and small) and for all possible values of k, including the extreme values of k=0 and k=n, thus avoiding the limitations of the normal approximation. Therefore, it allows the treatment of studies in which the available statistic is small (small n) and/or the events of success or failure are rare (k=0 or k=n), cases in which the normal approximation is inapplicable. Since, small statistics and rare events of a specific outcome are typical for animal experiments and clinical trials, the advocated method should be regarded as a helpful tool for the medical researcher.