Statistical hypothesis Testings (Probability X Possibility)

In statistics, the main tool for modeling uncertainties is certainly the probability measure. Other measures like possibility, necessity, impossibility, and plausibility are not part of the menu in statistics, probability and related courses (physics, biology and so on). This fact is indeed a very strong limitation that us, statisticians, have to deal with. The price of it is a low understanding of the statistical thinking and modeling, since a classical statistical model is a meta-probabilistic one and hence more general tools are required to understand it in depth. In general,  the probability rules are justified in terms of frequencies (Laplace),  game theory with its own definition of “coherence” (Ramsey, de Finetti, Savage, Lindley and Kadane), or desiderata (de Finetti, Richard Cox, Jaynes and Paris), and so on. Basically, in the core of all these arguments it is embedded a strong linear constraint, namely: 1. the frequency justification is based on counting frequencies; 2. the probabilistic “coherence” definition is always dependent on arbitrary linear rules; 3. the more basic axioms (desiderata) always assume a strictly increasing constraint on the involved functions. All of these imposed constraints can be easily refuted as immanent attributes of the coherent reasoning. The coherent reasoning is much broader than a set of quantitative specifications, it is qualitative rather than quantitative and it is much more related with aesthetics than otherwise. Therefore any type of axiomatization of Coherence should not be considered as the final word.

It should be clear that probability can be used for modeling uncertainties but it should not be imposed as the unique tool. As professor Zadeh put it wisely “a problem arises when “can” is replaced with “should,” as in the following dictum of a noted Bayesian, Professor E. Lindley”:

The only satisfactory description of uncertainty is probability. By this I mean that every uncertainty statement must be in the form of a probability; that several uncertainties must be combined using the rules of probability; and that the calculus of probabilities is adequate to handle all situations involving uncertainty… probability is the only sensible description of uncertainty and is adequate for all problems involving uncertainty. All other methods are inadequate… anything that can be done with fuzzy logic, belief functions, upper and lower probabilities, or any other alternative to probability can better be done with probability (Lindley 1987).

The study of plausibility measures can significantly broaden our views regarding the modeling process of uncertain events. An attentive statistician would note that on testing very restrictive hypothesis (e.g., the Hardy-Weinberg equilibrium or any physical law. Formally, a very restricted hypothesis is written as H_0: \theta \in \Theta_0 were \mbox{dim}(\Theta_0) < \mbox{dim}(\Theta) and \Theta is the model parameter space) the best probability estimate is zero, i.e., the probability of very restricted hypothesis is zero. As the probability is zero, should we claim that this very restrictive hypothesis is impossible? of course not, since as all (probability) models are always approximations, “zero-probability” events does occur in practice (the examples abound and I am not going to enumerate them). In these “zero-probability” cases, we can have a value pointing out how discrepant is this very restrictive hypothesis with the observed data, this value is not attained from the probabilistic framework. If a positive probability is set for a very restrictive hypothesis, then many paradoxes emerge (vide Bayes Factors, Lavine and Schervish, 1999).

In statistical hypothesis testing, those who are called Bayesians are not willing for testing very restrictive hypotheses, since as aforementioned the probabilities of such hypotheses are always zero. The other school (the frequentist, likelihoodist school or simply classical school) handles the problem in very different way. They consider that probability can describe some uncertain events but not all.

If we perform an experiment, then the observed data may be ideally modeled by a probability measure, however in practice we don’t know which is the actual probability measure the governs the data behavior. We can create a family of probability candidates and then choose one (or a small set of) probability measure(s) that may be appropriated for modeling the data. The two schools act as follows:

a) Bayesian statisticians impose another probability measure over the initial family of probability measures, called prior probability measure, and then compute the so-called posterior probability measure. Then, the posterior probability measure is used for testing any type of hypotheses regarding the initial family.

b) Classical statisticians just treat the problem by considering full possibility for all elements of the initial family. They use the so-called p-value as a measure of evidence for testing any type of hypotheses regarding the initial family. Also, there are several other methods (confidence intervals, most powerfull tests, and so on).

The p-value is not even a plausibility measure  (the plausibility measure includes many important measures such as probability and possibility)  over the parameter space producing some logical “problems”. (It has a quasi-possibilistic  behavior). In a very recent paper “A classical measure of evidence for general null hypotheses to appear in Fuzzy Sets and Systems, it is proposed a possibility measure for testing very general hypotheses (including very restrictive ones) which is free of logical contradictions.

Therefore, possibility measures can be used to test very restrictive hypotheses while probability cannot handle with it. Possibility measures can be justified in terms of game theory (Dubois and Prade) and we can also create a definition of coherence that matches with the possibility rules. The limit of the probability rules is not the limit of the coherent reasoning, actually, the limit of any quantitative artifact is not the limit of the coherent reasoning.

Referencies:

Lavine, M., Schervish, J.M. (1999). Bayes Factors: what they are and what they are not, The American Statistician, 53, 119-122.

Lindley, D.V. (1987). The probability approach to the treatment of uncertainty in Artificial Intelligence and expert systems, Statistical Science, 2, 17-24.

Advertisements