Putting Falsifiability in its Probabilistic Place

Karl Popper's view that "the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability" (all quotes come from Popper's essay, "Conjectures and Refutations") is now nearly canonical in the thinking of many scientists and scientifically minded people. According to Popper, a theory counts as scientific if it is, in principle, open to falsification. On his view, falsifiable theories are those that are risky, make prohibitions, and can be contradicted by possible observations. In part, Popper's motivation for seeing falsifiability as the quint-essential virtue of scientific thinking emerged from his (mistaken) belief that empirical confirmation is easy to come by (for more, see my other blog post). Freudian psycho-analysis, Adlerian individual psychology, and Marxist theory of history (Popper's three unfalsifiable punching bags) seemed capable of accommodating any relevant phenomenon and enjoying an "incessant stream of confirmations." For any human behavior, x, the theories of Freud and Adler seemed capable of explaining x in their own terms, hence finding "confirmations" all over the place. However, according to Popper, these theories lacked risky claims that could be contradicted by possible observations. That is, these theories were/are not falsifiable. Far from being a virtue to commend such theories as acceptable, their allegedly unfalsifiable status indicts them as unscientific.

In this blog, I'll briefly explain why falsifiability isn't what really matters when it comes to good scientific theories. As I will argue, what really matters is that a theory be sensitive to evidence, either positively or negatively. Theories that meet this qualification are capable of being falsified and rejected (strict falsifiability is hard to come by, so it's often preferable to speak of disconfirmation, where theory T is disconfirmed by observation statement E iff Pr(T|E) < Pr(T)). Furthermore, it will be seen that the "incessant stream of confirmation" allegedly received by the theories stated above were not really confirmations at all [1]. A theory that can be genuinely confirmed will also be open to genuine disconfirmation. The real problem with the above theories (as Popper understood them) is not that they are unfalsifiable, but that they are not sensitive to evidence.

A theory T is sensitive to evidence E (or has empirical traction) iff Pr(T|E) > Pr(T) or iff Pr(T|E) < Pr(T). In both of these cases, E is relevant to T, either increasing or decreasing its posterior probability. Moreover, J.L. Mackie's relevance criterion states that:

Pr(T|E) > Pr(T) iff Pr(E|T) > Pr(E|~T).

Mackie's relevance criterion gives us another way of thinking about evidential sensitivity. We can say that T is not sensitive to evidence iff

(1) Pr(E|T)/Pr(E|~T) = 1

Obviously, in order for the ratio in (1) to equal 1 it must be the case that Pr(E|T) = Pr(E|~T). Hence, T is not sensitive to the evidence, since the righthand inequality in Mackie's relevance criterion is not satisfied. So we can interpret all of this as saying that evidentially sensitive theories are either (a) those that lead us to expect some phenomenon more than alternative theories do, or (b) those that lead us to expect some phenomenon less than alternative theories do (of course, we want to construct theories that satisfy (a))[2]. If either (a) or (b) is true w.r.t. to T, then Pr(E|T)/Pr(E|~T) ≠ 1.

The unfalsifiable theories Popper complains about fail to satisfy (a) or (b). Freudians may have felt good when their theory seemed capable of explaining some behavior, B, but their joy was unsupported. Alderian individual psychology, a competing view, seemed just as capable of accounting for B. This can be interpreted as meaning that Pr(B|F) = Pr(B|~F). Consequently, Pr(B|F)/Pr(B|~F) = 1. So neither theory actually received any confirmation from B –– the "incessant stream of confirmations" was illusory.

The importance of the Bayes' Factor Pr(E|T)/Pr(E|~T) in theory testing also helps clear up another issue related to falsifiability that I've been wracking my brain over. I interpreted Popper's concern as being with theories that claim to predict both the occurrence of some event, as well as its absence. In other words, I understood him to be taking issue with theories for which the following is true:

(2) Pr(E|T)/Pr(~E|T) = 1

This means that Pr(E|T) = Pr(~E|T) = 1/2, since Pr(E|T) + Pr(~E|T) = 1. Popper seems to express this concern with the Marxist theory of history, M. If there is war in Popperland (some imaginary country), adherents of M exclaim, "aha! The proletariate are uprising." If there is peace, proponents of M say, "aha!" The bourgeoise are oppressing the proletariate." Both war and peace are thought to be equally predicted on M (or so Popper complained). So, in addition to the case made earlier, I understood Popper's charge of unfalsifiabiliy to extend over theories for which (2) is true. It's easy to believe that a theory T evades falsification when the occurrence of any event relevant to T is predicted just as strongly as its absence. It makes it seem like anything that could disconfirm T is actually predicted by T. Whether this is what Popper actually hand in mind or not is unclear. In any case, (2) is neither a sufficient or necessary condition for unfalsifiability. That is, a theory for which (2) is true is not necessarily unfalsifiable. Why? Because Pr(E|~H) is still relevant. So even if (2) is true with respect to T, it might turn out that Pr(E|T) < Pr(E|~T), making it the case that Pr(T|E) < Pr(T). Conditions (a) and (b), mentioned above, can still be satisfied, leading to the confirmation or disconfirmation of T. 

What really matters, then, is that a theory be sensitive to evidence by yielding predictions that are improbable on alternative theories. If it can do this, it can also gain the acclaimed status of falsifiable. Hence, falsifiability is achievable because of other probabilistic things going on with a theory. It is those things –– discussed above –– that really matter and which science should be interested in.

Girls don't want to date insensitive guys, and science doesn't want insensitive theories. Unfortunately,  many girls fail to realize what they're really looking for –– as did Popper. And I guess I failed to get what I was looking for: an academic and serious blog that didn't end with a laughable and awkward dating analogy. It might be appropriate to note that Popper, who rejected the idea of confirmation, would not have acknowledged the greatness of a Bayesian diagnosis of unfalsifiability (because of his weird views on confirmation), just like that cute barista at Biggbe Coffee barely acknowledged my presence the other day, even though I went out of my way to buy a chocolate muffin from her (which wasn't even that good!). Yeah, yeah –– maybe I'm being overly sensitive. But hey, with respect to theory testing, high sensitivity is a good thing (as this post has argued). Okay, enough with the equivocation and ridiculousness [3].
___________________________________________________________________
Footnotes:

[1] It's doubtful that Popper actually thought these theories received such streams of confirmation, given his views on confirmation. He probably means something like, "adherents of these theories believed their theories were receiving incessant streams of confirmation."
[2] For more on this, see Wesley Salmon, The Foundations of Scientific Inference. Also, my point can still be made even if I were to soften the conditions by saying that evidential sensitivity is lacking when Pr(E|T)/Pr(E|~T) ≈ 1 (i.e., Pr(E|T) ≈ Pr(E|~T)). This allows for variations in the degree to which T and ~T predict E and, hence, is more realistic. For simplicity, however, I will just take it be that Pr(E|T) = Pr(E|~T).
[3] A huge thanks to Tim McGrew for helpful conversations related to the content of this blog.