This was not a full consideration, but was a commentary on Popper's reformulation of his German work later in Conjectures and Refutations, linked to as part of a discussion in Why Evolution Is True. My original comments (numbering added) were:
1."It is easy to obtain confirmations, or verifications, for nearly every theory — if we look for confirmations."
This is very close to setting up a straw man, which is not a very good way to begin.
Every statistically controlled experiment or study sets up a bar for confirmation which in my opinion is not making it easy to find confirmation. We are apparently meant to assume the crudest and simplest (most naive) kind of activities, it appears.
2."Confirmations should count only if they are the result of risky predictions; that is to say, if, unenlightened by the theory in question, we should have expected an event which was incompatible with the theory — an event which would have refuted the theory."
Unenlightened by the germ theory, we should expect people to die, perhaps from cancer, without any signs of microbes. A risky prediction that the germ theory makes is that fatal diseases will be accompanied by germs, a risky prediction promptly disconfirmed by the autopsy of a victim of lung cancer. Ergo the germ theory is scientific, but wrong. This erroneous conclusion demanded by this principle of Popper’s suggests that no theory can be stretched beyond what it’s evidential foundation and internal logic permit, whether that allows "risky" conjectures or not. The failure of risky prediction may merely mean that the theory’s limits of validity have been reached.
3."Every "good" scientific theory is a prohibition: it forbids certain things to happen. The more a theory forbids, the better it is."
It is not clear how this is very different frm saying that the more a theory predicts, the better it is. The cell theory forbids that living things be composed of anything but cells or cell products. This formulation makes the cell theory sound like it is refuted by the abiogenetic origin of life.
Worse, this formulation forgets the most basic prohibition in science, the repudiation of the supernatural. Far better to say that every good scientific theory accepts the metaphysical postulates of philosophical materialism.
4."A theory which is not refutable by any conceivable event is non-scientific. Irrefutability is not a virtue of a theory (as people often think) but a vice."
This is just wrong. String theory or the multiverse concept may be wrong, and are as of this time immune to testing and refutation, but they are scientific in that they are deductions from well-tested scientific theories. Comparing the spectra of the Moon and green cheese to test the theory they are the same makes for a testable theory. The theory that there is however a Moon in a green cheese in another galaxy however is impossible to refute. According to this criterion, the first is a scientific
theory, but the latter is not.
5."Every genuine test of a theory is an attempt to falsify it, or to refute it. Testability is falsifiability; but there are degrees of testability: some theories are more testable, more exposed to refutation, than others; they take, as it were, greater risks."
The testability of theories lies partly in the phenomena they attempt to explain. This is not relevant to the scientificity of the theory. This criterion seems to be what I think they call a category mistake.
6."Confirming evidence should not count except when it is the result of a genuine test of the theory; and this means that it can be presented as a serious but unsuccessful attempt to falsify the theory. (I now speak in such cases of "corroborating evidence.")"
Again, disease germs are not found in all dead bodies. I suppose Virchow’s postulates could be rewritten as negative, disproof steps, but it is hard to see how this is useful. Worse, in complex phenomena, false negatives due to overextension of the theory will be far more likely. The real need for theories to be compatible with each other is entirely overlooked. (Incidentally, this is not really different from the insistence on the insistence on "risky" hypotheses.)
7."Some genuinely testable theories, when found to be false, are still upheld by their admirers — for example by introducing ad hoc some auxiliary assumption, or by reinterpreting the theory ad hoc in such a way that it escapes refutation. Such a procedure is always possible, but it rescues the theory from refutation only at the price of destroying, or at least lowering, its scientific status. (I later described such a rescuing operation as a "conventionalist twist" or a "conventionalist stratagem.")"
Only the simplest phenomena do not require numerous auxiliary hypotheses in theories. Further, life is ad hoc and it is not at all clear what possible criteria Popper has for labeling an auxiliary hypothesis ad hoc. Is the breaking of a natural damn an ad hoc hypothesis for the Channeled Scablands, which would otherwise refute theories of stratigraphy? A legitimate philosophy of science would be asking those questions, I thnk, not foolishly denying the necessity of auxiliary hypothese in perfectly good science.
8."We can sum up all this by saying that the criterion of the scientific status of a theory is its falsifiability, or refutability, or testability."
There are many trivial experiments. There are many false negatives from significant experiments. Experimental results must be interpreted in light of theories of many kinds of phenomena, and the experimental evidence for them, which means that scientificity also lies in the unitarity of explanations of nature. A parapsychologist can hypothesis ESP, make a risky prediction, test it, fail to refute it, yet we can confidently assert that sending information faster than light is impossible and that parapsychology therefore is not a scienctific enterprise. No matter how much it fits Popper’s criteria.
I understand that Popper once denied the theory of natural selection scientific status, which seems to me a natural conclusion of these criteria. Any experiments that falsify natural selection a la Popper really do tend to be rather trivial, don’t they?
Second thoughts (you knew this was coming didn't you?):
1. By the same token, it is easy to obtain falsifications for any theory, if we want to look for them. Sometimes it seems that those who adhere to a falsificationist perspective do so because it makes it easy to rule out tout court whole sciences without troubling to actually refute them. Popper personally, as The Open Society and Its Enemies shows, was primarily concerned to deny any social study the dignity of science. Creationists in my observation have very commonly adhered to a falsficationist perspective because, as an historical science, evolutionary theory is particularly vulnerable to Popperian falsificationism.
2. Additionally, some theories are complex enough that it is difficult to "predict" what should happen. Economics in particular in the social sciences has very clear, very simple mathematical models that simply do not permit of easy conclusions. If you look at the basic theories of standard economics, you would not predict a boom and bust economy. Marxian economics does. Score one for Marx and kick out the academics? Popper would have been the first to scorn the idea, as evidenced by his membership in the Mont Pelerin Society with the likes of Friedrich Hayek. No, there are a panoply of ad hoc explanations, auxiliary hypotheses, etc. Marxian economics of course has its own difficulties of this kind. The theory admits of moral factors, i.e., social custom and statutory law created by the activities of labor and capital, in the level of real wages. How then do you predict whether real wages inevitably decline over the secular trend? The answer, that you have to actually study real economies, globally and long-term, is not an evasion, marking an unscientific project. It is the mark of a real science that the theory should be extended and interpreted by real world information. Popper's insistence that the barebones of any theory have to be stuck too is remarkably metaphysical and dogmatic and all sorts of bad things that are opposed to real science. This is apparently the substance of the Duhem-Quine objection to falsificationism.
3. The more specific a theory is about its limitations, the better science it is. But I don't think this is what Popper means here. Suppose you have a theory that chieftains in technologically primitive societies compel their followers to increase production for their personal consumption and their ability to reward subordinates with the produce. Then by Popper's lights you are supposed to conclude that chieftains doing neither with the produce, but giving it away, even to non-subordinates, or even destroying the produce, is thereby impossible, in order to be doing good science. Frankly I think just making a simple example (by no means an absurd one either,) shows how wacky this is. He would then conclude that potlatch refutes the theory, and presumably the whole thing needs to be approached. Nonsense!
4. It occurs to me that I need to emphasize the "conceivable" is an equivocation. On reflection it occurs to me that Popper may have been trying to overextend the notion that, if a theory has two explanatory principles and the defender can switch at will from one to the other, then the critic cannot possibly refute the theory. (Psychoanalysis was probably the main offender in mind here.) The answer of course is that you refuse to accept this equivocation by the theory's proponent. You ask, in effect, exactly when is a cigar just a cigar? Then you see what the theory does and does not explain. Confusing this issue with the practical possibility of experiments may be useful in lambasting the historical sciences, but has no legitimate use in honest discourse.
5. Definitive experiments that refute hypothesese are much adored in science for good reason. They are also remarkably hard to come by. The numerous successes in evolutionary science in explaining facts in morphology, taxonomy, embryology and biogeography made Darwinism a successful scientific theory before someone contrived a controlled experiement that could refute it. Despite some facts that argued against it. By Popper's standards, evolutionary science was falsified. Again, the primary reason for this criterion appears to be to rule out by definition the very possiblity of an historical science. The problem with erecting the falsification standard is that it puts exceptions against mountains of examples following the rule without any standards as to why this should be so.
6. Rephrasing: There is a mountain of evidence confirming that people will claim to have private revelations from mystical sources and those who believe will discount any evidence disproving it. But this evidence should not count because we need to have a serious series of experiments showing that believers really will rationalize any disproofs of the revelation. Quite aside from the likelihood that detailed studies would show that some people do in fact reject the revelation, the injunction that we need to waste time on nonsense like this is far from a useful philosophy of science. Much less science. And, again, this is not a particularly absurd example!
7. All falsificationist/testbility perspectives founder on the inability to distinguish ad hoc assumptions from others. If you can't define ad hoc, the whole thing is a prime example of a theory for which there is no conceivable refutation!
8. Theories are scientific if they are materialistic; causal; consistent; comprehensive and confirmed by experience. Materialistic means that there are no supernatural agents or means. Causal means they identify cause and effect in an explanatory narrative, which includes identifying the role of random factors. Random factors may include repeated random occurrences whose effects can be probabilistically incorporated into the theory. Consistent means the elements of the theory are used in the same ways within the theory and the elements are consistent with other parts of nature as known. Comprehensive means that the theory aims to explain as much as possible, and to predict as specifically as possible. Confirmed by experience means historical experience as well as experiments.