JimDavies problem statement There are lots of things we would like to know about scientists and how they work, and it seems that a scientific approach to answering many of these questions is appropriate. However, there is something about using science to study science that sounds circular. Is the scientific study of science self-serving or vacuous? Circular Reasoning ------------------ Let's first examine the nature of circularity in general. Circular reasoning, or begging the question, is a logical fallacy that is committed by having an argument, the conclusion of which is assumed as a premise. In its simplest logical form, P, therefore P. Where P represents some proposition, or statement. The above case looks trivial, but here is an example of an argument that may initially appear plausible with that very structure (Audi 1995): "To allow every man an unbounded freedom of speech must always be, on the whole, advantageous to the State; for it is highly conducive to the interests of the community, that each individual should enjoy a liberty perfectly unlimited, of expressing his sentiments." In the above passage, the conclusion is "To allow every man an unbounded freedom of speech must always be, on the whole, advantageous to the State." And the justification is "it is highly conducive to the interests of the community, that each individual should enjoy a liberty perfectly unlimited, of expressing his sentiments." Though on the face of it these two statements are different, they both the premise and the conclusion express similar meanings, or propositions, so the argument as a whole is circular. Let's look at a more complicated circular argument. 1. Whatever the Bible says is true. 2. The Bible says that God exists. 3. Therefore, God exists. Strictly speaking this argument is valid and free of fallacy. For an argument to be valid it means that if the premises are true the conclusion must be true. What makes the above argument potentially circular is that it may be necessary to assume conclusion #3 to establish premise #1. For example: -1. There exists a God who wrote the Bible. 0. Whatever God writes is true. 1. Things written in the bible are true. So now we have what we might call an implicit circularity. The argument 1,2,3 can only be circular if it is fleshed out to include the sub-argument -1,0,1. Only then is it technically circular, because given just 1,2,3, it could be that #1 is established by some other means that do not require the existence of God. This notion of implicit circularity is important to possible circularity in scientific study of science, as I will show below. The Scientific Study of Science ------------------------------- There have been a number of researchers who have advocated a scientific approach to science studies. Some have advocated cognitive approaches (e.g. Giere 1988, De May 1982, Langley et al. 1987), some sociological (e.g. Barnes et al. 1996, Shapin & Shaeffer 1985). Indeed, for certain questions, this seems entirely appropriate. For example, you might want to know how the language a scientist speaks affects her choice of which theories to pursue. You could look at a few examples and make a generalization, but science would suggest a better way: Randomly sample many scientits from two very different linguistic communities, control for other possible factors, and see if there is a significant difference between groups with respect to theory choice. You would want to do these things, not because they were some basic assumptions of science, but for more commonsense reasons that we happen to know from doing science for so long. For example, we know that without a random sample you may be more likely to be generalizing from a non-representative group. And you would want to test for significance because then you could find the size of the effect, or how likely the perceived difference is to be due to chance, rather than due to the effect you are looking for. In the above case you would be using scientific methods and recommendations in your analysis. Is there a danger of circularity? Circularity is a relationship between the conclusion of an argument and the premises, so without a conclusion it is impossible to tell. So let's imagine that the experiment hypothesizes that speaking Icelandic and not being able to speak English has a strong effect on which theory is chosen in the field of plasma physics. Is there circularity here? Let's examine what the argument might look like: 1. Controlling for all other reasonable factors, with a low (0.05) probability of error, if it is found that many random Icelanders (who spoke no English) chose a certain of kind of theory x more often than a random sample of English-speakers then you can conclude that speaking Icelandic and not being able to speak English makes one more likely to choose theories of type x (compared to people who speak English and not Icelandic). 2. It was found that the Icelandic group was significantly more likely to prefer theories of type x than the English-speaking group. 3. Therefore, The hypothesis was supported. 4. Therefore speaking Icelandic and not being able to speak English makes one more likely to choose theories of type x compared to people who speak English and not Icelandic. The conclusion is not stated in any of the premises, nor would the conclusion be necessary to establish any of the premises. For this example, there appears to be no circularity. This example is one instance of using science to study science that is not circular. Similar studies would be similarly non-circular as well. CIRCULARITY IN THE SCIENTIFIC STUDY OF SCIENCE So far, so good. Now I will show an example of a scientific study of science that IS circular. To do this, the conclusion must be one of the premises. The uniformity principle is an assumption made by scientists, first identified explicitly as an assumption by David Hume. It assumes that the world is uniform, and because of this you can make generalizations based on what you have seen. The uniformity principle allows you to do inductive reasoning. You have a class of observed instances, you see some pattern in those instances, and you assume that the pattern holds true for the population from which the sample is drawn. For example, we observe 200 white swans, and conclude that all swans are white. It is possible that some are not white, but with a large random sample we assume this is unlikely. Let's try to show evidence for the principle of uniformity: 1. If it appears in a meta-analysis of scientific papers that generalizations made from random samples were supported in future observations, then we can conclude that one can generalize to unobserved instances from a randomly sampled set of observed instances. 2. In a study of 500 scientific papers across many fields, it was found that generalizations made from observations of a random sample were supported in future observations. 3. The hypothesis was supported. 4. It is true that one can generalize to unobserved instances from a randomly sampled set of observed instances. On the face of it this reasoning appears non-circular. However, recall an argument shown previously that also appeared non-circular: 5. Whatever the Bible says is true. 6. The Bible says that God exists. 7. Therefore, God exists. Argument 5,6,7 is circular because you need the existence of God to establish premise #5. The assumption was tacit. Where was the tacit assumption in argument 1,2,3,4? In short, the conclusion #4 is the uniformity principle, and the uniformity principle must be assumed to justify premise #1. Why would we believe that those 500 scientific papers would tell us about any other papers if we didn't already accept the uniformity principle? How could we assume that those generalizations would be successful in observations that still have not been made? Only by assuming the uniformity of the world. I will flesh this idea out further. Imagine a study that shows that a certain substance, call it foo, burns green. To show this, the scientist would burn many pieces of foo, and see what color was produced. If the hypothesis is supported, he concludes that ALL foo burns green. Well, he didn't test ALL foo, so how can he make such a conclusion? He can conclude this because he assumes the uniformity principle. That is, he assumes the world is uniform-- foo here acts like foo over there, foo in the present behaves like foo in the past and future, etc. Back to our example. The scientist attempts to show evidence for the uniformity principle. How does he do this? Naturally, as any scientist would: He observes a sample of generalizations made in scientific papers and generalizes. What makes him think that those in the sample will tell him anything about the ones not in the sample? The uniformity principle makes him think that, the very principle he is attempting to establish. I have shown a case in which a scientific approach to studying science is circular. What is it about this case that makes it so? It is circular because the conclusion of the study is one of the fundamental assumptions of science. All scientific arguments tacitly if not explicitly assume the fundamental assumptions of science, so any scientific study of science that is attempting to show evidence for one of these assumptions is using circular reasoning. The situation is somewhat more complicated than this, however. There are several assumptions of science, and not every scientific argument requires all of them. So if a scientific argument attempts to show evidence for an assumption of science that the argument itself does not require, then it would not be circular. In this way it would be possible to establish scientifically some of these assumptions. For example, many psychologists assume that most continuous, quantitativly measurable psychological traits vary according to a normal distribution (Kirk 1995). In fact many statistical tests specify that the results are not to be believed unless the trait is normally distributed in the population you have sampled from (as opposed to the sample itself, which can actually be tested for normality). Is this assumption of normality testable? 1. If most traits in a large random sample of traits are shown to be normally distributed then we can conclude that most human traits are normally distributed. 2. In a large random sample of traits, 90% were found to be normally distributed. 3. Therefore, most human traits are normally distributed. The hypothetical scientific argument 1,2,3 would show evidence for the assumption that human traits were by and large normally distributed. And nowhere in the analysis was there an assumption that most human traits are normally distributed. Indeed, such an assumption would make no sense, since the overall distribution of traits is not a human trait at all. This argument is not circular because the conclusion established does not have to be assumed for the argument to work. So some of what we call assumptions of science are not really assumptions at all, but rational beliefs that can be supported scientifically. Of course, these arguments make assumptions of their own. Eventually it may bottom out to a few untestable assumptions. I speculate that some of these might be the uniformity principle and Ockham's razor, but I will leave this analysis for another paper. To conclude I will summarize my points and suggest some future analyses. In summary, circular reasoning occurs when the conclusion is assumed in the premises, explicitly or tacitly. Circularity in any scientific argument is the same. There are circular scientific arguments which try to establish one of the scientific assumptions they are using (e.g. showing evidence for uniformity), non-circular scientific arguments which establish things aside from scientific assumptions (e.g. showing the language effect on theory choice), and finally non-circular arguments that establish assumptions of science that aren't assumed for that particular argument (e.g. finding that most traits are normally distributed.) What does this mean for science studies? One thing this analysis shows is that there are many questions about scientists and what they do that are well suited for scientific research, and that for the most part these scientists will not be engaging in circular reasoning. Some scientists are already doing this (e.g. Dunbar 1995). But there are parts of scientific practice that may be impossible to study scientifically without using circular reasoning. Though I will not argue the point here, these parts are perhaps better analyzed by philosophical methods. Kitcher (1993) suggests that we should get our ideas of good science methodology based on how well those methodologies fared in the past. Though he is not explicit about it, such an approach is rather scientific. My analysis suggests that for some parts of science method, notably the uniformity assumption, such a program could result in circular reasoning. An in-depth study of Kitcher's program would be a reasonable next step. References: Audi, R., ed (1995) The Cambridge Dictionary of Philosophy. Cambridge, Cambridge University Press. Barnes, B., D. Bloor, & J. Henry. (1996) Scientific Knowledge: A Sociological Analysis. Chicago, IL, University of Chicago Press. De May, M. (1992). The Cognitive paradigm. Dordrecht, Reidel. Dunbar, K. (1995). How scientists really reason: Scientific reasoning in real-world laboratories. In R.J. Sternberg, & J. Davidson (Eds). Mechanisms of insight. Cambridge MA: MIT press. pp 365-395. Giere, R. (1988) Explaining Science: A Cognitive Approach. Chicago, IL. University of Chicago Press. Kirk, R. E. (1995) Experimental Design: Procedures for the Behavioral Sciences. Pacific Grove, CA. Brooks/Cole Publishing Company. Langley P., H. A. Simon, G. L. Bradshaw, J. M. Zytkow (1987) Scientific Discovery. Cambridge, MA. MIT Press. Shapin, S., S. Shaeffer (1985) Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. Princeton, NJ. Princeton University Press.