Professor: As Abraham Maslow said, “when all you have is a hammer, the whole world looks like a nail.”
Student: The statement presupposes that one knows that something other than a hammer exists even if it is not available. It would not make sense if only “hammers” existed because there would be nothing to differentiate “hammer” from anything else. Indeed, there would be no word “hammer.” The saying must imply that there are known alternatives to hammers, yet still the whole world looks like a nail. What is that?
Professor: Willful ignorance.
Student: I would call that being stupid.
Professor: I was trying to be kind.
Student: They need to “own it.”
In addition to the remarkable clinical benefits attributable to Deep Brain Stimulation (DBS), it has become an increasingly used tool to investigate the operations of the brain. Scientists examine the effects of DBS on a specific target, such as the subthalamic nucleus, for various physiological or psychological functions, to infer the role of the target in the function. In the large majority of research publications, the effects of DBS are attributed to the structure being stimulated. Further, the effects are then thought to reflect the physiology or pathophysiology of the structure targeted by DBS.
Such inferences are invalid. This is not to say the inferences are true or false, only that the experiments (arguments) that contain the inference are invalid. For example, consider an experiment examining the effects of DBS targeted at the subthalamic nucleus on lexical function. Differences are noted in tests of lexical function when the stimulator is turned on compared to the situation when the stimulator is turned off. The inference is made that the subthalamic nucleus is involved in lexical functions. This is an example of the logical fallacy Post Hoc Ergo Propter Hoc (since b follows a, a must be the cause of b). This fallacy is a special case of the Fallacy of Limited Alternatives because b could be due to any number of causes other than a, particularly if a is complex and multifaceted.
In the case above, DBS in the vicinity of the subthalamic nucleus generates action potentials in virtually any axon in the vicinity. These include axons terminating on neurons within the subthalamic nucleus, which affects the next or post-synaptic neuron but also antidromically activates the neurons outside of the subthalamic nucleus. If these neurons have axon collaterals to other structures, then antidromic action potentials will proceed down the branch in an orthodromic manner to activate structures that have nothing to do with the subthalamic nucleus. Indeed, axons passing in the vicinity without any connection to the subthalamic nucleus may be activated.
The situation becomes even more complex. Within tens of milliseconds, activities generated in the vicinity of the DBS lead, such as in the subthalamic nucleus, can spread extensively. This spread is reinforced by resonance with repeated DBS pulses. Such extension with repeated pulses has been demonstrated by wider distribution of the electroencephalographic evoked potential to a brief train of DBS pulses in the vicinity of the subthalamic nucleus compared to single pulses (Baker KB, et al. 2002).
The DBS pulse is fairly indiscriminate in which axons within the volume of the electrical field are activated. Factors include proximity to the cathode, myelination, axonal diameter and axonal branching patterns. The DBS pulse stimulates anything where it is put, and targeting seldom obeys physiological organization. Further, the action potentials initiated in the axons are simultaneously affecting a great many axons. The DBS pulse is more analogous to a lightning bolt striking a large group of people and then inferring that the mass convulsions are a normal phenomenal. It is not as though a single person is struck and the effects on those remaining can be understood based on the physiological properties and contributions of that single strike.
None of the criticisms offered above is to say that DBS is not a valuable tool in research, quite the contrary. Clearly the clinical effects of DBS demonstrate the interactions between the DBS pulses and the physiology. Rather, the criticism is necessary to properly use DBS as a tool, particularly in drawing inferences from DBS experiments. Given the complexity of the responses to DBS, including those without any direct relationship to the target of DBS, how can anybody reasonably ascribe the effects of DBS to the physiology of the target? Specifically, how can anybody reasonable ascribe the effects of DBS in the vicinity of the subthalamic nucleus to physiology of the subthalamic nucleus? Note, anyone can say just about anything they want to say. However, if one wanted to be scientifically responsible, then one would withhold attribution or at the very least append the appropriate caveats. Yet this is seldom done.
One might argue that scientific reports making inferences attributing some property or function to the DBS target really is only describing the “facts” of the experiments and that inferences are only secondary or peripheral. They may argue that the reader is free to draw whatever inferences she may want. However, such as claim is not only disingenuous and self-serving, it is a disservice to science. It reduces science to “stamp collecting”. If that position was seriously adopted, then science would be reduced to the Solipsism of the Present Moment where no knowledge exists beyond very narrow and exact circumstances of a particular experiment. There would not be any generalizable knowledge. Most importantly, the actual observation resulting from a particular experiment could not be used to justify or support any future experiment. Science always would be just a set of dots (each dot corresponding to a specific and limited observation) and there would never be any lines connecting the dots. These lines are critical because it is interpolation between the dots onto the line and extrapolation of the lines connecting the dots that point to possible future knowledge.
Inferences are very important and consequently, it is important that inferences are sound. The epistemic importance of inferences is demonstrated above. The practical importance of inferences is seen in scientific publications. Most scientific publications are structured with introductions and discussions which serve the important purpose of connecting previous dots (in the introduction) to justify and set the context for the new dots (experimental observations). Finally, the new and old dots are interconnected by lines of inference to provide new knowledge and understanding and very importantly point the way to future research. Yet, it the great majority of scientific publications, the old dots are the past inferences and rarely if ever the actual observations of previous experiments. Thus, the past inferences take on the epistemic status of facts as they are used in lieu of the original observations in the argument that comprises the research. Inferences, regardless of their lack of soundness, become quasi-facts.
Most inferences derive from the hypotheses used in the scientific method. A hypothesis not refuted during the application of the Scientific Method is then taken as true and becomes the inference derived from the observational data. However, there is a flaw. The Scientific Method where a hypothesis gives rise to a testable prediction that if found is thought to validate the hypothesis. However, this use of the Scientific Method is the Fallacy of Confirming the Consequence which is of the form if a (hypothesis) implies b (prediction) is true and b is true, then a is true. However, b could be true for many reasons other than a. In the case of DBS in the vicinity of the subthalamic nucleus inducing a change in behavior or functions, the Scientific Method becomes if the subthalamic nucleus is involved in behavior A or function B, then DBS of the subthalamic nucleus should affect behavior A or function B, the experiments demonstrate this to be the case, thus the subthalamic nucleus is involved in behavior A or function B. However, as discussed above, DBS affects far more than the neurons in the subthalamic nucleus and these other effects could be responsible for the change in behavior A or function B. There is no way from the experiment alone to know which of the many possible responses to the DBS pulse is responsible for the change. Note, this does not mean that an effect of the DBS pulse directly on subthalamic nucleus neurons cannot mediate the change in behavior or function; only that one cannot conclude that is the case from the experiment alone.
Even if one were to grant that a change, such as a restoration of normal function, was consequent to DBS, this is not to say that the manner by which DBS restores the appearance of normal function is the same as the normal intact physiological mechanisms. The inability to attribute the mechanisms by which DBS restores a normal function to the same mechanisms that operate normally is due to the Inverse Problem. Typically, behaviors are mediated by the actions of the lower motor neurons in the brainstem and spinal cord. These neurons receive multiple and diverse inputs. Consequently, one cannot know from observing the consequence of lower motor neuron activations what is the source of the relevant actions onto the lower motor neurons.
To argue that normalization of behavior with DBS utilizes the same physiological mechanism normally involved in the behavior would be the logical Fallacy of Four Terms. Therefore, one cannot attribute presumed DBS-related mechanisms as providing insight into normal physiological mechanisms. The fallacy would be the following: Major premise – Normal function implies normal physiology; Minor premise – DBS restores normal function; and Conclusion – DBS is normal physiology. It would appear there are two terms, “normal physiology” and “DBS” that are linked through the third bridging term “normal function.” However, as described above, the “normal function” in the case of the normal condition may not be the same as “normal function” in the case of DBS. Thus, there are two different bridging terms, hence the fallacy of Four Terms. Again, it is important to appreciate that the fallacy does not mean that the DBS mechanisms cannot be the same as the normal mechanisms, but only that the experiment cannot provide confidence that they are the same. Once again, it is not that the scientists cannot offer the theory that normal mechanisms underlying the normal function is similar to DBS mechanisms. However, such inferences can only be tentative and thus, demand that the appropriate caveats be explicitly appended to any inferences, which is seldom done.
As can be appreciated, the inferences derived from demonstrating the predictions of hypotheses using the Scientific Method are indeterminate except when the experiments fail to demonstrate the prediction. In the latter case, failure to demonstrate the prediction is clear evidence for the falsehood of the hypothesis. Also, the Inverse Problem demonstrates that the same phenomena may have many causes and the phenomena alone cannot determine which cause is relevant. One has to bring knowledge extraneous to and independent of the experiment to help adjudicate which of the multiple causes is most probable. But it is important to note, the additional evidence cannot be directed solely at the inference that is being made. The additional evidence must be directed at all the reasonable alternative inferences. Failure to do so represents Confirmation Bias. Yet, rarely in scientific reports are alternative inferences and the relevant external knowledge ever raised.
The tendency towards limited reviews within the introductions and discussions of scientific papers risking Confirmation Bias has a long history. Charles Bazerman analyzed the rhetoric of scientific reports in the Philosophical Transactions of the Royal Society from 1665 – 1800 (Bazerman, C. 1997, pages 169-186). In the essay, Bazerman described four stages with indistinct boundaries. In the first period, 1665-1700, scientific papers were uncontested reports of events. In the second stage, from 1700-1760, experimental results appeared and the discussions centered over the results. It was not until the third period, 1760-1780, that papers “explored the meaning of unusual events through discovery accounts (Bazerman, C. 1997, page 184), which can be taken as discussions of the inferences drawn from the accounts.
During the fourth period, approximately 1790-1800, experiments were reported as claims for which the experiments were to constitute proof, but most were the Fallacy of Confirming the Consequence as described above. Further, experimenters were presenting claims that were solely the result of their individual efforts and not “recognizing the communal project of constructing a world of claims… Although the individual scientist has an interest in convincing readers of a particular set of claims, he does not yet explicitly acknowledge the exact placement of the claims in the larger framework of claims representing the shared knowledge of the discipline” (Bazerman, C., 1997, page 184).
The operative words are “communal project” and “shared knowledge of the discipline.” Since the founding of the Royal Society of London in 1661, the community charged with the scientific project was highly selective (Shapin S., Schaffer S., 1985). Public demonstrations of experiments meant to vouchsafe the results were limited to a select group, perhaps a very early instantiation of Confirmation Bias. Critics, such as the famous natural philosopher, Thomas Hobbes, were vigorously excluded. Certainly, the historical analysis of the progress of modern science by Thomas Kuhn (1962) demonstrates that this sort of exclusivity maintains Confirmation Bias even today.
In the final analysis, the state of knowledge regarding DBS is such that inferring neurophysiological functions from DBS effects is not possible. Rather, only possibilities can be offered and consequently, all reasonable possibilities should be considered. Papers attempting to do so have the obligation to clearly state the tentative and problematic nature of such inferences, which is best accomplished by vigorously presenting alternatives based on a thorough understanding of all the relevant literature. Editors of scientific journals should insist that this be done.
References:
Baker KB, Montgomery EB Jr, Rezai AR, Burgess R, Lüders HO. Subthalamic nucleus deep brain stimulus evoked potentials: physiological and therapeutic implications. Mov Disord. 2002 Sep;17(5):969-83. PubMed PMID: 12360546
Bazerman, C. “Reporting the Experiment: The Changing Account of Scientific Doings of the Philosophical Transactions of the Royal Society, 1665-1800”, in Landmark Essays on Rhetoric of Science: Case Studies, Vol. 11, R. A. Harris ed., Hermagoras Press, 1997, pages 169-186
Kuhn T. The Structure of Scientific Revolutions, University of Chicago Press, 1962
Shapin S, Schaffer S. Leviathan and the Air-Pump: Hobbes, Boyle, and the Experimental Life. Princeton University Press, Princeton, New Jersey, 1985, ISBN 0-691-08393-2