The inevitability of confirmation bias and how it affects physics
The Discovery of N-rays
Professor René Blondlot is about to make physics history.
Just last year, he announced his discovery of a previously unknown part of the electromagnetic spectrum: N-rays1–4. Allegedly, these rays are emitted by almost all objects and have a host of bizarre properties, like the ability to change the brightness of sparks and improve the resolution of distant objects4–7.
News of his discovery spread rapidly throughout the physics community, but many were unable to reproduce his experiments. Amidst the resulting scepticism, Nature sent Professor Robert W. Wood to Blondlot’s lab to investigate things further6–8.
This is where we are now – a dimly lit physics laboratory at the University of Nancy. Robert Wood watches with scrutiny as Blondlot and his lab assistant run through their demonstration. The alleged N-rays are split using an aluminium prism and observed using phosphorescent paint on a screen, which should glow ever so slightly when the rays impinge upon it2,5,7. In order to see the effects, the room has to be very dark, and the eyes of the observer sensitive to even the most minute changes7,9. If Blondlot is right and N-rays do indeed exist, a distinct spectrum will be observed5,7. With everything meticulously set up, all that is left to do is to let physics do its thing.
Wood watches with anticipation and sees something truly remarkable: absolutely nothing. Blondlot on the other hand, is absolutely convinced that the spectrum is distinctly visible, and he attributes Wood’s inability to see the pattern to a “lack of sensitiveness in the eye”5. In light of these assertions, Wood proposes a follow-up test: he would adjust the aluminium prism, and Blondlot would have to tell when the changes were being made and how the prism was aligned5.
Secretly though, Wood removes the prism altogether, ruining any chance of the spectrum being observed3,5. But neither Blondlot nor his assistant detect any change – they remain convinced that they see the same pattern, utterly oblivious to what has happened1,3,5–8.
Wood’s conclusions are quite self-explanatory:
“I am not only unable to report a single observation which appeared to indicate the existence of the rays, but left with a very firm conviction that the few experimenters who have obtained positive results have been in some way deluded5.”Robert W. Wood, in his paper The n-Rays5
Coming back to the present, we now know that the purported N-rays were just a figment of the imagination1,3,7–9. This is true not just of Blondlot and his assistant, but also of the other physicists who claimed to have replicated his experimental findings. How could all of these scientists have believed so firmly in the existence of non-existent rays? The answer is confirmation bias, and is also the delusion that Wood alluded to. This involves favouring information that conforms to our beliefs while ignoring evidence to the contrary10–13.
The reason for this bias is not entirely clear, but it likely stems from our natural impulses11, such our tendency to prefer coherent narratives in thinking. This has been shown in MRI studies, where observed brain activity suggests active yet unintentional attempts to minimise cognitive dissonance11,12,14. This leads to both avoidance of contradicting information because we do not like to be wrong, and seeking supporting evidence because we like to be correct11.
In physics, there are two main ways in which this happens. The first of these is the biased search for information9,11. In Blondlot’s experiment, he was explicitly looking for a miniscule change in the luminosity of phosphorescent paint. It appears that he tried so hard that he managed to force it into (his) existence!
The second way is through the biased interpretation of information9,11,15. This could be through assigning more weight to certain results, or by simply ignoring evidence that supports alternative explanations. As previously mentioned, one of the apparent features of the N-rays was that it could change the brightness of sparks4–7. It turns out that these changes were in fact due to normal, random fluctuations that sparks usually undergo regardless of any intervention5,7. However, when Blondlot observed such fluctuations, he quickly attributed them to the effect of the N-rays4,5,7. This just goes to show that our expectations can really distort how we view things.
What should I do?
The damage can be great if confirmation bias is ignored. Incidentally, Blondlot’s reputation was brought to its knees, and he never recovered from the resulting public humiliation1,3,7. Allowing biases to remain unchecked during the research process can also diminish both the quality of collected data and the accompanying analysis.. Imagine having to repeat an experiment from scratch all because of this… ugh! It can even sustain false theories despite contradicting evidence16.
Eradicating the bias, however, is not always a simple process. Consider the following question:
Q: Your physics experiment stubbornly refuses to obey the laws of physics. What should you say? (You may choose one or more answers.)
(A) “It’s probably just random error”
(B) “I’ll fake the data… Who could possibly know?”
(C) “What have I done wrong this time?”
(D) “Physics is broken!”
Options A to C all describe situations of confirmation bias – in some sense, a denial of the existing evidence. Option D would therefore be the “correct” option, right? As you can probably tell, things are not so simple. For instance, if the physical law that is violated is very well established experimentally, there is probably more reason to believe that something has gone wrong than to declare “the end of physics as we know it”. Of course, there is also some chance that the law is indeed wrong, but this would be very unlikely and the chances would have to be weighed accordingly17. You could therefore imagine how difficult it might be to suggest that a widely accepted law is flawed in some way.
Although it is practically impossible to eliminate confirmation bias completely11,12,16, there are fortunately still some things that we can do to mitigate it.
- List ideas as to why the hypothesis might be incorrect10–12. It can also be helpful to list alternative hypotheses and to see whether these are consistent with findings. Knowing about these other possibilities can help limit how much we search for evidence that only supports a particular view.
- Constantly ask yourself whether you are processing information in biased fashion10–12,17. What is your first reaction to a book titled “The Moon Landing Hoax”? Why do you have that particular impression? Would it be a good idea to look at what the book says? Doing this does not mean that you have to agree with whatever you see; rather, it means being aware of our reasoning and identifying weaknesses in how we see information.
Beyond the individual level, there are also other ways of combatting confirmation bias. One of these is scientific peer review, although it is important to note that this process is itself prone to the bias18! It is also useful to have attempts to replicate the original experiment. As a matter of fact, this was what eventually spelled doom for Blondlot’s N-ray theory – the failure to replicate the experiment consistently and independently3,7. These methods are far from being a cure-all, however16.
With the benefit of hindsight, it can be very easy to dismiss the N-ray debacle as a singular case of delusion, and to think that this is not relevant to ourselves. But it is important to realise that no one is infallible. Everybody is susceptible to confirmation bias, and if we’re not careful, there’s no telling whether or not we might see N-rays of our own.
By Anson Ho
- Simon M. Fantastically Wrong: The Imaginary Radiation That Shocked Science and Ruined its “Discover.” Wired. Published 2014. https://www.wired.com/2014/09/fantastically-wrong-n-rays/
- Weart S. A little more light on N rays. American Journal of Physics. Published online March 1978:306-306. doi:10.1119/1.11342
- Lagemann RT. New light on old rays: N rays. American Journal of Physics. Published online March 1977:281-284. doi:10.1119/1.10643
- Blondlot R. “N” Rays: A Collection of Papers Communicated to the Academy of Sciences. Longmans, Green; 1905. https://archive.org/details/nrayscollectiono00blonrich/page/n29/mode/2up
- Wood RW. The n-Rays. Nature. 1904;70(1822):530-531. https://zenodo.org/record/1429443
- Pilkington M. N-rays exposed. The Guardian. Published 2004. https://www.theguardian.com/education/2004/sep/02/research.highereducation4
- Nye MJ. N-Rays: An Episode in the History and Psychology of Science. Historical Studies in the Physical Sciences. Published online January 1, 1980:125-156. doi:10.2307/27757473
- Hines T. What Ever Happened to n-Rays? Skeptic. https://www.skeptic.com/eskeptic/10-10-13/
- Jeng M. A selected history of expectation bias in physics. American Journal of Physics. Published online July 2006:578-583. doi:10.1119/1.2186333
- Rasmussen L. Confirmation bias: 3 effective (and 3 ineffective) cures. Global Cognition. Published July 14, 2020. https://www.globalcognition.org/confirmation-bias-3-cures/
- Shatz I. The Confirmation Bias: Why People See What They Want to See. Effectiviology. Published 2020. https://effectiviology.com/confirmation-bias/#How_the_confirmation_bias_affects_people
- Noor I. Confirmation Bias. Simply Psychology. Published June 10, 2020. https://www.simplypsychology.org/confirmation-bias.html#imp
- Nickerson RS. Confirmation Bias: A Ubiquitous Phenomenon in Many Guises. Review of General Psychology. Published online June 1998:175-220. doi:10.1037/1089-26188.8.131.52
- Westen D, Blagov PS, Harenski K, Kilts C, Hamann S. Neural Bases of Motivated Reasoning: An fMRI Study of Emotional Constraints on Partisan Political Judgment in the 2004 U.S. Presidential Election. Journal of Cognitive Neuroscience. Published online November 2006:1947-1958. doi:10.1162/jocn.2006.18.11.1947
- Koehler JJ. The Influence of Prior Beliefs on Scientific Judgments of Evidence Quality. Organizational Behavior and Human Decision Processes. Published online October 1993:28-55. doi:10.1006/obhd.1993.1044
- Ball P. The trouble with scientists: how one psychologist is tackling human biases in science. Nautilus. Published May 14, 2015. https://nautil.us/issue/24/error/the-trouble-with-scientists
- Greenwald AG. Consequences of prejudice against the null hypothesis. Psychological Bulletin. Published online 1975:1-20. doi:10.1037/h0076157
- Mahoney MJ. Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cogn Ther Res. Published online June 1977:161-175. doi:10.1007/bf01173636