Saturday, September 12, 2015
CORRELATION IS NOT CONFIRMATION:
In this season of ramping up the hysteria toward the 2016 presidential election we are already hearing about the results of many polls per week proving that one candidate this, and another candidate that, trying to convince us that we should ignore common sense and rush with whatever herd they designate to wherever they want us to go. Why would I bring that up on a blog dedicated to rock art?
I recently read a very entertaining book on statistical correlation written by Tyler ViGen (Spurious Correlations, Hachette Books, New York, 2015). He printed a whole lot of impressive statistical correlations that cannot possibly be related, and yet, represent the kind of fringe analysis we see frequently being trumpeted as proof of this or that in rock art. A few of my favorite results from his studies are:
Federal funding for science, space, and technology vs. Suicides by hanging, strangulation, and suffocation. - displayed a 99.2% correlation (ViGen 2015:48-9).
And even better was, amount of Alcohol sold in grocery stores vs. Total number of bridges in the United States. - displayed a 99.3% correlation. "Why else would you build a bridge, except so that you could go buy alcohol?" (ViGen 2015:98-99).
I have long been a skeptic of the misuse of statistics in rock art analysis. "On July 12, 2010, I published a posting titled The Writing On The (Cave) Wall concerning a recent example of this sort of study which was related in a February 17, 2010, article by Kate Ravilious in New Scientist magazine entitled “The Writing On The Cave Wall” which made the ambitious claim that writing had been discovered on the walls of the painted European caves. According to this report a pair of scholars at the University of Victoria, British Columbia, Canada, graduate student Genevieve von Petzinger and her supervisor April Nowell undertook as a Master’s project a numerical analysis of all the signs found in 146 sites in France covering a date range from 37,000 to 12,000 B.P. The signs were compiled in a database for analysis. They found that 26 of these signs appeared frequently in numerous sites. The most common sign was a line that was found at 70 percent of the sites and across all time periods. The next most common symbols were what they called “open angles” and dots being found at 42 percent of the sites. Having found quantifiable examples of common occurrence of these symbols in conjunction with each other they made the amazing announcement that they represent a form of written communication, and astoundingly the scientific community, including anthropology and archaeology, seems to accept this statement. Why would scientists fall for this? Because, Petzinger and Nowell got their results from a statistical analysis of numbers in a database."
I personally know a very qualified and serious rock art researcher who got himself caught up in this kind of problem in his thesis project. I will not identify him because I do not wish him to read this and take it as a personal attack. He was a student at the time and it was his role to be highly ambitious and optimistic. In my opinion his thesis was motivated by very positive drives, it was the job of his advisers to exercise a little guidance and I believe he was led astray. His thesis examined rock art sites in West-Central Colorado to attempt to identify locations of ritual behavior by visual and acoustic indicators. These were then analyzed with seven statistical tools; Principal Components Factor Analysis, Cluster Analysis, Tukey’s tests, Jacquard Matrix, Chi Square Analysis, and Bayesian approach.
"This research project has identified at least three and in some cases four (and possibly more) attributes associated with the rock art of the study area that are suggestive of each of the seven characteristics of Ritual Behavior as identified by Rappaport. Some of the attributes can be utilized to demonstrate more than one characteristic of ritual and as such are key indicators of multiple ritual characteristics. All of the seven characteristics of ritual were associated with at least three attributes (out of a total of 145 dichotomized variables [for 22 panel locations totaling 3,190 values] that were ultimately tested in the study)." (citation withheld for anonymity).
“Too much data was collected and analyzed. In an effort to be comprehensive and complete this research project identified 145 data attributes for 22 panel locations creating a dichotomized dataset consisting of 3, 190 values was then established (only two values were missing in two panel locations due to access issues [motif depth measurements]). In retrospect many of these values were peripherally related to the research question and probably should not have been recorded because they took time and attention away from the main project.” ((citation withheld for anonymity) - the above underlining is mine).
So back to my title - CORRELATION IS NOT CONFIRMATION. Do we really believe that cutting federal funding for Science, Space, and Technology will cut the number of suicides by strangulation? Do we really believe that there is a real relationship between the amount of money spent on alcohol in grocery stores in the United States and the number of bridges built? People, let's use our common sense! End of rant.
July 12, 2010, The Writing On The (Cave) Wall, http:rockartblog.blogspot.com.
2015 Spurious Correlations, Hachette Books, New York.