Fisking the Paul "Study"
by MikeGeneIf you'll remember, over a year ago, we took a look at the Gregory Paul "study" that purported to show a negative correlation between societal health and religiosity. It appears that this study has become the darling of the New Atheist movement and even someone like Michael Shermer recently abandoned critical thinking in order to peddle these "results" to his readers at Scientific American.
What Shermer failed to mention is that in the next issue of the Journal of Religion and Society, social scientists Gerson Moreno-Riaño, Mark Caleb Smith, and Thomas Mach responded to Paul's study.
Here are a few of their observations:
It is the opinion of the authors that once all of the methodological issues are considered, Paul's findings and conclusions are rendered ineffectual.
When one adds this plethora of hypotheses along with Paul's conceptual superfluity, it is impossible to know what in fact is being measured, tested or falsified. This lack of measuring clarity and falsification not only causes Paul to violate the scientific principles of corrigibility and incrementalism but allows Paul to engage in theoretical though meaningless comparisons of nations based on simple scatter plots and outlying data.
Even if one sets aside the above methodological difficulties that frame Paul's study, his execution of the analysis leaves much to be desired. As noted earlier, Paul states his findings in strong language, but throughout his analysis, he handles his data with uncharacteristic modesty. In truth, there his analysis has much to be modest about.
Paul does not provide the reader with a meaningful opportunity to evaluate his findings, for he provides no correlation coefficients. He also fails to determine or report the significance of these correlations, so the reader is left to trust Paul's judgment that a negative correlation between theism and indicators of societal health has been established. Statistics exist so that we are not required to trust the subjective judgment alone, particularly in regard to matters as weighty as theism, democracy, and the social good.
How do I read all this? Paul's study is junk science.
But the best is saved for last:
In order to make incremental progress in demonstrating or falsifying any relationship between religiosity and secularism and societal health, one would have at least to address the following issues:
1. Conceptual clarity and consistency of use regarding the terms "religiosity" and "secularism."
2. Clear, distinct, and falsifiable hypotheses.
3. Explicit and adequate implementation of comparative analysis suggestions as found in Jowell and Madden.
4. Clear presentation of statistical tests.
5. Adequate discussion of the role of individual and formal political structures as these relate to the concepts in question.
6. Adequate assessment of the relationship between individual attitudes and behaviors and the relationship of these to macro-societal characteristics.
How do I read this? If Paul wants to make his case, he should consider using something called"¦"¦.the scientific method.
January 6th, 2007 at 6:32 pm
Well, quite obviously, Gerson Moreno-Riaño, Mark Caleb Smith, and Thomas Mach are a bunch religious hucksters trying to undermine sound science.
Comment by macht — January 6, 2007 @ 6:32 pm
January 6th, 2007 at 7:39 pm
The question for me is will Shermer post this update in Scientific American. As a "follower of the scientific method" he surely should of course maybe he is unaware of the challenge.
What I find interesting is that so many who claim to be "followers of science" or "followers of the evidence" or "rationalists" are just as must driven and biased by underlieing emotional or personal views. I know for example that Shermer used to be a "believer", desperately trying to convert people. (at least this is the view I have from listening to his stories)
Everytime I heard his story I could only think of how flaky the foundations of his original belief's must have been and without any foundation he watched them crumble with some simple questions. I could easily see this leading to someone having a leftover emotional bias against his original beliefs.
What's also interesting is that Shermer claims to be agnostic, yet spends his time arguing very much along the lines of someone who wasn't. I'm always interested by agnostics who spend their time arguing against theistic viewpoints of reality. Makes me wonder.
If I saw him post a study showing religous belief was good for you — I'd be bloody suprised.
Comment by Plump-DJ — January 6, 2007 @ 7:39 pm
January 6th, 2007 at 8:11 pm
Sounds a lot like Mel Konner's criticisms of Dawkins at Beyond Belief 2006 (episode 9 if I recall correctly). I would not expect to see any "retraction" or follow-up in SA.
Comment by undecided — January 6, 2007 @ 8:11 pm
January 6th, 2007 at 9:17 pm
dj,
I think this is one of the most important points. Those who posture as if they are the Ambassadors of Science and Reason seem awfully quick to abandon science and reason when it serves their own agenda.
Comment by MikeGene — January 6, 2007 @ 9:17 pm
January 6th, 2007 at 9:20 pm
Plump-DJ wrote:
There's a lot of confusion and disagreement about what the labels atheist and agnostic mean. I used to consider an atheist to be someone who was sure God(s) didn't exist, and so I called myself an agnostic to indicate my uncertainty. Now I see "atheist" as simply indicating a lack of a belief in God: a-theism. Since I see no reason to think that God exists, I am an atheist in the latter sense, while acknowledging that we cannot prove God's non-existence.
Regarding his own beliefs, Shermer clears up the confusion thus:
Comment by keiths — January 6, 2007 @ 9:20 pm
January 7th, 2007 at 6:04 pm
There was an interesting discussion on my blog on what the functional difference between atheism and agnosticism is anyway. I'm not convinced that it's as significant as many agnostics would like to make out.
The problem with this sort of study is that it becomes part of "the picture". The fact that it has already been refuted is ignored – it is just presented over and over again. In fact, "it" isn't – instead we just get the vague accusation that "scientists have shown that countries who don't believe in evolution fall behind in science." Again, on my blog, this has appeared at least twice in comments in the last three weeks, I think.
Comment by Exile From Groggs — January 7, 2007 @ 6:04 pm
January 8th, 2007 at 11:20 pm
When someone is an atheist or agnostic they usually mean the disbelieve in TRIBAL gods of barbarian holy texts like christianity, islam and the like.
A real god would look nothing like the monstrosity's you see in our ancestors books.
Comment by Frozen1 — January 8, 2007 @ 11:20 pm
January 9th, 2007 at 10:56 am
Frozen1, I don't mean to follow you in a few comments, but is there any chance you could bracket some of that anti-religious zeal? Seems not to be contributing to the type of atmosphere desired here at TT.
Comment by thechristiancynic — January 9, 2007 @ 10:56 am
January 9th, 2007 at 3:20 pm
Exile,
FWIW, when Darwin was told by atheistic dinner guests that his purportedly agnostic views were practically the same as theirs, but just less aggressively stated, his reply was (to the effect) "Why should you be so aggressive?".
Comment by Pez — January 9, 2007 @ 3:20 pm
January 13th, 2007 at 8:02 pm
[...] h/t: Telic Thoughts and Thinking Christian. [...]
Pingback by Magic Statistics - “I accept no responsibility for statistics, which are a form of magic beyond my comprehension.” — Robertson Davies » Gregory Paul shot down again — January 13, 2007 @ 8:02 pm