Activism and the problem with blog polls
by KrauzeA well-known problem with on-line polls is when special interest groups attempt to sway the outcome. For example, a national newspaper might put a poll on their website asking if corporate taxes are too high. An organisation interested in lowering corporate taxes might then distribute the link among grass-root supporters, encouraging them to vote "Yes". If they're successful, they will manage to sway the outcome of the poll, creating the impression that "the public thinks that corporate taxes are too high". A similar yet rarely acknowledged problem is when the respondents in a letter poll correspond with each other, coordinating their replies.
For a while, I've been disappointed with the lack of sociological data concerning how those who see themselves as "defending science" perceive intelligent design. There is plenty of evidence that this group suffers from stereotypes (such as Douglas Altshuler claiming that, "People in the ID community have said that we don't even know how bees fly."), but no systematic surveys. So I decided to do my own little informal survey, e-mailing various ID critics and asking them about intelligent design.
The biggest blog devoted to criticism of intelligent design is The Panda's Thumb, so for my group of respondents, I e-mailed its contributors as well as those which PT identifies as Science and Evolution Blogs*. A total of 36 persons received this mail:
Dear recipient,
You have been contacted because you contribute to a blog which has been identified as a "pro-science blog". I am conducting a survey on outsiders' perception of intelligent design, and I would appreciate your input. The results will be published on Telic Thoughts, an independent blog about intelligent design, and every reply will be treated as anonymous. Please read the following carefully, and send your answer to [my e-mail address.]
For the purpose of this survey, "creationism" will be defined as "a belief in the literal interpretation of the account of the creation of the universe and of all living things related in the Bible" (source: Dictionary.com). "Evolution" will be defined as "the theory that all modern life forms are derived from one or a few common ancestors via descent with modification".
Please answer the following:
On which points are intelligent design and creationism identical?
A. Both creationism and intelligent design require one to have a particular interpretation of the Biblical creation account.
B. Both creationism and intelligent design require one to accept a particular age of the Earth and of the universe.
C. Both creationism and intelligent design require one to reject evolution.
D. Both creationism and intelligent design identify the Christian God as the creator.
E. Both creationism and intelligent design hold that there is an intelligence behind certain features of nature.
F. There are no points of similarity between creationism and intelligent design.
G. None of the above options accurately describe the relationship between creationism and intelligent design.
(Please check all that apply.)
Thanks in advance,
Krauze
http://telicthoughts.com/
Unfortunately, some ID critics didn't like the scientific method to be applied to themselves were afraid of providing what could be seen as ammunition to the enemy. Within 27 minutes, one of the respondents, Wesley Elsberry of The Austringer, had posted the contents of my letter, advising others to reply by choosing "G". And within hours, other blogs had followed, including the highly popular Pharyngula. As another respondent, Tara Smith, said, "If you received [a mail], check out their comments before sending your answer back." Predictably, all of the respondents who replied either chose "G" or refused to participate in the survey (as it was of course their right to do, the survey being voluntary).
The reaction of these "defenders of science" is itself an interesting piece of sociological data, and I might deal with them in a later post. For now, I will leave you with a question: If the poll was indeed "wretchedly incomplete", as Elsberry claims, why did he see it necessary to notify his fellow bloggers immediately after receiving the poll? Was he worried that some might feel that one or more of my options adequately described their perception of intelligent design?
*)I excluded the contributors to RealClimate and Philosophy of Biology, which would otherwise have given me an additional 53 persons to poll. RealClimate is a blog dealing with environmental issues, and has no my knowledge never written about intelligent design. And most of the contributors to Philosophy of Biology seemed to be "silent members", having posted little or not at all. Investigating the beliefs of ID critics by polling a group of people of whom more than half have never spoken about intelligent design is bad practice, so I left that part of the group out.
January 30th, 2006 at 10:33 am
IDiots can't do ANY kind of science right
Yesterday, PZ Meyers , and a slough of other "pro-science" bloggers received this "survey" from an IDiot at Telic Thoughts:
For the purpose of this survey, "creationism" will be defined as "a belief in the literal interpretation of the account o…
Trackback by culturekitchen — January 30, 2006 @ 10:33 am
January 30th, 2006 at 1:03 pm
The difference between Biblical Creationism and ID:
Biblical Creationism is an argument from ignorance, with some deity making humans from clay or whatever as the conclusion made from that ignorance.
ID is an argument from lack of imagination, with some heavy wink-wink-nudge-nudge as to the conclusion.
Of course, that's just from personal experience with IDers. Both are a case of affirmative conclusions (design) from negative premises (I don't know/I can't imagine evolution/cosmology/abiogenesis explaining this), so in terms of logic and science, both are dead on arrival.
ID isn't even a hypothesis, since you can't falsify it. Whenever a flaw in the apparent design is pointed out, the believers can claim that we can't understand the designer, effectively doing a Pee Wee Herman "I meant to do that!"
I have yet to see a positive way to determine design in organisms. Archaeologists have it easy: They already know something about the designers of stone arrowheads. SETI also has something that makes it easy for them: They know a purpose: They're assuming the aliens want to be heard through the mass of static, so they need only think about what they would do in the aliens' place.
As far as I've been able to tell, we know nothing about this designer person(s). We don't know what, if any, purpose it would have for making life. Until we can answer those questions, I don't see how we can tell the difference between something designed, and something merely complicated.
Additionally, ID seems to rely on a false dilemma fallacy: Even if evolution found a problem that couldn't be solved for centuries, or even if it was outright falsified, it would do nothing for ID: We'd just go back to "We don't know."
Of course, if someone can come up with a form of ID unlike all the evasive, DOA ones I've seen, I'm all ears. I won't be holding my breath, though.
Comment by Bronze Dog — January 30, 2006 @ 1:03 pm
January 30th, 2006 at 2:19 pm
Hi Bronze Dog,
Why don't you take a crack answering the question in my post?
If the poll was indeed "wretchedly incomplete", as Elsberry claims, why did he see it necessary to notify his fellow bloggers immediately after receiving the poll?
Comment by Krauze — January 30, 2006 @ 2:19 pm
January 30th, 2006 at 2:47 pm
I don't know about Elsberry. Personally, I find the poll to be inadequate because it doesn't offer enough responses. This poll would be better done as an essay response.
Superficially, ID is trying to pass itself off as "E" being the only similarity. However, with everything exposed from the Dover trial, it's clear that (most of) ID is a political attempt at stealth Biblical Creationism. Also, because of the inherent argument from lack of imagination, it also implies "C", the rejection of evolution: Simply because people like Behe can't see how evolution can explain a flagellum, he rejects all possible evolutionary explanations in favor of ID, based on his faith that no evolutionary answer is even possible.
If I had to guess, it's that two-faced nature of the ID movement that science bloggers are choosing "G".
Comment by Bronze Dog — January 30, 2006 @ 2:47 pm
January 30th, 2006 at 3:12 pm
ID "Survey"
Added Later: krauze is claiming some sort of conspiracy about the poll he sent out. So let me clear it up for him, we did not receive marching orders from Elsberry. The response was the same because we all recognize…
Trackback by Afarensis — January 30, 2006 @ 3:12 pm
January 30th, 2006 at 3:19 pm
Hi Bronze Dog,
"Personally, I find the poll to be inadequate because it doesn't offer enough responses. This poll would be better done as an essay response."
Sure, the poll could be improved. As I said, it was just an informal poll. However, imagine this scenario:
Elsberry and the other bloggers keep quiet and reply without instructing each other on what to do. If the poll came back with a majority of G's, it would have been clear that I had failed to capture an important part of the critics' perception. At this point, the critics could have performed their own poll, with the options they saw fit. This would have been the scientific way to settle who was right.
But now, this scenario is impossible, because the critics have been busy telling everyone in the blogosphere what they should choose. There is no longer any way of telling the difference between "The poll failed because Krauze sucks at formulating polls" and "The poll failed because the critics coordinated their responses". This is a research opportunity that the critics destroyed.
Comment by Krauze — January 30, 2006 @ 3:19 pm
January 30th, 2006 at 3:32 pm
How would your conspiracy theory be indistinguishable from the other possibility that people disagree about the flaws, and some of those bloggers felt like explaining why they think it's flawed?
That's the problem with conspiratorial thinking: It's unfalsifiable, and easy to use to demonize your opponents. I think you're mistaking, consciously or unconsciously, recommendations for orders. They may have simply told people to vote "G" because they thought it was the correct response for hypothetical reasons 1, 2, and 3.
Additionally, I fail to see how someone could put down all the appropriate options. It's my understanding that scientific polls are very hard to make. A simple rephrase of a question can lead people.
Comment by Bronze Dog — January 30, 2006 @ 3:32 pm
January 30th, 2006 at 3:47 pm
Hi Bronze Dog,
"How would your conspiracy theory be indistinguishable from the other possibility that people disagree about the flaws, and some of those bloggers felt like explaining why they think it's flawed?"
This explanation would make sense if the critics were criticizing a result that had already been published. But Elsberry was criticizing a poll that had yet to be concluded, less than half an hour after having received the e-mail. Why the rush?
Besides, it's not a "conspiracy theory" to suggest that some of the critics were attempting to coordinate responses. In the comments, Eslberry says, "Hopefully, the issues are now clear to other webmasters and bloggers." Tara Smith adviced people to "check out their comments before sending your answer back." "Afarensis" said he cut-and-pasted a response suggested by another blogger.
"They may have simply told people to vote "G" because they thought it was the correct response for hypothetical reasons 1, 2, and 3."
But the poll wasn't a test to see how well various critics knew their "reasons". It was a poll to figure out what people believed. Why would Elsberry have to remind others of what they believe?
"Additionally, I fail to see how someone could put down all the appropriate options. It's my understanding that scientific polls are very hard to make. A simple rephrase of a question can lead people."
Indeed, responses are easily swayed by something as fickle as the phrasing of a question. So why is it a "conspiracy theory" to think that the responses were influenced by high-profile bloggers telling people how to respond?
Comment by Krauze — January 30, 2006 @ 3:47 pm
January 30th, 2006 at 4:04 pm
Hi Krauze,
As a creationist and IDist, I would answer:
The most complete way of the relationship is that Special Creation is a sufficient condition for Intelligent Design. However, Intelligent Design is a necessary but not sufficient condition for Special Creation.
For example, if we demonstrate scientifically the earth is Young, that wpi;d generally be sufficient evidence for ID (unless one is a die-hard anti-IDist).
If we confirm CSI is evident in biology, that would indicate design, but it would be insufficient in and of itself to prove Special Creation.
I applaud your civility towards the critics. My patience with them has run out.
You're a better man than I….
Sal
Comment by Salvador T. Cordova — January 30, 2006 @ 4:04 pm
January 30th, 2006 at 4:16 pm
Hi Krauze,
I am sorry to say that as a Bible believing "creationist" who is very supportive of ID I wouldn't have been able to answer your poll very well either.
B, C, and D require making certain assumptions about creationism before assessing whether they can be linked to assumptions about ID.
However, I certainly wouldn't need someone to tell me what my appropriate responses should be, and I wouldn't go ahead and submit that person's suggestions as my response.
Comment by Pez — January 30, 2006 @ 4:16 pm
January 30th, 2006 at 4:26 pm
Hi Pez,
"I am sorry to say that as a Bible believing "creationist" who is very supportive of ID I wouldn't have been able to answer your poll very well either."
Nothing wrong with that. You would just have replied "G" or refused to reply at all. That's the respondent's way of telling the polster that he screwed up formulating the poll.
Comment by Krauze — January 30, 2006 @ 4:26 pm
January 30th, 2006 at 4:37 pm
This explanation would make sense if the critics were criticizing a result that had already been published. But Elsberry was criticizing a poll that had yet to be concluded, less than half an hour after having received the e-mail. Why the rush?
If the poll is flawed, the poll is flawed. Period.
Besides, it's not a "conspiracy theory" to suggest that some of the critics were attempting to coordinate responses.
You mean like how tries to coordinate votes by giving people reasons to vote one way over another?
"Afarensis" said he cut-and-pasted a response suggested by another blogger.
So what? He might have thought that other blogger said it better than he could.
But the poll wasn't a test to see how well various critics knew their "reasons". It was a poll to figure out what people believed. Why would Elsberry have to remind others of what they believe?
It wasn't about what the bloggers believed: It was about how to express what they believed. Pay attention.
Indeed, responses are easily swayed by something as fickle as the phrasing of a question. So why is it a "conspiracy theory" to think that the responses were influenced by high-profile bloggers telling people how to respond?
1. It's more about the tone. You make discussion and agreement sound as if it was somehow akin to cheating.
2. Would you give out a test that featured a misleading question and then bite the head off of anyone who pointed out the misleading part? That's what you seem to be doing.
Comment by Bronze Dog — January 30, 2006 @ 4:37 pm
January 30th, 2006 at 4:39 pm
Brackets error: "You mean like how [insert political party] tries to coordinate votes…"
Comment by Bronze Dog — January 30, 2006 @ 4:39 pm
January 30th, 2006 at 4:48 pm
Hi Krauze,
I have to apologize.
Before commenting on blogs I like to go away and weigh my thoughts for a while to make sure I am not being too reactionary.
In so doing, I forgot that you had defined "creationism" and "evolution" at the beginning of your poll.
This pretty much eliminates the small flaw about requiring the respondents' double assumptions that I thought I had seen in the poll.
ps,
Of course, I would have checked letter E.
Comment by Pez — January 30, 2006 @ 4:48 pm
January 30th, 2006 at 5:24 pm
"The reaction of these "defenders of science" is itself an interesting piece of sociological data, and I might deal with them in a later post." Are you sure it wasn't that other interesting bit of sociological data that you were really after in the first place?
It would be extraordinarily difficult to get an untainted sample of real opinions. For one thing, many critics consciously play a double game, defining "creationism" as "belief in a designing intelligence" so as to include ID and then equating ID to "creationism" per the definition you quoted in order in order to exploit liberal prejudices towards the latter connotation. Of course these critics are not going to blow their cover by answering honestly.
For another, given the nature of the anti-ID campaign, endlessly repeating the charge that ID is a Trojan Horse ruse (pun intended!), critics are going to be swayed by their own propaganda into believing there must be an ulterior motive behind the question.
Comment by mb — January 30, 2006 @ 5:24 pm
January 30th, 2006 at 5:29 pm
Considering the old edition of "Of Pandas and People" featuring "Creationism" instead of "Intelligent Design" and the sorts of people supporting it, I think the Trojan Horse was blown wide open in Dover.
Besides, Trojan Horse or not, even if you do separate it from its Biblical counterpart, ID is still not science, and thus doesn't belong in the classroom… Except, maybe in a logic/philosophy class for discussion as an example of argument from ignorance/lack of imagination.
Comment by Bronze Dog — January 30, 2006 @ 5:29 pm
January 30th, 2006 at 5:41 pm
Hi Bronze Dog,
"If the poll is flawed, the poll is flawed. Period."
Considering that I don't make polls for a living, I can live with that. But you forgot my question: Why the rush in criticizing a poll that hadn't been published yet?
Krauze: "Besides, it's not a "conspiracy theory" to suggest that some of the critics were attempting to coordinate responses."
Bronze Dog: "You mean like how [insert political party] tries to coordinate votes by giving people reasons to vote one way over another?"
There are three distinct concepts to keep track of here:
- A vote, in which the people communicate their will.
- A test, in which a person's ability or knowledge is estimated.
- A poll, in which the opinion or beliefs of a group of people are estimated.
So no, attempts to sway the outcome of an election isn't the same as attempts to sway the outcome of a poll.
Krauze: ""Afarensis" said he cut-and-pasted a response suggested by another blogger."
Bronze Dog: "So what? He might have thought that other blogger said it better than he could."
In that case, we don't get the opinion of "Afarensis", but the opinion of that other blogger.
Besides, you missed the statements from Elsberry and Smith.
But the poll wasn't a test to see how well various critics knew their "reasons". It was a poll to figure out what people believed. Why would Elsberry have to remind others of what they believe?
"It wasn't about what the bloggers believed: It was about how to express what they believed. Pay attention."
The poll didn't ask respondents to write an essay on their views. The poll presented a number of options, one of which was "None of the above options accurately describe the relationship between creationism and intelligent design." Choosing the option that each correspondent felt appropriate didn't require any special skills in "expressing what they believed".
"1. It's more about the tone. You make discussion and agreement sound as if it was somehow akin to cheating.
2. Would you give out a test that featured a misleading question and then bite the head off of anyone who pointed out the misleading part? That's what you seem to be doing."
It was a poll, not a test. Pay attention.
Comment by Krauze — January 30, 2006 @ 5:41 pm
January 30th, 2006 at 5:57 pm
Why the rush in criticizing a poll that hadn't been published yet?
Because it had visible flaws. Why wait until it's published?
So no, attempts to sway the outcome of an election isn't the same as attempts to sway the outcome of a poll.
I fail to see the difference. Well, there is a difference in consequences of the results, but that's irrelevant, since we're talking about your flawed methods of obtaining those results.
In that case, we don't get the opinion of "Afarensis", but the opinion of that other blogger.
Nonsense. What you got was Afarensis using someone else's eloquence for an opinion he shared.
The poll didn't ask respondents to write an essay on their views.
Which is already one big problem, as I pointed out.
Choosing the option that each correspondent felt appropriate didn't require any special skills in "expressing what they believed".
Think about that next time you answer an overly wordy multiple choice or true/false question. The principle is the same: Sometimes people don't see a deception or an error put in the question or the responses they're allowed.
It was a poll, not a test. Pay attention.
Irrelevant to my point. And since I think you're now dodging said point via the "trivial objections" fallacy, I think I'll sign off.
Comment by Bronze Dog — January 30, 2006 @ 5:57 pm
January 30th, 2006 at 6:19 pm
"Afarensis" said he cut-and-pasted a response suggested by another blogger.
Actually, krauze, if some suggested to me how to respond to a question I'd have to tell them to stick it. I used John's answer because I liked it better than the one I was going to use. Which was just to point out the questions had flawed choices for answers and snarkily suggest you not get a job doing opinion surveys. Instead, I used John's remark because it made the same point a little bit more diplomacy. If you were seriously interested in learning what people believed you would have opted for a poll with more than one question and more open ended answers instead of trying to force the answer you want. Perhaps before you do another poll you could take a class and actually figure out how to create a poll that gets you the data you are looking for… As far as the "rush" to criticize a poll that hadn't been published goes, why would anybody participate in a poll that is obviously methodologically flawed? You complain because now we can't tell the difference btween the poll sucked because of critic coordination or bad poll construction. The fact that the bloggers who responded on thier blogs were unanimous in saying the poll sucked should settle that question.
Comment by afarensis — January 30, 2006 @ 6:19 pm
January 30th, 2006 at 7:04 pm
Hi Bronze Dog,
"Why wait until it's published?"
Because that's when the majority of people will actually become aware of it. In addition, if Elsberry was simply concerned about correcting a badly worded poll, why did he phrase it in terms of making it "clear to other webmasters and bloggers", considering that these were the very people who had been polled? Why not making it clear to "readers of this blog"
"I fail to see the difference. Well, there is a difference in consequences of the results, but that's irrelevant, since we're talking about your flawed methods of obtaining those results."
No, actually, we aren't. We're talking about your labelling as a "conspiracy theory" my suggestion that Elsberry and other bloggers attempted to influence the responses to the poll.
"What you got was Afarensis using someone else's eloquence for an opinion he shared."
But respondents weren't asked to be "eloquent". If "Afarensis" thought that "G" best represented his opinion, he didn't need someone else's "eloquence" to express this.
Krauze: "The poll didn't ask respondents to write an essay on their views."
Bronze Dog: "Which is already one big problem, as I pointed out."
Plenty of polls don't involve respondents writing essays. Besides, the quality of the poll isn't relevant to the question of the activism on the part of ID critics.
"Sometimes people don't see a deception or an error put in the question or the responses they're allowed."
What happened to the argument about a lot of bloggers independently reaching the conclusion that the poll was flawed?
"Irrelevant to my point. And since I think you're now dodging said point via the "trivial objections" fallacy, I think I'll sign off."
You didn't get the bannination you were hoping for, huh?
Comment by Krauze — January 30, 2006 @ 7:04 pm
January 30th, 2006 at 7:12 pm
Hi Afarensis,
I'll have more to say about this tomorrow, but it's night on my side of the planet, and I'm turning in now.
Comment by Krauze — January 30, 2006 @ 7:12 pm
January 30th, 2006 at 8:11 pm
"If "Afarensis" thought that "G" best represented his opinion, he didn't need someone else's "eloquence" to express this."
Umm, actually, I felt "H. 'Intelligent design' is a subset of the arguments previously labeled 'creation science'." to be the best answer… I would also point out that if you go read some of those blog posts (or reread as the case may be) the quality of the poll was the central issue rather than any percieved attempts at activism.
Comment by afarensis — January 30, 2006 @ 8:11 pm
January 30th, 2006 at 11:35 pm
Krauze, I'm surprised that you could write such an obviously flawed survey, and then when the people you emailed tell everyone about its flaws, you say: "Unfortunately, some ID critics didn't like the scientific method to be applied to themselves." which is the wrong lesson to learn from this effort. But I didn't come to bash you, I'm commenting here because I am honestly curious about one aspect of the survey.
You defined creationism as a belief in the literal interpretation of the account of the creation of the universe and of all living things related in the Bible. (Lets ignore the fact that the two accounts of creation in the bible are mutually contradictory) This seems to be describing a subset of creationism often referred to as young-earth creationism or biblical literalism. However, there are several other forms of creationism that make references to the christian bible, but do not fit the definition given by you. For example, the many forms of old-earth creationism are not completely literal in their interpretations. Gap creationism attempts to be literal in some respects, as well as day-age creationism. But because they are not wholly literal they don't seem to fit under the defintion you gave in the survey. Link to explanations.
Finally, there is old-earth progressive creationism, which interprets the bible rather loosely. It suggests that the deity created organisms over time with increasing complexity, following the pattern of evolution, but requiring the deity to step in and make the new organisms.
What I'm curious about is, where do these other forms of creationism fit into the boundaries that you have drawn in your survey?
If Intelligent Design tries to distance itself only from the literal interpretation version(s) of creationism, then this survey would make sense. However, Intelligent Design advocates are trying to distance themselves from all forms of biblical creationism, including progressive creationism, so this survey is worthless. If you were to say Biblical literal creationism in the survey, and then turn around and apply that result to all forms of creationism (or not explain that your survey excludes those other forms) then you would be equivocating and arriving at a fallacious answer.
Also, please add me to your list of "pro-science" blogs. http://www.inoculatedmind.com/
And if you decide to re-word the survey to a form that doesn't persuasively define the terms, and is applicable to reality, feel free to include me in it. (Find my email in my user profile)
Karl
Comment by Inoculated Mind — January 30, 2006 @ 11:35 pm
January 30th, 2006 at 11:46 pm
S. Cordova said:
That's pretty close to the Christian position. We believe that God is behidn all features of nature, though not necessarily the proximate cause.
Why not a discussion about the differences between creationism/IDism on one side, and Christianity on the other?
Comment by edarrell — January 30, 2006 @ 11:46 pm
January 30th, 2006 at 11:51 pm
It's a form of a push-poll, really, and push-polls are inherently corrupted. As with Mutually Assured Destruction, it's a silly game: The only way to win (get close to accuracy) is not to play.
Comment by edarrell — January 30, 2006 @ 11:51 pm
January 31st, 2006 at 12:14 am
This is what I emailed Krauze, and I did so before I got any other emails about the poll:
As with virtually all polls of this nature, the list of options is too simplistic and leaves the false impression that the questions as stated adequately cover the subject. Also, I note that you define creationism and evolution, but not intelligent design "“ I have found that the same people give different explanations of what ID is depending on the circumstances, and different people who are ID supporters explain it differently also. Furthermore, your list of options does not include the statements that I think would most accurately describe the ways that ID is a from of creationism.
By the way, this is a not an anonymous reply for your survey, but rather a personal and public reply from me.
Thanks,
Jack Krebs
President, Kansas Citizens for Science
http://www.kcfs.org
member of the Panda's Thumb and the KCFS Discussion forums
Comment by jkrebs — January 31, 2006 @ 12:14 am
January 31st, 2006 at 12:34 am
Jack Krebs:
This strikes me as an utterly ridiculous criticism. If the list of options was "too simplistic" or "leaves false impressions," check G. Problem solved. If there was no "˜G', the complaint would have merit.
Second, as for not defining ID, well, Krauze tells us his objective "“
"I am conducting a survey on outsiders' perception of intelligent design." If he defines ID for the subject, he shapes their perception of ID – the very thing he is trying to get a handle on. Krauze sent this survey to people who have been complaining about ID for years, constantly equating it with creationism. Don't they know what they have been complaining about all this time?
Comment by MikeGene — January 31, 2006 @ 12:34 am
January 31st, 2006 at 4:12 am
In answer to your original question:
I think the most likely reason is that Elsberry has an – ah – less than lively respect for the honesty of Intelligent Design Creationists, and wanted to warn against participating in a survey that was obviously going to be misrepresented by the institution that did the survey.
Y'see, people on the pro-science side usually treat surveys as being generally honest, even when they come from creationists, and even when they are rather amateurish. And there being at least two or three different ways to interpret both the question and each answer (another warning sign), the chance of somebody falling for your little game was more than just marginal.
But why do you ask us? Why don't you ask Elsberry himself?
An aside: In my world, 36 is not a very impressive sample size. Even assuming that everyone replies, just one recipient not taking the poll seriously would skew the result by more than 2.5 %. And that number goes up geometrically with falling response frequency.
Another aside: Kudos for allowing critisism to stay up. An unusual gesture in the creationist blogosphere (none named coughdembskicough, none forgotten).
- JS
Comment by JS — January 31, 2006 @ 4:12 am
January 31st, 2006 at 8:45 am
Hi Jack,
"This is what I emailed Krauze, and I did so before I got any other emails about the poll:"
Did you get any other emails?
Comment by Krauze — January 31, 2006 @ 8:45 am
January 31st, 2006 at 9:40 am
Krauze,
I've got to say that although I suppose it was not your intention, the survey has revealed a great deal about the sociology of the self-named "pro-scientists". Not the least of which is their tribal behavior.
Now is that pro-science tribe networked or what? They've got well-defined strategies for action, it seems. Oh well, as we've seen from the Christian fundies in America, such networked tribal behavior often causes paranoia and paranoid-driven tribal action.
27 minutes!?!
Comment by bipod — January 31, 2006 @ 9:40 am
January 31st, 2006 at 9:56 am
Krauze,
All surveys are designed to extract some information. If, as you said, this was meant to be a "scientific" approach, then you had a hypothesis, right? The pro-scientists such as Elsberry have done their typically confident psychologizing, inferring your intentions with the poll. Some of Mr. "I'm a Real Falcon Scientist and You're Not" Elsberry's reasoning seems much more reasonable than other parts (for example, I doubt that you saw "F" as the most honest answer, because it would amount to absurd ignorance – though Elsberry sounds pretty sure of himself here;-)
I'm interested: what data had you hoped to extract from doing the survey?
Comment by bipod — January 31, 2006 @ 9:56 am
January 31st, 2006 at 12:32 pm
Hi Inoculated Mind,
"Krauze, I'm surprised that you could write such an obviously flawed survey, and then when the people you emailed tell everyone about its flaws, you say: "Unfortunately, some ID critics didn't like the scientific method to be applied to themselves." which is the wrong lesson to learn from this effort."
But Elsberry wasn't just criticizing a "flawed survey"; he was reminding "other webmasters and bloggers" what the correct answer was supposed to be, and he was doing it within half an hour of having recieved the e-mail. Why the rush?
"You defined creationism as a belief in the literal interpretation of the account of the creation of the universe and of all living things related in the Bible."
Actually, I used the definition that Dictionary.com had already supplied. Where do you think I should have gone to get an authorative definition of creationism?
"(Lets ignore the fact that the two accounts of creation in the bible are mutually contradictory)"
Yes, let's do that, as it's completely irrelevant.
"What I'm curious about is, where do these other forms of creationism fit into the boundaries that you have drawn in your survey?"
I'm perfectly willing to extend the label "creationism" to old-earth creationism. The question is what the average person thinks when he or she hears the word "creationism".
"If you were to say Biblical literal creationism in the survey, and then turn around and apply that result to all forms of creationism (or not explain that your survey excludes those other forms) then you would be equivocating and arriving at a fallacious answer."
Indeed, I would. It was for that reason that I intended to post the full letter, containing the definition of "creationism" (which I also did, even when it became apparent that the poll wasn't going to be published).
"Also, please add me to your list of "pro-science" blogs. http://www.inoculatedmind.com/"
As I explained in my post, my selection was based on the list of "Science & Evolution Blogs" on The Panda's Thumb. So if you want to be added to any lists, you need to contact the folks at PT.
Comment by Krauze — January 31, 2006 @ 12:32 pm
January 31st, 2006 at 12:48 pm
Krauze:
Actually, you could've left the definition of creationism out of your survey altogether, and it wouldn't have mattered. For instance, take one of your choices:
If the critic thinks that this isn't true of creationism, then they could simply not choose B, since obviously then ID and creationism couldn't possibly share this particular piece of content. The same goes for any of the other choices. This whole complaint about you "misdefining" creationism is a bit of a red herring.
Comment by Deuce — January 31, 2006 @ 12:48 pm
January 31st, 2006 at 1:02 pm
Hi Krauze,
Didn't you say your poll was informal? If it was scientific, could you please explain the methodology you are using?
KC
Comment by KC — January 31, 2006 @ 1:02 pm
January 31st, 2006 at 2:33 pm
Hi JS,
"I think the most likely reason is that Elsberry has an – ah – less than lively respect for the honesty of Intelligent Design Creationists, and wanted to warn against participating in a survey that was obviously going to be misrepresented by the institution that did the survey."
Yes, I think that's a good assessment as well. So much for the many bloggers independently finding the poll flawed. Elsberry's prejudices against ID supporters are irrelevant; what matters is the consequences of his actions.
"And there being at least two or three different ways to interpret both the question and each answer (another warning sign), the chance of somebody falling for your little game was more than just marginal."
And which different interpretations are those? Besides, does questions that ben interpreted in multiple ways by themselves make a poll suspect? For example, consider this poll, highlighted by the NCSE. Shall we review the multiple ways in which those questions can be interpreted?
"But why do you ask us? Why don't you ask Elsberry himself?"
Actually, I did. In my post, I linked to Elsberry's blog, thereby leaving a trackback. As you can see from his comments, he didn't want to answer it.
"An aside: In my world, 36 is not a very impressive sample size."
I agree. I considered expanding the size of the sample by including the contributors to Philosophy of Biology. But for reasons I explained in my post, I had to exclude them. I intended to wait for the results to come back before considering to expand the sample, say by combing Technorati for blogs discussing intelligent design.
Comment by Krauze — January 31, 2006 @ 2:33 pm
January 31st, 2006 at 2:41 pm
Hi Bipod,
"I'm interested: what data had you hoped to extract from doing the survey?"
Instead of talking about what I had hoped for, I'd rather talk about what I expected. I actually expected a considerable number of critics choosing C: "Both creationism and intelligent design require one to reject evolution." Much of the rhetoric is that intelligent design is anti-evolution (just consider Art's comments), and I would have liked to get some actual numbers on this.
Comment by Krauze — January 31, 2006 @ 2:41 pm
January 31st, 2006 at 4:36 pm
Hi KC,
"Didn't you say your poll was informal? If it was scientific, could you please explain the methodology you are using?"
Considering that science is just a formalized way of doing what we normally do to make sense of the world, we actually use "the scientific method" in many circumstances that fail to qualify as science.
In this case, the scientific method is simply that of performing experiments, gathering data, and drawing conclusions from ones observations. Or, e-mail a bunch of people, ask them what they think, and then calculate some percentages and even do some statistics (sadly, I never got to those last points).
However, I have since come to regret the "they didn't like the scientific method to be applied to themselves". I think JS' suggestion is closer to the truth – Elsberry was worried about giving a "creationist" some data he might quote-mine. So I've changed that part of my post.
Comment by Krauze — January 31, 2006 @ 4:36 pm
January 31st, 2006 at 4:49 pm
Krauze, you seem to have inadvertently crossed out the second half of your post, and all the comments
Comment by Deuce — January 31, 2006 @ 4:49 pm
January 31st, 2006 at 4:50 pm
The cross out tag wasn't closed properly. Damn you krauze making me work while i'm not working at work.
Comment by Guts — January 31, 2006 @ 4:50 pm
January 31st, 2006 at 4:55 pm
Please delete the trackback from culturekitchen. We don't need that kind of garbage here at TT.
Comment by MatthewCromer — January 31, 2006 @ 4:55 pm
January 31st, 2006 at 6:48 pm
Hi Deuce,
That's weird. I checked it after I edited it, and it looked fine.
Oh, and Matthew: I regard trackbacks from places like culturekitchen as batches of honor.
Comment by Krauze — January 31, 2006 @ 6:48 pm
February 1st, 2006 at 2:19 pm
I think the most likely reason is that Elsberry has an – ah – less than lively respect for the honesty of Intelligent Design Creationists, and wanted to warn against participating in a survey that was obviously going to be misrepresented by the institution that did the survey.
Yes, I think that's a good assessment as well. So much for the many bloggers independently finding the poll flawed.
Fallacy. The two are not mutually exclusive. I, for instance, first saw the poll on The Mad Biologist, and concluded that it was a stunt before seeing Elsberry's post. Call me prejudiced that way, but I just don't trust polls made by parties with an – ah – appreciable stake, shall we say, in the subject at hand.
And which different interpretations are those?
I'll concede that my comment was less than perfectly clear. What I meant to say is that the entire poll is ambiguious, because Intelligent Design Creationism is a front for YECism contrieved after YECism got slapped down in Edwards vs. Aguillard.
So, you can respond in (at least) two wholly different ways: You can respond as if you accept the Disco Inst.'s version of the story, in which case the 'correct' answer is E, or you can reply as if you regard IDC as a ploy to circumvent a clear-cut SCOTUS decision, in which case the correct answer is G – none of the above – since the option 'IDC is a politcal stunt to advance creationism' isn't there.
This is a problem similiar to – say – the problem you are faced with if you are asked about the relationship between Hamas and Israel. On the one hand, Hamas holds an ideological position that Israel should be wiped from the map, but on the other hand, Hamas is a somewhat reasonable and pragmatic bunch, so in reality they accept that Israel is there to stay.
This is not a problem if the survey is carried out with the intent to gain information. Given a sufficiently large sample (!), such problems will emerge as clear patterns in the answers. But if the pollster is less than fully honest, that ambiguity can be manipulated.
If you answer E, because that's what the people pushing IDC says it is, then a dishonest pollster could take the result and say "see, not even the other side thinks that we're YECs," the same way a dishonest politician can (and does) distort the fact that Hamas formally has the destruction of Israel on its political agenda, despite the fact that they know full well that it's not gonna happen.
And if you answer G, then you're basically saying 'the poll is flawed,' and providing a pretext for a dishonest pollster to scrap the thing without further comment. So from the perspective of someone doubting your honesty and integrity, it looks like a 'heads, I win, tails, forget about it' situation. Needless to say, you don't want your friends to be used in that way. Hence the 'activism.'
An aside: Since the poll is badly and fundamentally flawed, what's all the ruckus about anyway? If your intentions were bona fide, then you wouldn't have been able to use the results anyway, Elsberry or no Elsberry.
Another aside: If you want to conduct a poll about IDC in the future, be careful to split every question into two parts: What the IDC advocates claim IDC is, and what IDC actually is. Whether or not the IDC advocates are actually lying, the distinct impression on our side of the fence is that they do…
For example, consider this poll, highlighted by the NCSE. Shall we review the multiple ways in which those questions can be interpreted?
Frankly, the only poll I'm interested in is the number of publications and citations in the peer-reviewed litterature over the past – say – five years.
But, no, it would be a pointless exercise to point out flaws and ambiguities in the NCSE poll, since the fundamental honesty and intellectual integrity of the NCSE is well established.
Now, if you can point out a case of quote-mining or misrepresentation of their results, I might reevaluate my assessment of the NCSE's honesty, in which case questioning their poll techniques might be relevant. But the fundamental difference is that their honesty is established, whereas the jury is still out on the honesty of Telic Thoughts.
Or, e-mail a bunch of people, ask them what they think, and then calculate some percentages and even do some statistics
Statistics being a pet pee-we of mine, I'll comment on this too. You need more than 36 people to do statistics. As in far more than 36 people. Medical experiments use test and control groups a couple of thousand big. And they skirt the very edges of what can be reasonably called statistical evidence.
The reason I'm picky about statistics is that there's so much bad statistics floating around in society, and that ignorant and/or less than honest politicians routinely conflate good and bad statistics – playing on the fact that 95 % of the population is effectively mathematically illiterate.
In short, it pisses me off (actually I'm more pissed off at the reporters who ought to do the math themselves rather than accept at face value what they're told, but that's another story entirely). So I vent steam whenever I see bad numbers…
However, I have since come to regret the "they didn't like the scientific method to be applied to themselves". I think JS' suggestion is closer to the truth – Elsberry was worried about giving a "creationist" some data he might quote-mine. So I've changed that part of my post.
A public retraction. Kudos. Your credibility is improving in my eyes (not hard, since it started out at rock bottom, but every little step counts). And no, I'm not being sarcastic. I take as much fun as the next guy in pointing and laughing when you screw up, so it's only fair that I commend when you do something right.
Comment by JS — February 1, 2006 @ 2:19 pm
February 1st, 2006 at 2:21 pm
Ah, there's a SNAFU in my post just previous: The first two paragraphs are from different posters, but that's not marked the way this page formats things.
- JS
Comment by JS — February 1, 2006 @ 2:21 pm
February 1st, 2006 at 2:46 pm
JS,
Perhaps you can explain what relationship Krause has with any creationist organization.
Comment by MatthewCromer — February 1, 2006 @ 2:46 pm
February 3rd, 2006 at 1:47 pm
Huh? Where do I claim that he has such an affiliation? I reread my posts carefully, and I don't see that claim made anywhere. AFAIK, Krauze has no such relationship (but I couldn't tell you for certain, since I haven't made a big point of studying his background).
- JS
Comment by JS — February 3, 2006 @ 1:47 pm
February 3rd, 2006 at 6:19 pm
Hi JS,
"Call me prejudiced that way, but I just don't trust polls made by parties with an – ah – appreciable stake, shall we say, in the subject at hand."
Interesting. So it was the source of the poll that made you conclude that it was "a stunt".
"So, you can respond in (at least) two wholly different ways: You can respond as if you accept the Disco Inst.'s version of the story, in which case the "˜correct' answer is E, or you can reply as if you regard IDC as a ploy to circumvent a clear-cut SCOTUS decision, in which case the correct answer is G – none of the above – since the option "˜IDC is a politcal stunt to advance creationism' isn't there."
Yes, the poll allows people with different beliefs to reply in different ways. And since the purpose of the poll was to discover people's beliefs, how is that a problem?
"And if you answer G, then you're basically saying "˜the poll is flawed,' and providing a pretext for a dishonest pollster to scrap the thing without further comment."
Yes, a sufficiently fradulous way for ID supporters to act can always be imagined. However, why would this scenario be such a catastrophe? The ID supporter forgets his silly little poll, and everything continues as if it never happened.
"So from the perspective of someone doubting your honesty and integrity, it looks like a "˜heads, I win, tails, forget about it' situation. Needless to say, you don't want your friends to be used in that way. Hence the "˜activism.'"
Except that Elsberry actually advised people to give the reply that allegedly would allow me to "forget about it". Obviously, something's missing from your analysis.
"An aside: Since the poll is badly and fundamentally flawed,"
That's your subjective impression.
"what's all the ruckus about anyway? If your intentions were bona fide, then you wouldn't have been able to use the results anyway, Elsberry or no Elsberry."
How do you know? We don't know how the poll would have fallen out without the activism of Elsberry and his friends.
Krauze: "For example, consider this poll, highlighted by the NCSE. Shall we review the multiple ways in which those questions can be interpreted?"
JS: "Frankly, the only poll I'm interested in is the number of publications and citations in the peer-reviewed litterature over the past – say – five years."
Now, that can't be the case, as you're spending your time discussing this poll, rather than making your "citations in the peer-reviewed litterature" argument somewhere else. So, why are you reluctant to apply your criteria to the poll from NCSE?
"But, no, it would be a pointless exercise to point out flaws and ambiguities in the NCSE poll, since the fundamental honesty and intellectual integrity of the NCSE is well established."
Then why make claims about alleged ambiguity in my poll, when all that really matters is the source?
"You need more than 36 people to do statistics. As in far more than 36 people. Medical experiments use test and control groups a couple of thousand big. And they skirt the very edges of what can be reasonably called statistical evidence."
Yeah, because there's no difference between an informal poll to be "published" on a blog and the testing of medicine that will be available through pharmacies, right?
"A public retraction. Kudos. Your credibility is improving in my eyes (not hard, since it started out at rock bottom, but every little step counts)."
Another interesting statement. So, knowing absolutely nothing about me, except that I'm an ID supprter, you start out by assuming the worst.
Comment by Krauze — February 3, 2006 @ 6:19 pm