[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[APD] Baloney Detection

I was reading a page about "baloney detection" 
(http://www.physics.smu.edu/%7Epseudo/baloney.html) and it struck me 
that some of the points have particular relevance in our hobby. With all 
the products out there (most of which are probably legit), it helps to 
keep a few of these points in mind:

1. How is the claim/discovery announced?

If a claim or discovery is announced by holding a press conference, 
rather than being published in a scientific journal, beware. Such a 
claim may be unverified. Also beware of "discovery" announcements that 
ask you to subscribe (pay) to some newsletter.

3. Is anecdotal evidence cited?

Anecdotes, or individual stories, are not scientifically useful. Such a 
story suggests only that the individual did so-and-so (assuming no 
fakery). Nothing about larger populations can be inferred from it. Any 
description of an invention or product which uses large numbers of 
anecdotes or testimonials is suspect. You also must remember that 
testimonials can be bought; you can pay someone to do it for you. Also - 
the Federal Trade Commission will prosecute false claims, but 
testimonials are not regulated.

7. Has anyone tried to disprove the claim?

It is very important to find out if other investigators have tried to 
replicate the work. Toward this end, real scientists will publish 
complete information about the work, enough for another competent 
investigator to attempt replication. Others will repeat the work and 
publish the results.

8. Is this source offering a new explanation for observed phenomena, or 
simply attacking the existing explanation?

As with claims of suppression, this claim does not offer any evidence 
that the discovery is valid. Anyone attacking an existing explanation 
had better offer evidence. Absent evidence, all you know is that they do 
not like the existing explanation, for whatever reason.

12. What does the bulk of evidence point to - the new claim or something 

Is the source focusing on one small thing and ignoring a huge base of 
accumulated evidence that points to something else? Be sure to look at 
the claim in the context of current knowledge.

13. What kind of reasoning has been used?

Beware of "Post hoc, ergo propter hoc" thinking; this implies confusion 
about causation. This latin adage translates as "after this, therefore 
because of this." If A happened then B happened, then obviously A caused 
B. This is a very bad assumption. A claim that A caused B must be 
accompanied by credible evidence showing how A caused B. To put it 
simply, correlation does not mean causation.

16. Is it possible to test the claim?

Explanations that do not make any testable predictions are of no use. 
They do not add to knowledge.

20. Is the claim/discovery really spectacular?

Scientists will tell you that "extraordinary claims require 
extraordinary evidence." They mean exactly that. If the claim is that 
life has been found on Mars or that someone has found a way to eliminate 
aging, really extraordinary evidence will be required to demonstrate it. 
A vague observation or a few anecdotes will not do the job.

21. Beware of special pleadings.

Be especially wary of any claim or excuse that the claimed effect cannot 
be measured for some reason. It may be that "the presence of a skeptic 
contaminates the effect", or "attempting to measure the effect destroys 
it", or "bad vibrations interfered", or something like that. ***An 
"effect" that cannot be measured likely does not exist.*** (emphasis added).

Jerry Baker
Aquatic-Plants mailing list
Aquatic-Plants at actwin_com