Re: [Harp-L] RE: THE COMB DEBATE REVISITED: SPAH Test

[Michelle LeFree <mlefree@xxxxxxxxxxxxxxxxxxxxxx>]

Venky Ramakrishna wrote:
> *Obviously the test listener will be biased because they may own/use one
> type of harmonica or another. FWIW, I'd suggest double-blinding and
> randomizing the comb/cover plates. Have sax/trombone/trumpet players in the
> room as control subjects.*
>    
Brendan and Vern, Venky is spot on here. If you hope to generate any 
statistical significance in your results you will need to design an 
experiment that employs randomized double-blinded sampling. Both the 
combs and the players have to be selected at random ahead of time (using 
a random umber generator). Only the managers of the experiment can 
possess the true data and they should use that knowledge only after the 
experiment is complete. Then the data should be subjected to extensive 
statistical analysis using tests like ANOVA (analysis of variance)  or 
the like:

http://en.wikipedia.org/wiki/Analysis_of_variance.

A statistician would first decide what level of statistical significance 
is desired in the results and then back-calculate using statistical 
models appropriate to the data to calculate the number of samples that 
would be required to yield that predetermined accuracy. That is the 
manner in which the big league players determine answers to questions 
like "what is the number of patients required" in evaluating a new 
medical procedure or medication. Billions of dollars hang in the balance 
in those undertakings so you know they need to get it right the first 
time. And they have to be able to defend their results.

I would expect that if you do this kind of statistical analysis on your 
experiment as it stands you will find that with only three observers you 
will need hundreds if not thousands of data points. I would further 
expect that if you increase the number of observers you will 
significantly reduce the number of samples required. I realize that the 
choice of only three listeners may have been based on acoustical 
considerations and the desire to have the listeners in close proximity 
to the players. But the statistical considerations of having more 
listeners may mitigate that.

Google "statistics of double blind experiments" or "experimental design 
statistics" for a wealth of information.

Gents, there would be little point in going to a lot of trouble and 
expense to carry out an experiment that yields answers that are limited 
to the subjective opinions of the observers.

For another unsolicited opinion, I'd like to see a Manji comb added. ;-)

At least that's the way I see it.

Michelle