Re: [Harp-L] RE: THE COMB DEBATE REVISITED: SPAH Test



Venky Ramakrishna wrote:
*Obviously the test listener will be biased because they may own/use one
type of harmonica or another. FWIW, I'd suggest double-blinding and
randomizing the comb/cover plates. Have sax/trombone/trumpet players in the
room as control subjects.*
Brendan and Vern, Venky is spot on here. If you hope to generate any statistical significance in your results you will need to design an experiment that employs randomized double-blinded sampling. Both the combs and the players have to be selected at random ahead of time (using a random umber generator). Only the managers of the experiment can possess the true data and they should use that knowledge only after the experiment is complete. Then the data should be subjected to extensive statistical analysis using tests like ANOVA (analysis of variance) or the like:

http://en.wikipedia.org/wiki/Analysis_of_variance.

A statistician would first decide what level of statistical significance is desired in the results and then back-calculate using statistical models appropriate to the data to calculate the number of samples that would be required to yield that predetermined accuracy. That is the manner in which the big league players determine answers to questions like "what is the number of patients required" in evaluating a new medical procedure or medication. Billions of dollars hang in the balance in those undertakings so you know they need to get it right the first time. And they have to be able to defend their results.

I would expect that if you do this kind of statistical analysis on your experiment as it stands you will find that with only three observers you will need hundreds if not thousands of data points. I would further expect that if you increase the number of observers you will significantly reduce the number of samples required. I realize that the choice of only three listeners may have been based on acoustical considerations and the desire to have the listeners in close proximity to the players. But the statistical considerations of having more listeners may mitigate that.

Google "statistics of double blind experiments" or "experimental design statistics" for a wealth of information.

Gents, there would be little point in going to a lot of trouble and expense to carry out an experiment that yields answers that are limited to the subjective opinions of the observers.

For another unsolicited opinion, I'd like to see a Manji comb added. ;-)

At least that's the way I see it.

Michelle




This archive was generated by a fusion of Pipermail 0.09 (Mailman edition) and MHonArc 2.6.8.