Re: [Harp-L] RE: THE COMB DEBATE REVISITED: SPAH Test

[Vern <jevern@xxxxxxx>]

Here is a brief description of the planned test protocol:

The purpose is to see if players (not listeners in the audience) can attribute distinctive sounds to comb materials.

One set of covers and reedplates will be used.  They have been slightly modified to facilitate rapid comb changes.

The players will be blindfolded.  
The harp has an attached weight to mask the weight differences of brass, wood, and plastic combs.
There will be an aromatic material inside the lower cover (e.g. Mentholatum) to mask odor differences in wooden combs.
There will be a coating on the front of the comb that the player can touch with the tongue to mask taste and texture differences. 

Combs of several different materials will be placed in the harp in random order.

All of the players will play each comb and rate them against a set of adjectives selected by Brendan. e.g. warm, bright, loud, etc.
We have a plentiful supply of alcohol swabs for sanitizing the harp between players.

There will be a total number of ratings for (number of players) x (number of combs) x (number of adjectives)

This is not necessarily the best design for rigorous statistical analysis, but it should be fun and entertaining.

If the adjective ratings are evenly distributed among the comb materials, this will indicate the players' inability to distinguish one comb material from another by the sound.  
If the ratings have a high correlation between adjectives and materials, this will indicate that the ability to distinguish among them may exist.
For example, if the brass comb has a high total score for "bright" and a low score for "warm" and the pear wood comb has the opposite, that would signify that the players were able to distinguish between brass and wood.

Vern

On Jul 13, 2010, at 8:19 PM, Michelle LeFree wrote:

> Venky Ramakrishna wrote:
>> *Obviously the test listener will be biased because they may own/use one
>> type of harmonica or another. FWIW, I'd suggest double-blinding and
>> randomizing the comb/cover plates. Have sax/trombone/trumpet players in the
>> room as control subjects.*
>>   
> Brendan and Vern, Venky is spot on here. If you hope to generate any statistical significance in your results you will need to design an experiment that employs randomized double-blinded sampling. Both the combs and the players have to be selected at random ahead of time (using a random umber generator). Only the managers of the experiment can possess the true data and they should use that knowledge only after the experiment is complete. Then the data should be subjected to extensive statistical analysis using tests like ANOVA (analysis of variance)  or the like:
> 
> http://en.wikipedia.org/wiki/Analysis_of_variance.
> 
> A statistician would first decide what level of statistical significance is desired in the results and then back-calculate using statistical models appropriate to the data to calculate the number of samples that would be required to yield that predetermined accuracy. That is the manner in which the big league players determine answers to questions like "what is the number of patients required" in evaluating a new medical procedure or medication. Billions of dollars hang in the balance in those undertakings so you know they need to get it right the first time. And they have to be able to defend their results.
> 
> I would expect that if you do this kind of statistical analysis on your experiment as it stands you will find that with only three observers you will need hundreds if not thousands of data points. I would further expect that if you increase the number of observers you will significantly reduce the number of samples required. I realize that the choice of only three listeners may have been based on acoustical considerations and the desire to have the listeners in close proximity to the players. But the statistical considerations of having more listeners may mitigate that.
> 
> Google "statistics of double blind experiments" or "experimental design statistics" for a wealth of information.
> 
> Gents, there would be little point in going to a lot of trouble and expense to carry out an experiment that yields answers that are limited to the subjective opinions of the observers.
> 
> For another unsolicited opinion, I'd like to see a Manji comb added. ;-)
> 
> At least that's the way I see it.
> 
> Michelle
>