[Harp-L] Comb Materials: the Perfect Test Harmonica?

[Vern]

Thank you for the description of your test design.   I approve of your precautions to make certain that comb material was the only difference among the harps.  In the following comments, I will explain why I think your controls were inadequate to allow listeners to demonstrate the ability to distinguish among comb materials by their sound alone.


> On May 14, 2016, at 10:27 AM, Tom Halchak <info at bluemoonharmonicas.com> wrote:

> …..And thank you for your account of the comb test at SPAH 2010.  It is
> unfortunate that in spite of your best efforts the test did not come off
> the way you wanted and failed to provide any meaningful data…….

Granting that the conditions for that test did not satisfy Brendan, some conclusions  are inescapable.  Players recorded differences that did not agree with the materials or each other.  When someone plays the same exact harp three times and perceived differences cannot come from the materials.

> …….it is tragic that one of the participants deliberately sabotaged the test
> and influenced others to perhaps do the same.

He did not sabotage the entire test…only a small part that were his responses.  It may not have been sabotage at all.  It may have been that he heard no differences and that is what he reported. It is certain that he wasn’t happy about the test.   Unfortunately, he isn’t available to clarify the question.  I do not to reveal the names of test participants.
> 
> ...there was no attempt to conceal or deceive either the players or the
> audience about the comb material being played …..

There was no deceit in the 3 previous tests.  Deceit is causing someone to believe something that is not true.  It was made plain that the listeners or players would not know which material was being played.  

> …….I solicited the assistance of four very respected professional players…..
> 
Given your goal was to entertain as well as test, this choice is understandable. However, machine-blown harps would have eliminated the spurious human-player variables.  Such a machine was used for a part of the 1997 test.  Yes…this discussion has been ongoing for 20 years!  It is resurrected periodically when newbies  start asserting again that “wood is warm” or “metal is bright”.  This old curmudgeon detects the aroma of BS and comes roaring from his den for another round.
> 
> ….. the demand for high quality custom combs is something that was present before Blue Moon came into existence, it is present now, and it will be present when I am dead and gone. 
Stability, appearance, resistance to moisture, and other things are all valid reasons to  buy custom combs.  My only point is that comb material does not affect tone.

> 
> 
>  …...Taking the time to disassemble the harp, clean the reed plates as Michelle Lefree has stated she did, and then reassembling the harp with a different comb had to have
> taken five minutes more or less.  That’s a long time between listening to
> different comb materials with zero opportunity to do a side by side…..

The reassembly was a little over a minute.  However, it took minutes longer for each participant to wait his turn at playing it.  They were asked for their impressions the sound of the harp presented, not for comparisons with others. 

The time-consuming use of the same reed plates was adopted in response to objections to a previous test that the different reed plates affected the sound.  

You can’t win.  After the test, those disappointed with the results always find some  “fatal flaw” in the test conditions.  The claim is: “If not for this egregious defect in the test procedure, we would have been able to identify the materials from their tones."

Any perceptible difference that doesn’t last even a few minutes is irrelevant in the real world of harmonica playing. Harmonicas change from song to song, not from second to second.

> 
> At the end of the workshop we had a contest to see who would win “The
> Golden Ear Award”.  Everyone had a score sheet.  One by one each of the pro
> players went behind a curtain out of sight and played each of the
> harmonicas in a random order.

This was an attempt at blinding the test.  However, a serious effort to allow the listeners to demonstrate that they could detect materials uninfluenced by confirmation bias would require tighter controls. 

What were the recorded results?  What was the success rate?  Did the listeners tend to agree with each other even if wrong about the material?

Here are some recommended controls: 
… The harps should be blown by a machine to avoid human player variables.
… The testers should not comment on which numbered harp is being played.
… The listeners should not discuss or share their perceptions during the test.
… There should be more than one harp with each of the comb materials.  
… There should be fewer types of materials, maybe just brass and wood.
… There should be a large number of playings, 50 or more.
… The sequence should be random, allowing repetitions of the same harp and material.

Before the blinded portion of the test starts, it would be OK to have a “get-acquainted” session where harps are played with their comb materials known.  It would be OK to record listener’s impressions to compare with the results from the blinded test.

There should be someone from your test that would be willing to accept my $1000 wager that they cannot distinguish among comb materials under the controls listed above. You could provide the harps.  Any takers?

Vern