Harmonica production was too low in the 1860s for harmonicas to have been very widespread in the US during slavery times. Production wasn't really large enough until about the 1870s or '80s, which was in fact a time of great hope for black folks, with African Americans actually getting elected to State office, until white racists figured out ways to shut them out of the entire democratic process for the better part of a century.
The popular image of the harmonica being played by Civil War soldiers was created by Hollywood screenwriters. They remembered the very real phenomenon of the harmonica being very widespread during the first world war of 1914-1918, when annual harmonica production (and exportation) was well into the multiple millions, and projected that back to the Civil war scenes they were writing (soldiers gone off to war, a past era, harmonica). Some folks I know will disagree, but I'm pretty certain this is a false image. The production and distribution just weren't there.