Music & Language Studies

MEREWETHER, Frank C. and Murray Alpert. “The Components and Neuroanatomic Bases of Prosody.” Journal of Communication Disorders 23/4-5 (Aug-Oct 1990): 325-336.

A review of the literature for a psychiatric audience. Notes some contradictory findings. Of particular note was one reviewed article by Danly et al., which indicated severe disturbance of fundamental frequency (presumably in speech production) in aphasics, but not in RH lesions. This directly contradicts several studies by Elliott D. Ross and colleagues. They summarize the literature as

indicating the left hemisphere’s dominance for inflection indicating syntax, and the right hemisphere’s dominance for those aspects that indicate the speakers’ intentions, emotions, or feelings.

Also, reviewed are some studies regarding rate of speech, which appears to be moderated by dopamine levels in the brain.

BELL, William L., Diana L. Davis, Anna Morgan-Fisher, and Elliott D. Ross. “Acquired Aprosodia in Children.” Journal of Child Neurology 5 (Jan 1990): 19-26.

Records case histories for two 10-year old girls who suffered from mild acquired right-hemispheric damage, and comparisons to seven age-matched female controls. Patients and controls were assessed for spontaneous and imitative affective prosody in speech, as well as spontaneous requested singing of “Happy Birthday.” Both patients showed initial deficits in spontaneous affective prosody. [1] Case 2 exhibited severe loss of imitative affective prosody as well, whereas the affective repetition deficits for Case 1 were reported as minor. Similarly, relative to controls, singing was impaired for Case 1, whereas it appeared normal for Case 2. This lead the authors to suspect a direct correlation between modulation of pitch in speech and singing. Since the findings of Gordon & Bogen 1974 appeared to contradict this conclusion, the authors confirmed by private communication with Dr. Bogen, that the normality of speech reported therein referred only to propositional and not affective components of speech prosody.

It is interesting to compare this, however, with the findings of Yamadori, et al 1977, which note that text production is normally intact during singing by Broca’s aphasics. Therefore, while pitch may be dominantly encoded in the right hemisphere for both speaking and singing (though I believe the evidence is inconclusive at this point) [2], text appears to have dual encoding in both hemispheres. It may be that right-hemispheric encoding of words is purely phonological, that the sounds of the words may be separately or redundantly encoded from a holistic representation in the opposite hemisphere, which includes sound or articulatory elements. Empirical studies could address this question, with brain scans of singers during production of songs with both familiar (native) and unfamiliar (foreign) words. However, if dual-encoding is found to be the case, it may be that singers switch back and forth between linguistic and phonological processing of text sounds, in the course of song production. Therefore brain imaging techniques that showed good temporal resolution would be essential. [3]

[1] Happily, normal functioning was recovered by both patients within a matter of months.

[2] This is complicated by issues regarding the acquisition of song. Children and inexperienced singers often attend first to linguistic, then rhythmic, and only last to melodic elements of singing. It is possible that processing for melody in song is more variable in the population than other aspects of vocal production, which may account for ambiguous and contradictory evidence in the literature. This compares to findings that early musical training, in particular for pianists and violinists, often causes a shift to left-hemisphere dominance for procedures that are otherwise right-hemispheric in the general population.

Additionally, it must be pointed out that pitch processing for music (and thus assumedly for singing) is more precise than it is for prosody in speech. Non-lexically significant pitch (i.e. non-tone-dependent intonation, in both tone and nontone languages) appears to be processed as a gestalt. Because of this, it seems a reasonablehypothesis to expect intonational pitch processing and musical pitch processing to be encoded differentially in the brain. Such redundancies in the brain may have arisen as a result of separate adaptive pressures converging on auditory processing. And this may explain some of the confusions which arise as a result of trying to define boundaries for musical and linguistic phenomena. It just may be that acoustic signals are processed redundantly, and varyingly, by different members of the population, and under different expectational conditions.

[3] It is commonly acknowledged that fMRI, while giving better spatial data than PET scans, trades this off with lousy temporal data. New techniques combine both MRI and PET technologies, but these may still prove inadequate for the type of study proposed. We can only hope that the technologies will continue to progress, along with the greater complexity of the questions researchers ask.

ROSS, Elliott D., Britt Anderson, and Anna Morgan-Fisher. “Crossed Aprosodia in Strongly Dextral Patients.” Archives of Neurology 46 (Feb 1989): 206-9.

Reports on two cases of strongly right-handed individuals, suffering from left-hemisphere infarctions, but exhibiting aprosodias, rather than aphasias. Both patients showed deficits both in the imparting and comprehension of affective prosody .The resulting syndrome is called a crossed aprosodia, which corresponds to the rare incidence of crossed aphasias. This finding of crossed aprosodias is interpreted as evidence that propositional and prosodic aspects of language are encoded in opposite hemispheres, regardless of whether dominance is normal or anomalous.

FERNALD, Anne. “Intonation and Communicative Intent in Mothers’ Speech to Infants: Is the Melody the Message?” Child Development 60 (1989): 1497-1510.

Conducted a perceptual study, using 40 experienced parents and 40 students inexperienced with infants as informants. Stimuli were recordings were made of both infant-directed and adult-directed speech of 5 mothers of 12-month-old preverbal infants, which were then filtered to eliminate linguistic content, leaving only the prosodic information of each utterance. Subjects were asked to identify (forced choice) the speaker’s intent for each utterance (attention-bid, approval, prohibition, comfort, game/telephone). Listeners were able to deduce speakers’ intentions far more accurately for infant-directed speech than for adult-directed speech. This lead the researchers to conclude that the prosodic patterns of infant-directed speech are not only more animated but more informative than for adult-directed speech, and that these may provide reliable cues to preverbal infants in comprehending speaker’s intent.

SHIPLEY-BROWN, Frances, William O. Dingwall, Charles I. Berlin, Grace Yeni-Komshian, and Sandra Gordon-Salant. “Hemispheric Processing of Affective and Linguistic Intonation Contours in Normal Subjects.” Brain and Language 33 (1988): 16-26.

Conducted dichotic listening experiments to test the hypothesis that processing of affective intonation is dissociable from processing of linguistic intonations. The distinction between these domains is based on Monrad-Krohn’s categories of prosodic function (see Monrad-Krohn 1963). Affective intonation is presented as contrasts between emotional affect (happy, sad, angry), whereas linguistic intonation relates to different types of utterances (statement, question, continuation). The results indicated that both of these conditions exhibited a left ear advantage (right hemispheric processing), however this effect was stronger for the affective condition. The authors argue that this can be explained in terms of a continuum, wherein the more linguistic the function of intonation, the less dominantly it is processed by the RH.

Since they do not cite Edmondson et al 1987, I can only assume that they were unaware of the study, which demonstrated relatively normal functioning of linguistic tone, but flattening of affective pitch contours. It was argued therefore that linguistic tone remains LH dominant, even in the presence of RH dominance for affective prosody. On the one hand, this might support the argument that processing of pitch in speech is along a continuum, such that the more affective the more RH dominant, the more linguistic the more LH. Additionally, however, it raiises questions as to the appropriateness of Monrad-Krohn’s distinction (adopted by this study) between linguistic and affective prosody, without attending to the special case of lexical tone, which seems to be subsumed in his description of intrinsic prosody. Thus the experimental design of this study is weakened.

ROSS, Elliott D., Jerold A. Edmondson, G. Burton Seibert, and Richard W. Homan. “Acoustic Analysis of Affective Prosody during Right-Sided Wada Test: A Within-Subjects Verification of the Right Hemisphere’s Role in Language.” Brain and Language 33 (1988): 128-45.

Reviews evidence from a within-subjects study of five right-handed patients, each undergoing a WADA test (which causes transient “paralysis” to each brain hemisphere separately, as a presurgical screening procedure for epileptics being considered for prophylactic commisurotomies). Findings show the left-hemisphere condition causing loss of propositional speech, and the right-hemisphere condition causing a loss of affective speech. These findings further confirm the contention that

the right hemisphere modulates dominantly the affective components of language.

MEHLER, Jacques, Peter Jusczyk, Ghislaine Lambertz, Nilofar Halsted, Josiane Bertoncini, and Claudine Amiel-Tison. “A Precursor of Language Acquisition in Young Infants.” Cognition 29 (1988): 143-78.

A series of studies were done on four-day-old French and 2-month-old American infants. Both groups were able to distinguish between utterances in the native speech of the mother, and speech utterances from another language. Follow up studies indicated that these infants were not likewise able to distinguish between the utterances of two foreign languages. Further experiments were conducted with filtered speech, such that the segmental features were no longer clearly present. These final experiments indicate that infants might possibly use prosodic cues to classify utterances as either native or non-native speech.

GORELICK, Philip B. and Elliott D. Ross. “The Aprosodias: Further Functional-Anatomical Evidence for the Organisation of Affective Language in the Right Hemisphere.” Journal of Neurology, Neurosurgery, and Psychiatry 50 (1987): 553-60.

Reviews the recent literature regarding aprosodias, and presents further evidence to support the earlier contention (Ross 1981) that right-hemispheric focal lesions cause aprosodic deficits, which mirror aphasic deficits due to analogous lesions in the left hemisphere.

EDMONDSON, Jerold A., Jin-Lieh Chan, G. Burton Seibert, and Elliott D. Ross. “The Effect of Right-Brain Damage on Acoustical Measures of Affective Prosody in Taiwanese Patients.” Journal of Phonetics 15 (1987): 219-33.

Examined eight right-handed native-speakers of Taiwanese, a tone language. All patients showed signs of emotional flattening in their speech, though none showed deficits in the production of tone or other segmental aspects of speech. As the authors wrote:

The major emphasis of this paper is to investigate loss of affective prosody from focal right-brain damage in patients who speak a tone language. (220)

Comparison was made to findings for similar patients who were native-speakers of English.
The acoustic signals produced by patients’ speech utterances was analyzed along 12 parameters. In general, as expected, it was found that somewhat different parameters are normally involved in altering the affective content of an utterance. Unexpectedly, however, the researchers found that there was a flattening of frequency modulation, even at the local level. Although the degree of flexibility available in normal tonal speech is significantly narrower than for non-tonal speech, it yet remains an important marker of emotional affect.

The authors believe the explanation for this finding may lie in the fact that tones in context are allowed some prosodic variation from their citation forms without disrupting lexical information. Although the presence of tones in a language clearly places constraints on the manipulation of F0… enough freedom or play in the required precision of tone contrasts appears to remain for speakers to exploit this freedom for affective purposes. (230)

In conclusion:

it was once again found that communicative abilities of humans are lateralized according to behavior itself (affective vs. linguistic) and not according to the physical/acoustical carrier that expresses this behavior. …[A]lthough difference acoustic profiles underlie affective prosody for Taiwanese vs. English patients, the behavioral consequences are the same, i.e. affective flattening of voice. Thus, human languages show the features of a composite that is the product both of specific neurological organization of brain tissue and of the brain’s ability to react to the acoustical properties of a particular language, i.e. tone vs. non-tone, during the experience of language acquisition.

Thus, we have further evidence that expectancies play a major role in the perception and production of speech sounds, such that when language exploits aspects of pitch manipulation as part of the segmental level, as is the case in tone languages, such features are processed in the brain in fundamentally different ways from the same parameters when not within a segmentally contrastive context, even for speakers of tone languages. It is yet to be understood just how the perceptual apparatus is able to obtain this effect from the acoustic signal, though the motor theory of speech perception suggested elsewhere appears to provide one likely explanation.

SERAFINE, Mary Louise, Janet Davidson, Robert G. Crowder, and Bruno Repp. “On the Nature of Melody-Text Integration in Memory for Songs.” Journal of Memory and Language 25 (1986): 123-35.

Reports on experiments intended to follow up on possible theories explaining away the integration effect described in Serafine, Crowder & Repp 1984, above. The first new experiment tested an hypothesis which posited that apparent integration was the result of subtle semantic assignments of meaning to the melodies involved, in ways familiarly termed word-painting in musical fields. The target songs (original presentations) however, had their words replaced by nonsense syllables, which nonetheless sounded like words (i.e. “Cape Cod girls they have no combs” became “Tade top berf shey jaze mo tong”). Thus, any effect of semantics would be destroyed. The experiment however failed to support this hypothesis.

Melodies were recognized better when they were paired with their original text than when paired with another, even if equally familiar text. Since this effect held when nonsense texts were used, the semantic hypothesis must be ruled out as an explanation for the integration effect.(129)

A second hypothesis was that the integration effect was not due to a strengthening of memory on the basis of pairing the melody with words, but rather the converse of a decrement effect due to the distracting influence of wrong words.

Perhaps the melody by itself could be recognized well without the original words, but adding new or mismatched words somehow disguises the retained melodic information. (129)

This hypothesis was tested by counterposing a hummed version of the song with properly matched and mismatched stimuli. The original meaningful words were retained in this experiment. The argument is that a hummed version should be recognized more easily than a mismatched version, and equally well as the properly matched target (Experiment 2). The findings were somewhat inconclusive. Although they failed to support the decrement hypothesis, they did not strongly disconfirm it. However,

melodies were better recognized in the presence of their original words than on their own, without words. (131)

Likewise, the words on their own should be more easily recognized without a novel melody, and equally so with the original melody (Experiment 3). Experiment 3 returned to the nonsense syllables, counterposing the sung originals with spoken versions. The findings for Experiment 3 were strong, since recognition of the text without a melody (whether original or mismatched) was near chance. That is, memory for nonsense words was better in the context of the melody with which they were first presented, than it was without a melody. In discussion of these findings, the experimenters note, however:

Integration of melody and text in memory for songs is an experimental result, not an explanation. (133)

They suggest certain avenues for further investigation. Specifically, they identify that the subtle influences that melody and text have on one another result in minute changes in the acoustic signal, which may likely be perceptually salient to a listener.

we suggest that integration in memory may result from other, more subtle effects that melody and text have on each other. These may be thought of, broadly, as prosodic effects in that they concern the nonsemantic sound pattern of either melody or text,. For example, a text’s consonant pattern, vocal timbres, and accents may affect the attack and decay patterns, stresses, or other aspects of tones in a melody….What this means is that a melody is physically different depending on the words to which it is sung. (133)

Although the authors argue that the hummed stimuli rule out the possibility that a melody could be equally recognized without its text, it would be interesting to test this hypothesis not with a hummed variation, but rather by pairing an instrument with the voice in the first presentation (say, a flute or violin, doubling the voice part), and playing only the instrumental version of the melody in the test condition. Would subjects be equally able to recognize the melody presented without a voice, and thus without the original words?

It has been shown that various stimulus conditions alter the mental mechanism by which sounds are processed. Perhaps the human voice (even without linguistically meaningful sounds) provides a certain degree of distraction which alters a listener’s strategy in analyzing the sound signal. If that were the case, the decrement hypothesis might prove correct. However, rather than it being the deleterious influence of improper words, it is the presence of the human voice that alters the strategies employed to make sense of the input signal.

ROSS, Elliott D., Jerold A. Edmondson, and G. Burton Seibert. “The Effect of Affect of Various Acoustic Measures of Prosody in Tone and Non-Tone Languages: A Comparison Based on Computer Analysis of Voice.” Journal of Phonetics 14 (1986): 283-302.

Ross, et al. set out to compare the ability to signal affect through prosody in three tone languages (Taiwanese, Mandarin, and Thai) with such ability in English. The results indicate that the presence of tone in a language severely restricts the manipulation of local pitch frequencies (referred to as F0, or fundamental frequency), for the purpose of signalling affect. This is primarily due to the fact that such local pitch manipulations might result in changes of semantic meaning:

In non-tone languages such alterations will not disturb linguistic information. However, in tone languages, in which the relative height between and among tones may be contrastive, such alterations could disrupt linguistic information. For example, if one had a low-mid sequence of two flat tones, a change in F0 Variation might produce a low-high sequence. (298)

In English, it is known that the right hemisphere tends to be dominant for the processing of prosodic features of speech. However, this study was largely motivated by difficulties in finding such effects for speakers of tone languages. Interestingly, the researchers note:

if the modulation of a specific acoustical parameter is lateralized in the brain to either the right or left hemisphere, the lateralization is dependent on the behavioral properties of the parameter and not on its acoustical properties. (300)

Again, we have further evidence, therefore, that a major factor involved in the process of audition, human minds are largely influenced by expectancies regarding what type of stimuli they are receiving. This leads one to suspect that much of this can be voluntarily altered by a listener, in order to perceive an input signal in various ways. For instance, it is quite possible that a skilled listener could choose to hear a spoken sentence as melody and rhythm, ignoring its linguistic function. Indeed, the composers Janáček and Mussorgsky are known to have made just such claims.

KELLEY, Darcy B. “A Motor Theory of Song Perception.” Trends in Neurosciences 9/4 (Apr 1986): 149-150.

Kelley reviews some work done by researchers studying motor responses to auditory stimuli in songbirds. The findings indicate that a major factor in the processing (and by implication the comprehension) of conspecific-produced auditory stimuli is a motor neuron loop associated with the production of such sounds in the syrinx of the listening songbirds. This suggests “that song may only be totally intelligible to a nervous system capable of singing.” Interestingly, in the birdsong species studied, it is the males who sing. The studies indicated therefore the possibility that the songs, rather than being courtship displays for the females, may be combat displays between the males.

Since research dealing with humans has made similar connections between auditory and articulatory processes (cf. Kimura 1967), the potential ramifications of a motor theory of song perception, corresponding to a motor theory of speech perception, are wide ranging, especially in the realms of speech therapy and vocal pedagogy. In terms of comparative studies, it would be useful to ascertain whether there are gender differences in perception in humans. However, since male and female humans sing (in fact, females of our species appear to sing more than males), the process of the development of such skills may differ considerably from the process in birds.