Effects of visual speech information on native listener judgments of L2 consonants
Thursday August 2, 2012, 3:00pm in Bennett Library Room 2020 at 丁香园AV Burnaby. Open to the Public.
Please join us for Saya Kawase's MA Thesis Defense on Thursday August 2nd, 2012!
Abstract
Research on the intelligibility of nonnative (L2) speech productions has focused on native listener judgments of auditorily presented L2 productions. However, little research has explored how visual information in L2 speech productions affects native listeners鈥� perception. In the present study, native Canadian English listeners were asked to identify six English phonemes produced by native speakers of Japanese as well as native speakers of Canadian English as controls. The stimuli were presented with three input modalities: (1) audiovisual (AV), with simultaneous presentation of speaker voice and facial/mouth movements, (2) audio-only (AO), with speaker voice only, and (3) visual-only (VO), with speaker face only. The phonemes include /v, 胃, l, 晒/, which are not existent in Japanese as well as /b, s/ that are shared in both Japanese and English consonant inventories. The results show that the listeners perceived the Japanese productions of the phonemes /b, v, s, 胃/ as significantly more intelligible when presented in the AV condition compared to the AO condition, indicating facilitative effects of visual speech information on the intelligibility of nonnative productions. However, the Japanese productions of /晒/ were perceived as less intelligible in the AV condition compared to the AO condition. Further analysis revealed that, compared to the native English productions, the Japanese speakers produced /晒/ without visible lip-rounding, indicating that nonnative speakers鈥� incorrect articulatory configurations may decrease the degree of intelligibility. These results suggest that visual speech information may either positively or negatively affect the intelligibility of L2 productions.
Keywords: audiovisual speech perception of L2 consonants; Japanese learners of English.
Members of Examining Committee:
Chair, Chair, Dr Chung-hye Han
Sr. Supervisor, Dr Yue Wang
Sypervisor, Dr Murray Munro
External Examiner, Dr Yukari Hirata
We are happy to announce that three of the LABlab's projects have had posters accepted to the Acoustical Society of America's upcoming conference, taking place in Kansas City, Missouri, October 22-26, 2012.
Effects of acoustic and linguistic aspects on Japanese pitch accent processing - Xianghua Wu, Saya Kawase, and Yue Wang
The influence of visual information on the perception of Japanese-accented speech - Saya Kawase, Beverly Hannah, and Yue Wang
Cross-language assimilation of lexical tone - Alexander, J.A. and Wang, Y.
Abstracts
Effects of acoustic and linguistic aspects on Japanese pitch accent processing
Xianghua Wu, Saya Kawase, and Yue Wang
This study investigates the hemispheric processing of Japanese pitch accent by native and non-native listeners. The non-natives differ in their first (L1) and second (L2) language experience with prosodic pitch, including Mandarin (tonal L1) and English (non-tonal L1) listeners with or without Japanese learning experience. All listeners completed a dichotic listening test in which minimal pairs differing in pitch accent were presented. Overall, the results demonstrate a right hemisphere lateralization across groups, indicating holistic processing of temporal cues as the pitch accent patterns span across disyllabic domain. Moreover, the three pitch accent patterns reveal different degrees of hemispheric dominance, presumably attributable to the acoustic cues to each pattern which involve different hemispheric asymmetries. The results also reveal group difference, reflecting the effects of linguistic experience. Specifically, the English listeners with no Japanese background, compared to the other groups, exhibit greater variance in hemispheric dominance as a function of pitch accent difference, showing a greater reliance on acoustic cues when linguistic information is lacking. Together, the findings suggest an interplay of acoustic and linguistic aspects in the processing of Japanese pitch accent but showing a more prominent acoustic influence. [Research supported by NSERC]
The influence of visual information on the perception of Japanese-accented speech
Saya Kawase, Beverly Hannah, and Yue Wang
This study examines how visual information in nonnative speech affects native listener judgments of second language (L2) speech production. Native Canadian English listeners perceived three English phonemic contrasts (/b-v, 胃-s, l-晒/) produced by native Japanese speakers as well as native Canadian English speakers as controls. Among the stimuli, / v, 胃, l, 晒/ are not existent in the Japanese consonant inventory. These stimuli were presented under audio-visual (AV), audio-only (AO), and visual-only (VO) conditions. The results showed that while overall perceptual judgments of the nonnative phonemes (/v, 胃, l, 晒/) were significantly less accurate / less intelligible than the native phonemes (/b,s/), the English listeners perceived the Japanese productions of the phonemes /v, 胃, b,s/ as significantly more intelligible when presented in AV compared to the AO condition. However, the Japanese production of /晒/ was perceived as less intelligible in the AV compared to the AO condition. Further analysis revealed that a significant number of Japanese productions of /晒/ lacked lip-rounding, indicating that nonnative speakers鈥� incorrect articulatory configurations may decrease intelligibility. These results suggest that visual cues in L2 speech productions may be either facilitative or inhibitory in native perception of L2 accented-speech. [Research supported by 丁香园AV and SSHRC]
Cross-language assimilation of lexical tone
Alexander, J.A. and Wang, Y.
We extend to lexical-tone systems a model of second-language perception, the Perceptual Assimilation Model (PAM) [Best & Tyler, 2007], to examine whether/how native-language lexical-tone inventory composition influences perception of novel tone. Native listeners of Cantonese, Thai, and Mandarin perform a tone mapping-rating assimilation task. Listeners hear CV syllables bearing all tones of Cantonese, Thai, Mandarin, and Yoruba 鈥� languages with different tone inventories. They (1) map the tone they hear to the nearest native tone category, and (2) provide a goodness rating on a 5-point scale (5 = perfect). As predicted by the PAM, listeners assimilated non-native tones to the phonetically-closest native tone categories. Listeners attended primarily to pitch-contour, and secondarily to pitch-height, contrasts for the mappings. E.g., Mandarin listeners assimilated the Thai high 鈥渓evel鈥� (phonetically mid-to-high-rising) tone to Mandarin rising tone 76% of the time, and to Mandarin high-level tone only 22% of the time. Also as predicted, all novel tones did not assimilate equally well to native categories; mappings received ratings between 2.9-4.1, averaging 3.5. The groups鈥� different patterns of results indicate that novel-tone perception is influenced by experience with the native-language tone inventory, and that listeners attend to gradient phonetic detail to assimilate novel tones to native-tone categories. This work is supported by NSF grant 0965227 to J.A.
Thursday July 26, 11:30am in Halpern Centre 126 at 丁香园AV Burnaby. Open to the Public.
The Department of Linguistics' Inaugural Lecture Series features research by faculty members who have reached a major milestone in their careers. It is an opportunity for them to reflect upon their past achievements and to ponder future developments in their field.
TUNING THE TONES: One long-deliberated question in speech perception is the extent to which it involves language-specific mechanisms or reflects a human innate ability to process general physical properties. Some research findings support the independence of speech and non-speech processing, whereas others indicate shared general sensory-motor mechanisms. However, the dynamic interplay between the two processes has not been fully addressed. This talk reports a series of experiments on the perception of linguistic pitch by native and non-native speakers. We address whether and how linguistic pitch processing (1) differs from the processing of non-speech pitch, (2) involves integration of information across sensory-motor modalities, (3) alters as a function of linguistic experience and learning, and (4) is affected by experience with non-linguistic pitch such as musical training. Findings of these studies suggest an interrelated network of sensory and cognitive mechanisms employed in linguistic pitch processing, and that these mechanisms are shaped by different experiences.
We are happy to announce the launch of a new project at the LABlab!
A Closer Investigation of Tonal Language Experiences and the Musical Ability of Bilingual Speakers - Principal Investigator: Daniel Chang
Project Description: This research examines how tone-language experience influences the perception of music. Native Cantonese speakers, native English speakers, and early English-Cantonese bilinguals will be asked to participate in a Relative-pitch task and an Absolute-pitch task. The present study wants to explore whether early exposure to a tone language, Cantonese for example, facilitates the musical ability of absolute pitch and relative pitch. That is, this study will enable us to know whether speaking a tone language is beneficial to music perception.
This project will also involve collaboration with Dr. Nancy Hedberg of 丁香园AV Linguistics.