Prosodic Structure Shapes the Temporal Realization of Intonation and Manual Gesture Movements PurposePrevious work on the temporal coordination between gesture and speech found that the prominence in gesture coordinates with speech prominence. In this study, the authors investigated the anchoring regions in speech and pointing gesture that align with each other. The authors hypothesized that (a) in contrastive focus conditions, the gesture ... Article
Article  |   June 01, 2013
Prosodic Structure Shapes the Temporal Realization of Intonation and Manual Gesture Movements
 
Author Affiliations & Notes
  • Núria Esteve-Gibert
    Universitat Pompeu Fabra, Barcelona, Spain
  • Pilar Prieto
    Institució Catalan de Recerca i Estudis Avançats, Universitat Pompeu Fabra, Barcelona, Spain
  • Correspondence to Núria Esteve-Gibert: nuria.esteve@upf.edu
  • Editor: Jody Kreiman
    Editor: Jody Kreiman×
  • Associate Editor: Shari Baum
    Associate Editor: Shari Baum×
Article Information
Speech, Voice & Prosody / Speech
Article   |   June 01, 2013
Prosodic Structure Shapes the Temporal Realization of Intonation and Manual Gesture Movements
Journal of Speech, Language, and Hearing Research, June 2013, Vol. 56, 850-864. doi:10.1044/1092-4388(2012/12-0049)
History: Received February 7, 2012 , Revised August 10, 2012 , Accepted September 20, 2012
 
Journal of Speech, Language, and Hearing Research, June 2013, Vol. 56, 850-864. doi:10.1044/1092-4388(2012/12-0049)
History: Received February 7, 2012; Revised August 10, 2012; Accepted September 20, 2012
Web of Science® Times Cited: 5

PurposePrevious work on the temporal coordination between gesture and speech found that the prominence in gesture coordinates with speech prominence. In this study, the authors investigated the anchoring regions in speech and pointing gesture that align with each other. The authors hypothesized that (a) in contrastive focus conditions, the gesture apex is anchored in the intonation peak and (b) the upcoming prosodic boundary influences the timing of gesture and intonation movements.

MethodFifteen Catalan speakers pointed at a screen while pronouncing a target word with different metrical patterns in a contrastive focus condition and followed by a phrase boundary. A total of 702 co-speech deictic gestures were acoustically and gesturally analyzed.

ResultsIntonation peaks and gesture apexes showed parallel behavior with respect to their position within the accented syllable: They occurred at the end of the accented syllable in non–phrase-final position, whereas they occurred well before the end of the accented syllable in phrase-final position. Crucially, the position of intonation peaks and gesture apexes was correlated and was bound by prosodic structure.

ConclusionsThe results refine the phonological synchronization rule (McNeill, 1992), showing that gesture apexes are anchored in intonation peaks and that gesture and prosodic movements are bound by prosodic phrasing.

Acknowledgments
This research was funded by two research grants awarded by the Spanish Ministry of Science and Innovation (Grant FFI2009-07648/FILO, “The Role of Tonal Scaling and Tonal Alignment in Distinguishing Intonational Categories in Catalan and Spanish,” and Grant FFI2012-31995, “Gestures, Prosody and Linguistic Structure”); by a grant from the Generalitat de Catalunya (2009SGR-701), awarded to the Grup d’Estudis de Prosòdia; by Grant RECERCAIXA 2012 (for the project “Els Precursors del Llenguatge. Una Guia TIC per a Pares i Educadors” awarded by Obra Social ‘La Caixa’); and by the Consolider-Ingenio 2010 (CSD2007-00012) grant. A preliminary version of this article was presented at the following conferences: Architectures and Mechanisms for Language Processing (Paris, France, September 1–3, 2011), Gesture and Speech in Interaction (Bielefeld, Germany, September 5–7, 2011), V International Conference for Gesture Studies (Lund, Sweden, July 24–27, 2012), and Laboratory Phonology (Stuttgart, Germany, July 27–29, 2012). We would like to thank participants at those meetings—especially Mark Swerts, Heather Rusiewicz, Stefan Kopp, Zofia Malisz, Stefanie Shattuck-Hufnagel, Adam Kendon, John J. Ohala, and Robert Port—for their helpful comments. We thank Aurora Bel, Louise McNally, and José Ignacio Hualde for being part of the PhD project defense and, in particular, for their useful advice regarding the methodology used in this study. We also thank Joan Borrès for his help with running the experiment and carrying out the statistical analysis, and Paolo Roseano for his comments on Praat annotations.
Order a Subscription
Pay Per View
Entire Journal of Speech, Language, and Hearing Research content & archive
24-hour access
This Article
24-hour access