A Weighted Reliability Measure for Phonetic Transcription Purpose The purpose of the present work is to describe and illustrate the utility of a new tool for assessment of transcription agreement. Traditional measures have not characterized overall transcription agreement with sufficient resolution, specifically because they have often treated all phonetic differences between segments in transcriptions as equivalent, thus ... Research Article
Research Article  |   December 01, 2006
A Weighted Reliability Measure for Phonetic Transcription
 
Author Affiliations & Notes
  • D. Kimbrough Oller
    The University of Memphis, Memphis, TN
  • Heather L. Ramsdell
    The University of Memphis, Memphis, TN
  • Disclosure
    Disclosure×
    The first author is one of the primary developers of the LIPP phonetic transcription and phonological analysis software used in this research and was previously a stockholder in the Intelligent Hearing Systems, which produces the software. Although he does not receive royalties or other payment from sales of the software, he does consulting work for a fee with individuals attempting to apply the software in their work.
    The first author is one of the primary developers of the LIPP phonetic transcription and phonological analysis software used in this research and was previously a stockholder in the Intelligent Hearing Systems, which produces the software. Although he does not receive royalties or other payment from sales of the software, he does consulting work for a fee with individuals attempting to apply the software in their work.×
  • Contact author: D. Kimbrough Oller, School of Audiology and Speech-Language Pathology, The University of Memphis, 807 Jefferson Avenue, Memphis, TN 38105. E-mail: koller@memphis.edu.
Article Information
Research Issues, Methods & Evidence-Based Practice / Speech, Voice & Prosody / Speech / Research Articles
Research Article   |   December 01, 2006
A Weighted Reliability Measure for Phonetic Transcription
Journal of Speech, Language, and Hearing Research, December 2006, Vol. 49, 1391-1411. doi:10.1044/1092-4388(2006/100)
History: Received September 28, 2005 , Accepted April 28, 2006
 
Journal of Speech, Language, and Hearing Research, December 2006, Vol. 49, 1391-1411. doi:10.1044/1092-4388(2006/100)
History: Received September 28, 2005; Accepted April 28, 2006
Web of Science® Times Cited: 5

Purpose The purpose of the present work is to describe and illustrate the utility of a new tool for assessment of transcription agreement. Traditional measures have not characterized overall transcription agreement with sufficient resolution, specifically because they have often treated all phonetic differences between segments in transcriptions as equivalent, thus constituting an unweighted approach to agreement assessment. The measure the authors have developed calculates a weighted transcription agreement value based on principles derived from widely accepted tenets of phonological theory.

Method To investigate the utility of the new measure, 8 coders transcribed samples of speech and infant vocalizations. Comparing the transcriptions through a computer-based implementation of the new weighted and the traditional unweighted measures, they investigated the scaling properties of both.

Results The results illustrate better scaling with the weighted measure, in particular because the weighted measure is not subject to the floor effects that occur with the traditional measure when applied to samples that are difficult to transcribe. Furthermore, the new weighted measure shows orderly relations in degree of agreement across coded samples of early canonical-stage babbling, early meaningful speech in English, and 3 adult languages.

Conclusions The authors conclude that the weighted measure may provide improved foundations for research on phonetic transcription and for monitoring of transcription reliability.

Acknowledgment
This work was supported by the National Institute on Deafness and Other Communication Disorders (R01DC006099; D. Kimbrough Oller, principal investigator, and Eugene Buder, co-principal investigator) and by the Plough Foundation.
Order a Subscription
Pay Per View
Entire Journal of Speech, Language, and Hearing Research content & archive
24-hour access
This Article
24-hour access