Comparing Reliability of Perceptual Ratings of Roughness and Acoustic Measures of Jitter Acoustic analysis is often favored over perceptual evaluation of voice because it is considered objective, and thus reliable. However, recent studies suggest this traditional bias is unwarranted. This study examined the relative reliability of human listeners and automatic systems for measuring perturbation in the evaluation of pathologic voices. Ten experienced ... Research Article
Research Article  |   February 01, 1995
Comparing Reliability of Perceptual Ratings of Roughness and Acoustic Measures of Jitter
 
Author Affiliations & Notes
  • C. Rose Rabinov
    Division of Head and Neck Surgery UCLA School of Medicine, and Audiology and Speech Pathology VA Medical Center, West Los Angeles
  • Jody Kreiman
    Division of Head and Neck Surgery UCLA School of Medicine, and Audiology and Speech Pathology VA Medical Center, West Los Angeles
  • Bruce R. Gerratt
    Division of Head and Neck Surgery UCLA School of Medicine, and Audiology and Speech Pathology VA Medical Center, West Los Angeles
  • Steven Bielamowicz
    Division of Head and Neck Surgery UCLA School of Medicine, and Audiology and Speech Pathology VA Medical Center, West Los Angeles
  • Contact author: Jody Kreiman, PhD, Audiology and Speech (126), West Los Angeles VA Medical Center, 11301 Wilshire Boulevard, Los Angeles, CA 90073. E-mail: ianpjek@mvs.oac.ucla.edu
Article Information
Speech, Voice & Prosodic Disorders / Voice Disorders / Hearing & Speech Perception / Acoustics / Speech, Voice & Prosody / Speech / Research Articles
Research Article   |   February 01, 1995
Comparing Reliability of Perceptual Ratings of Roughness and Acoustic Measures of Jitter
Journal of Speech, Language, and Hearing Research, February 1995, Vol. 38, 26-32. doi:10.1044/jshr.3801.26
History: Received March 17, 1994 , Accepted June 30, 1994
 
Journal of Speech, Language, and Hearing Research, February 1995, Vol. 38, 26-32. doi:10.1044/jshr.3801.26
History: Received March 17, 1994; Accepted June 30, 1994

Acoustic analysis is often favored over perceptual evaluation of voice because it is considered objective, and thus reliable. However, recent studies suggest this traditional bias is unwarranted. This study examined the relative reliability of human listeners and automatic systems for measuring perturbation in the evaluation of pathologic voices. Ten experienced listeners rated the roughness of 50 voice samples (ranging from normal to severely disordered) on a 75 mm visual analog scale. Rating reliability within and across listeners was compared to the reliability of jitter measures produced by several voice analysis systems (CSpeech, SoundScope, CSL, and an interactive hand-marking system). Results showed that overall listeners agreed as well or better than “objective” algorithms. Further, listeners disagreed in predictable ways, whereas automatic algorithms differed in seemingly random fashions. Finally, listener reliability increased with severity of pathology; objective methods quickly broke down as severity increased. These findings suggest that listeners and analysis packages differ greatly in their measurement characteristics. Acoustic measures may have advantages over perceptual measures for discriminating among essentially normal voices; however, reliability is not a good reason for preferring acoustic measures of perturbation to perceptual measures.

Acknowledgments
This research was supported in part by NIDCD grant # DC 01797 and by VA research funds. A preliminary version was presented at the Workshop on Standardization in Acoustic Voice Analysis, which took place at the Denver Center for the Performing Arts in February, 1994. We thank Lorraine Ramig, James Hillenbrand, and two anonymous reviewers for their many helpful comments.
Order a Subscription
Pay Per View
Entire Journal of Speech, Language, and Hearing Research content & archive
24-hour access
This Article
24-hour access