Reliability of Speech Naturalness Ratings of Stuttered Speech During Treatment This study evaluated the reliability with which relatively sophisticated and unsophisticated judges used a 9-point scale to rate the speech naturalness of speech samples from 10 clients in a treatment program for stuttering that employed prolonged speech. Judges rated repeated speech samples from different speakers during various phases of the ... Research Article
Research Article  |   October 01, 1992
Reliability of Speech Naturalness Ratings of Stuttered Speech During Treatment
 
Author Affiliations & Notes
  • Mark Onslow
    School of Communication Disorders The University of Sydney, Australia
  • Roger Adams
    Department of Behavioural Sciences The University of Sydney, Australia
  • Roger Ingham
    Department of Speech and Hearing Sciences University of California Santa Barbara
  • Contact author: Mark Onslow, School of Communication Disorders, The University of Sydney, PO Box 170, Lidcombe, NSW, 2141, Australia.
Article Information
Speech, Voice & Prosodic Disorders / Fluency Disorders / Research Issues, Methods & Evidence-Based Practice / Speech / Research Articles
Research Article   |   October 01, 1992
Reliability of Speech Naturalness Ratings of Stuttered Speech During Treatment
Journal of Speech, Language, and Hearing Research, October 1992, Vol. 35, 994-1001. doi:10.1044/jshr.3505.994
History: Received July 18, 1991 , Accepted January 13, 1992
 
Journal of Speech, Language, and Hearing Research, October 1992, Vol. 35, 994-1001. doi:10.1044/jshr.3505.994
History: Received July 18, 1991; Accepted January 13, 1992

This study evaluated the reliability with which relatively sophisticated and unsophisticated judges used a 9-point scale to rate the speech naturalness of speech samples from 10 clients in a treatment program for stuttering that employed prolonged speech. Judges rated repeated speech samples from different speakers during various phases of the program. Different groups of sophisticated and unsophisticated judges made ratings at either 15 sec, 30 sec, or 60 sec intervals while listening to the samples. Of the reliability indices, intraclass correlations were significantly higher for sophisticated judges although the consistency and agreement of unsophisticated judges were generally equivalent to that of sophisticated judges. Both agreement scores and intraclass correlations were higher when ratings were made at 60 sec rather than 30 sec intervals. The predominant variable that influenced judgment reliability appeared to be differences among the subjects. The methodology partially replicated Martin, Haroldson, and Triden’s (1984) initial investigation on the use of this scale. However, the levels of intra- or interjudge reliability in this study were lower than the levels achieved by Martin et al.’s judges. There were important differences between the Martin et al. study and this one that may account for the findings, and these are discussed.

Acknowledgments
The authors express their appreciation to Joan Rosenthal, Deborah Veltman, Ginnie Black, Phil Sorbello, Annette Uebergang, and Peter Frank for their contributions at various stages in the conduct of this study. This research was supported in part by the National Institute of Deafness and Other Communication Disorders, Grant No. 1 ROI DC00060-01A1, and by a grant from the Quota Club of Camden, Sydney, Australia.
Order a Subscription
Pay Per View
Entire Journal of Speech, Language, and Hearing Research content & archive
24-hour access
This Article
24-hour access