A Potential Bias in Subjective Ratings of Mental Effort Purpose Subjective reports of listening effort are frequently inconsistent with behavioral and physiological findings. A potential explanation is that participants unwittingly substitute an easier question when faced with a judgment that requires computationally expensive analysis (i.e., heuristic response strategies). The purpose of this study was to investigate whether participants substitute ... Research Article
Research Article  |   September 19, 2018
A Potential Bias in Subjective Ratings of Mental Effort
 
Author Affiliations & Notes
  • Travis M. Moore
    Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN
  • Erin M. Picou
    Department of Hearing and Speech Sciences, Vanderbilt University, Nashville, TN
    Department of Hearing and Speech Sciences, Vanderbilt University Medical Center, Nashville, TN
  • Disclosure: The authors have declared that no competing interests existed at the time of publication.
    Disclosure: The authors have declared that no competing interests existed at the time of publication. ×
  • Correspondence to Travis M. Moore: travis.m.moore@vanderbilt.edu
  • Editor-in-Chief: Sean Redmond
    Editor-in-Chief: Sean Redmond×
  • Editor: Daniel Fogerty
    Editor: Daniel Fogerty×
Article Information
Hearing & Speech Perception / Attention, Memory & Executive Functions / Hearing / Research Articles
Research Article   |   September 19, 2018
A Potential Bias in Subjective Ratings of Mental Effort
Journal of Speech, Language, and Hearing Research, September 2018, Vol. 61, 2405-2421. doi:10.1044/2018_JSLHR-H-17-0451
History: Received December 5, 2017 , Revised March 28, 2018 , Accepted May 21, 2018
 
Journal of Speech, Language, and Hearing Research, September 2018, Vol. 61, 2405-2421. doi:10.1044/2018_JSLHR-H-17-0451
History: Received December 5, 2017; Revised March 28, 2018; Accepted May 21, 2018

Purpose Subjective reports of listening effort are frequently inconsistent with behavioral and physiological findings. A potential explanation is that participants unwittingly substitute an easier question when faced with a judgment that requires computationally expensive analysis (i.e., heuristic response strategies). The purpose of this study was to investigate whether participants substitute the question “How did I perform?” when asked “How much effort did that take?”.

Method Participants completed 2 sets of online surveys containing a text-based, multiple-choice synonym task. Expected performance and mental effort were manipulated across sets in 4 experiments, using a visual masking technique shown to correlate with speech-reception-testing in noise. Experiment 1 was designed to yield stable accuracy and differing effort across sets. Experiment 2 elicited differing accuracy and stable effort. Experiments 3 and 4 manipulated accuracy and performance in opposite directions. Participants included 273 adults (aged 19–68 years, M = 38.4 years).

Results Experiment 1 revealed no influence of perceived performance on ratings of effort when accuracy was stable. Experiment 2 showed that ratings of effort differed inversely with ratings of performance (lower performance and increased effort). Experiments 3 and 4 also demonstrated that participants rated effort in a manner inversely related to performance, regardless of the effort inherent in the condition.

Conclusions Participants likely substitute an easier question when asked to rate the multidimensional construct of mental effort. The results presented here suggest that perceived performance can serve as a ready heuristic and may explain the dissociation between subjective measures of listening effort and behavioral and physiological measures.

Acknowledgments
This project was funded in part by Sonova AG (awarded to Erin Picou), and the development of REDCap was supported by the Vanderbilt CTSA Grant UL1 TR000445 from NCATS/NIH. The article contents are the authors' sole responsibility and do not necessarily represent official National Institutes of Health views. The authors would like to thank Todd Ricketts, Rocky Nachalo, and Jake Cantor for their insight and suggestions throughout the project.
Order a Subscription
Pay Per View
Entire Journal of Speech, Language, and Hearing Research content & archive
24-hour access
This Article
24-hour access