An Examination of Distributional Characteristics and Psychometric Properties of Rating Scales Under Different Scale Label Configurations

Abstract

Rating scales (e.g., Likert scales) have become an indispensable tool in the social sciences for measuring latent constructs. Despite their practicality, it is well known that rating scales are sensitive to design choices made during the scale construction process. In the present study, we investigate the effects of different scale labeling configurations on the distributional characteristics and psychometric properties of rating scales by analyzing data from two 7-point rating scales that were altered with respect to their scale labels. Specifically, we investigate the differences in composite score means, variances, reliability, factor structure, and measurement invariance (i.e., metric, scalar, and strict factorial invariance) of two scales that measure divergent constructs (life satisfaction life; academic time management & procrastination). Results indicate that distributional characteristics of rating scales are affected by differences in labeling configurations to varying degrees, however the psychometric properties of the scales are not influenced by changes in labeling. In particular, it was found that strict factorial invariance held, indicating that the scales, although different with respect to scale labels, maintain equal difficulty, discrimination between groups. Implications and recommendations for scale developers and directions for future research are discussed.

Alfonso J. Martinez

Assistant Professor of Psychometrics and Quantitative Psychology

My research interests include generalized latent variable modeling, Bayesian analysis, and computational statistics.