A cautionary note on the finite sample behavior of maximal reliability
Aguirre-Urreta, M. I., Rönkkö, M., & McIntosh, C. N. (2019). A cautionary note on the finite sample behavior of maximal reliability. Psychological Methods, 24 (2), 236-252. doi:10.1037/met0000176
Published inPsychological Methods
© American Psychological Association, 2018.
Several calls have been made for replacing coefficient α with more contemporary model-based reliability coefficients in psychological research. Under the assumption of unidimensional measurement scales and independent measurement errors, two leading alternatives are composite reliability and maximal reliability. Of these two, the maximal reliability statistic, or equivalently Hancock’s H, has received a significant amount of attention in recent years. The difference between composite reliability and maximal reliability is that the former is a reliability index for a scale mean (or unweighted sum), whereas the latter estimates the reliability of a scale score where indicators are weighted differently based on their estimated reliabilities. The formula for the maximal reliability weights has been derived using population quantities; however, their finite-sample behavior has not been extensively examined. Particularly, there are two types of bias when the maximal reliability statistic is calculated from sample data: (a) the sample maximal reliability estimator is a positively biased estimator of population maximal reliability, and (b) the true reliability of composites formed with maximal reliability weights calculated from sample data is on average less than the population reliability. Both effects are more pronounced in small-sample scenarios (e.g., <100). We also demonstrate that the composite reliability estimator for equally weighted composite exhibits substantially less bias, which makes it a more appropriate choice for the small-sample case. ...
PublisherAmerican Psychological Association
MetadataShow full item record
Showing items with similar title or keywords.
Reliability and validity of a new accelerometer-based device for detecting physical activities and energy expenditure Yang, Yanxiang; Schumann, Moritz; Le, Shenglong; Cheng, Sulin (PeerJ Inc., 2018)Background Objective assessments of sedentary behavior and physical activity (PA) by using accelerometer-based wearable devices are ever expanding, given their importance in the global context of health maintenance. This ...
Liu, Yang; Wang, Mei; Villberg, Jari; Torsheim, Torbjorn; Tynjälä, Jorma; Lv, Yan; Kannas, Lasse (Springer Netherlands, 2012)This study comprises two sub-studies. Study I assessed the test-retest reliability of Family Affluence Scale (FAS II) items among 95 students aged 11 and 15 years old in Beijing. Study II investigated the completion rate ...
Intra- and inter-rater reliability of thoracic spine mobility and posture assessments in subjects with thoracic spine pain Takatalo, Jani; Ylinen, Jari; Pienimäki, Tuomo; Häkkinen, Arja (Biomed Central, 2020)Background: The thoracic spine (TS) has been neglected in the study of the spine despite its essential role in the stability and posture of the entire spinal complex. Therefore, there is an inevitable need to investigate ...
Reliability of upper-limb diaphyseal mineral and soft-tissue measurements using peripheral Quantitative Computed Tomography (pQCT) Jenkins, Mark A.; Hart, Nicolas H.; Rantalainen, Timo; Chivers, Paola; Newton, Robert U.; Nimphius, Sophia (Hylonome Publications, 2018)Objectives: To quantify between-day reliability of upper-body diaphyseal measurements (radius, ulna, humerus) using peripheral Quantitative Computed Tomography (pQCT). - Methods: Fourteen males (age: 25.8±2.3 years,) ...
Uimonen, Mikko; Repo, Jussi P.; Grönroos, Kiira; Häkkinen, Arja; Walker, Simon (Korean Society of Exercise Rehabilitation, 2021)There is a lack of validated instruments measuring motivation for physical activity (RM4-FM) in the Finnish language. The study examined the translated RM4-FM instrument’s psychometric properties in a sample of healthy, ...