A cautionary note on the finite sample behavior of maximal reliability
Aguirre-Urreta, M. I., Rönkkö, M., & McIntosh, C. N. (2019). A cautionary note on the finite sample behavior of maximal reliability. Psychological Methods, 24(2), 236-252. https://doi.org/10.1037/met0000176
Published in
Psychological MethodsDate
2019Copyright
© American Psychological Association, 2018.
Several calls have been made for replacing coefficient α with more contemporary model-based reliability coefficients in psychological research. Under the assumption of unidimensional measurement scales and independent measurement errors, two leading alternatives are composite reliability and maximal reliability. Of these two, the maximal reliability statistic, or equivalently Hancock's H, has received a significant amount of attention in recent years. The difference between composite reliability and maximal reliability is that the former is a reliability index for a scale mean (or unweighted sum), whereas the latter estimates the reliability of a scale score where indicators are weighted differently based on their estimated reliabilities. The formula for the maximal reliability weights has been derived using population quantities; however, their finite-sample behavior has not been extensively examined. Particularly, there are two types of bias when the maximal reliability statistic is calculated from sample data: (a) the sample maximal reliability estimator is a positively biased estimator of population maximal reliability, and (b) the true reliability of composites formed with maximal reliability weights calculated from sample data is on average less than the population reliability. Both effects are more pronounced in small-sample scenarios (e.g., <100). We also demonstrate that the composite reliability estimator for equally weighted composite exhibits substantially less bias, which makes it a more appropriate choice for the small-sample case.
...
Publisher
American Psychological AssociationISSN Search the Publication Forum
1082-989XKeywords
Publication in research information system
https://converis.jyu.fi/converis/portal/detail/Publication/28231311
Metadata
Show full item recordCollections
Related funder(s)
Research Council of FinlandFunding program(s)
Postdoctoral Researcher, AoFAdditional information about funding
This research was supported in part by a grant from the Academy of Finland (grant 311309). Parts of this research were carried out while Mikko Rönkkö was at the Department of Industrial Engineering and Management, Aalto University. We acknowledge the computational resources provided by the Aalto Science-IT project.License
Related items
Showing items with similar title or keywords.
-
Test-retest reliability of survey items on ownership and use of physical activity trackers
Ng, Kwok W.; Badura, Petr; Dzielska, Anna; Kokko, Sami; Woods, Catherine B.; Hamrik, Zdenek (Palacký University Olomouc, 2019)Background: Researchers are interested in using digital tools, such as physical activity trackers (PAT), to promote and measure physical activity. PAT is a fast-growing research area that measures movement therefore, ... -
Reliability and Validity Study of the Finnish Adaptation of Scoliosis Research Society Questionnaire Version SRS-30
Kyrölä, Kati; Järvenpää, Salme; Ylinen, Jari; Mecklin, Jukka-Pekka; Repo, Jussi Petteri; Häkkinen, Arja (Lippincott Co., 2017)Study Design. A prospective clinical study to test and adapt a Finnish version of the Scoliosis Research Society 30 (SRS-30) questionnaire. Objective. To perform cross-cultural adaptation and evaluate the validity of the ... -
Construct validity and reliability of the physical activity parenting questionnaire for children (PAP-C)
Laukkanen, Arto; Aunola, Kaisa; Korhonen, Elisa; Barnett, Lisa M.; Sääkslahti, Arja (Biomed Central, 2021)Background Children’s perception of parenting is hypothesised to significantly affect their physical activity (PA). This study aimed to examine construct validity, factorial invariance and reliability of a new tool: ... -
Test−Retest Reliability of Isokinetic Ankle, Knee and Hip Strength in Physically Active Adults Using Biodex System 4 Pro
Tuominen, Juho; Leppänen, Mari; Jarske, Heidi; Pasanen, Kati; Vasankari, Tommi; Parkkari, Jari (MDPI AG, 2023)Background: The isokinetic dynamometry is considered a gold standard in muscle strength testing. The reliability of lower limb isokinetic strength measurements has not been thoroughly evaluated. Objective: To examine the ... -
Validity and reliability of the motivation for physical activity (RM4-FM) questionnaire
Uimonen, Mikko; Repo, Jussi P.; Grönroos, Kiira; Häkkinen, Arja; Walker, Simon (Korean Society of Exercise Rehabilitation, 2021)There is a lack of validated instruments measuring motivation for physical activity (RM4-FM) in the Finnish language. The study examined the translated RM4-FM instrument’s psychometric properties in a sample of healthy, ...