Too Small to Succeed : Small Samples and the p-Value Problem
Aguirre-Urreta, M. I., Rönkkö, M., & McIntosh, C. N. (2024). Too Small to Succeed : Small Samples and the p-Value Problem. Data Base for Advances in Information Systems, 55(3), 12-49. https://doi.org/10.1145/3685235.3685238
Published in
Data Base for Advances in Information SystemsDate
2024Copyright
© 2024 ACM
Determining an appropriate sample size is a critical planning decision in quantitative empirical research. In recent years, there has been a growing concern that researchers have excessively focused on statistical significance in large sample studies to the detriment of effect sizes. This research focuses on a related concern at the other end of the spectrum. We argue that a combination of bias in significant estimates obtained from small samples (compared to their population values) and an editorial preference for the publication of significant results compound to produce marked bias in published small sample studies. We then present a simulation study covering a variety of statistical techniques commonly used to examine structural equation models with latent variables. Our results support our contention that significant results obtained from small samples are likely biased and should be considered with skepticism. We also argue for the need to provide a priori power analyses to understand the behavior of parameter estimates under the small sample conditions we examine.
...
![showless](/themes/JYX2//images/showless.png)
![showmore](/themes/JYX2//images/showmore.png)
Publisher
ACMISSN Search the Publication Forum
1532-0936Keywords
Publication in research information system
https://converis.jyu.fi/converis/portal/detail/Publication/216083820
Metadata
Show full item recordCollections
- Kauppakorkeakoulu [1330]
Related funder(s)
Research Council of FinlandFunding program(s)
Postdoctoral Researcher, AoFAdditional information about funding
Mikko Rönkkö acknowledges the Academy of Finland grant number 311309.License
Related items
Showing items with similar title or keywords.
-
Premature conclusions about the signal‐to‐noise ratio in structural equation modeling research : A commentary on Yuan and Fang (2023)
Schuberth, Florian; Schamberger, Tamara; Rönkkö, Mikko; Liu, Yide; Henseler, Jörg (John Wiley & Sons, 2023)In a recent article published in this journal, Yuan and Fang (British Journal of Mathematical and Statistical Psychology, 2023) suggest comparing structural equation modeling (SEM), also known as covariance-based SEM ... -
Rejoinder to Comments on Recent Developments in PLS
Evermann, Jöerg; Rönkkö, Mikko (Association for Information Systems, 2023)When we were first invited to write an essay on the use of PLS for CAIS, we wanted to focus on recent developments to help applied IS researchers, and the CAIS community of authors, reviewers, and editors make use of the ... -
Recent Developments in PLS
Evermann, Jöerg; Rönkkö, Mikko (Association for Information Systems, 2023)Partial least squares (PLS) path modeling is a widely used method in the information systems (IS) discipline for estimating linear structural equation models. At the same time, researchers have debated its relative merits ... -
Structural Parameters under Partial Least Squares and Covariance-Based Structural Equation Modeling : A Comment on Yuan and Deng (2021)
Schuberth, Florian; Rosseel, Yves; Rönkkö, Mikko; Trinchera, Laura; Kline, Rex B.; Henseler, Jörg (Routledge, 2023)In their article, Yuan and Deng argue that a structural parameter under partial least squares structural equation modeling (PLS-SEM) is zero if and only if the same structural parameter is zero under covariance-based ... -
Comparison of three ordinal logistic regression methods for predicting person’s self-assessed health status with functional, haemodynamic covariates
Markkanen, Merri-Lotta (2023)Lääketieteen parissa perinteiset kyselytutkimukset ovat yhä suosittuja, jonka myötä myös järjestysasteikollisten muuttujien analyysia suoritetaan paljon. Modernin teknologian kehittyminen näkyy kuitenkin myös tällä ...