A checklist to assess the quality of survey studies in psychology
Protogerou, C., & Hagger, M. S. (2020). A checklist to assess the quality of survey studies in psychology. Methods in Psychology, 3, 100031. https://doi.org/10.1016/j.metip.2020.100031
Published in
Methods in PsychologyDate
2020Copyright
© 2020 the Author(s)
Study quality is emerging as an essential component of evidence syntheses and allows practitioners and policymakers to make informed decisions based on the quality of the evidence reviewed. Study quality is typically assessed by checklists of pre-determined quality criteria. Few study quality checklists have been systematically evaluated, and none have been developed specifically for survey studies in psychology. The present study addresses this evidence gap by developing the quality of survey studies in psychology (Q-SSP) checklist, using an expert-consensus method. An international panel of experts in psychology research and quality assessment (N = 53) evaluated the inclusion and importance of candidate quality items and offered commentary. The resulting checklist was used to evaluate a set of survey studies and inter-rater reliability of checklist scores was computed. A preliminary test of criterion validity of checklist scores was conducted using on a sample of survey studies with ‘known differences’ in study quality verified by experts. Experts exhibited high agreement on inclusion and importance ratings of the candidate items. Minor adjustments were made to the candidate items based on experts' feedback. Inter-rater reliability of study quality scores using the checklist was high. Some evidence for criterion validity of scores using the checklist was obtained. Overall, we provide preliminary data to support the Q-SSP checklist as a potential means to evaluate the quality of survey studies in psychology. We recommend a large-scale study using the Q-SSP checklist to assess study quality in studies with known differences in quality verified by experts.
...
Publisher
ElsevierISSN Search the Publication Forum
2590-2601Keywords
Publication in research information system
https://converis.jyu.fi/converis/portal/detail/Publication/41631723
Metadata
Show full item recordCollections
- Liikuntatieteiden tiedekunta [2895]
Related funder(s)
TEKESAdditional information about funding
Martin Hagger’s contribution was supported by a Finland Distinguished Professor (FiDiPro) award (1801/31/2105) from Business Finland.License
Related items
Showing items with similar title or keywords.
-
A Case For a Study Quality Appraisal in Survey Studies in Psychology
Protogerou, Cleo; Hagger, Martin (Frontiers Research Foundation, 2019)The lack of replication of key effects in psychology has highlighted some fundamental problems with reporting of research findings and methods used (Asendorpf et al., 2013; Open Science Collaboration, 2015). Problems ... -
Upper secondary school students' perceptions of using mobile phone applications in the formal assessment of speaking skills
Jääskeläinen, Katja (2020)Tutkimuksen aiheena on lukion oppilaiden käsitykset älypuhelinsovellusten käytöstä englannin suullisen kielitaidon arvioinnissa. Suullinen kielitaito, sen arviointi ja mobiilioppiminen (Mobile assisted language learning, ... -
Draamakasvatus opettajan työvälineenä peruskoulun alaluokilla : laadullinen survey-tutkimus luokanopettajien draamakasvatuksen näkemyksistä ja käytänteistä
Herttua, Marika; Viitala, Anna (2003) -
Nordic adolescents responding to demanding survey scales in boring contexts : Examining straightlining
Aarø, Leif Edvard; Fismen, Anne‐Siri; Wold, Bente; Skogen, Jens Christoffer; Torsheim, Torbjørn; Arnarsson, Ársæll Már; Lyyra, Nelli; Löfstedt, Petra; Eriksson, Charli (John Wiley & Sons, 2022)Introduction Straightlining, or identical responses across all items within a multi-item scale, is often taken as an indication that responses to all items in a questionnaire are of poor quality. The purpose of this study ... -
Quality of environmental impact assessment in Finland
Jalava, Kimmo (University of Jyväskylä, 2014)