The development of a new generic risk-of-bias measure for systematic reviews of surveys
Author(s) / Creator(s)
Nudelman, Gabriel
Otto, Kathleen
Abstract / Description
It is important to evaluate risk of bias of the primary studies included in systematic reviews and meta-analyses. Since tools pertinent to surveys are scarce, the goal of the current research was to develop a measure to address this need. In Study 1, an initial list of 10 relevant topics was compiled from previous measures. In Study 2, the list was refined into an eight-item risk-of-bias measure via discussion and a pilot study. In Study 3, experienced researchers used the measure to asses 70 studies, demonstrating high interrater agreement (weighted Kappa = .82). Inexperienced raters also utilized the measure to code 26 different studies included in a prior meta-analysis, which resulted in adequate interrater agreement (weighted Kappa = .64) and excellent convergent validity (r = .66). Thus, the new measure, designed to be accessible and flexible, can increase standardization of risk-of-bias evaluations and contribute to the interpretation of systematic reviews and meta-analytic findings.
Keyword(s)
risk of bias systematic review meta-analysis survey quality correlational studiesPersistent Identifier
Date of first publication
2020-12-22
Journal title
Methodology
Volume
16
Issue
4
Page numbers
278–298
Publisher
PsychOpen GOLD
Publication status
publishedVersion
Review status
peerReviewed
Is version of
Citation
Nudelman, G., & Otto, K. (2020). The development of a new generic risk-of-bias measure for systematic reviews of surveys. Methodology, 16(4), 278-298. https://doi.org/10.5964/meth.4329
-
meth.v16i4.4329.pdfAdobe PDF - 228.29KBMD5: 46f7c398da1add06a5db4df0e5f953f9
-
There are no other versions of this object.
-
Author(s) / Creator(s)Nudelman, Gabriel
-
Author(s) / Creator(s)Otto, Kathleen
-
PsychArchives acquisition timestamp2022-04-14T11:24:47Z
-
Made available on2022-04-14T11:24:47Z
-
Date of first publication2020-12-22
-
Abstract / DescriptionIt is important to evaluate risk of bias of the primary studies included in systematic reviews and meta-analyses. Since tools pertinent to surveys are scarce, the goal of the current research was to develop a measure to address this need. In Study 1, an initial list of 10 relevant topics was compiled from previous measures. In Study 2, the list was refined into an eight-item risk-of-bias measure via discussion and a pilot study. In Study 3, experienced researchers used the measure to asses 70 studies, demonstrating high interrater agreement (weighted Kappa = .82). Inexperienced raters also utilized the measure to code 26 different studies included in a prior meta-analysis, which resulted in adequate interrater agreement (weighted Kappa = .64) and excellent convergent validity (r = .66). Thus, the new measure, designed to be accessible and flexible, can increase standardization of risk-of-bias evaluations and contribute to the interpretation of systematic reviews and meta-analytic findings.en_US
-
Publication statuspublishedVersion
-
Review statuspeerReviewed
-
CitationNudelman, G., & Otto, K. (2020). The development of a new generic risk-of-bias measure for systematic reviews of surveys. Methodology, 16(4), 278-298. https://doi.org/10.5964/meth.4329en_US
-
ISSN1614-2241
-
Persistent Identifierhttps://hdl.handle.net/20.500.12034/5699
-
Persistent Identifierhttps://doi.org/10.23668/psycharchives.6303
-
Language of contenteng
-
PublisherPsychOpen GOLD
-
Is version ofhttps://doi.org/10.5964/meth.4329
-
Is related tohttps://doi.org/10.23668/psycharchives.4415
-
Keyword(s)risk of biasen_US
-
Keyword(s)systematic reviewen_US
-
Keyword(s)meta-analysisen_US
-
Keyword(s)survey qualityen_US
-
Keyword(s)correlational studiesen_US
-
Dewey Decimal Classification number(s)150
-
TitleThe development of a new generic risk-of-bias measure for systematic reviews of surveysen_US
-
DRO typearticle
-
Issue4
-
Journal titleMethodology
-
Page numbers278–298
-
Volume16
-
Visible tag(s)Version of Recorden_US