HSK 845 - CCRAM: Systematic Review & Meta-Analysis
Course Description
Systematic Review & Meta-Analysis Course Description
One of the greatest scientific challenges and opportunities in the information age is making sense and making use of a vast sea of scholarly findings. Every time we write an introduction or acquaint or reacquaint ourselves with a field of study, we have to come to grips with the tsunami of articles that are all at our fingertips. By some estimates, scientific output doubles every nine years or so, but the hours in our days remains a constant twenty-four. Consequently, we all struggle with finding the relevant articles for our theses, introductions, meta-analyses and systematic reviews. Furthermore, we are expected often to search multiple databases, readily deduplicate them, keep them up to date, and do so all with transparency and replicability.
Once a set of articles that are relevant to one’s reviews have been relocated, the next challenge is making sense of the findings, which always vary from study to study. Reasons for such cross-study variation include differences in measurement and experimental procedures, stimuli, and rigor, populations sampled, time of study, and so forth. Meta-analysis is used to quantitatively extract the signal from the noise in this study-to-study variation in results as well as find systematic and reliable moderators of cross-study variation.
This course provides a theoretical and practical introduction to systematic reviews and meta-analysis. Using the latest machine learning algorithms incorporated into a cloud-based online platform, we can vastly reduce the time required to conduct a search and collect the data needed for a meta-analysis. Participants will learn how to conduct effective searches across multiple databases, upload them into the HubMeta platform for automatic deduplication, easily screen them using an advanced refinement of the PRISMA protocols (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), annotate or tag them for later use, train RAs in the process, keep a record of their steps, and readily update their search. Whether the project is an individual’s thesis or a comprehensive meta-analysis with a large, international team, this course will make the process radically more replicable and easier. Time reduced can be over 90%, making this a necessity for anyone with limited resources.
Participants can choose just to focus on the literature review element of the course, which includes search, screening and organizing. For those who want to extend their learning into meta-analysis, we later include how to reduce coding time by ten-fold by using the latest techniques and resources, including integrated online platforms that draw on optical character recognition, and automatic error detection. We will replicate and critique a previous tier-1 meta-analysis in real time. The goal of this unit of the course is to enable participants to readily publish their own competitive meta-analytic project.
Course Details
LEARNING STATEMENT
In this course, you will learn about the underlying principles and the practical applications of systematic review and meta-analysis.
For the systematic review unit, the topics covered include:
- Conducting effective searches
- Database selection
- Automatic article acquisition
- Establishing and training a research team
- Defining and conducting a literature search iteratively
- Citation chaining
- Deduplication and title/abstract screening
- Full-text acquisition and screening
- Dealing with foreign language articles
- Tagging and sorting article for later use
For meta-analysis, the topics covered include:
- Creating a taxonomy and connecting measures to constructs
- Converting effect sizes to a common metric
- The data entry process
- Dealing with dependent effect sizes and time series data
- Psychometric corrections for meta-analysis
- Meta-analytic models and their weighting schemes
- Outlier analysis
- Publication bias analysis
- Fundamental meta-analytic results
- Establishing potential moderators
- Meta-regression and moderator analysis
- Meta-analytic structural equal modeling (MASEM)
- Integrating other research designs
- Open Science reporting
COURSE DELIVERY AND TUITION
This course is offered online through Zoom. The first unit on systematic reviews will take place on October 28, 2022, from 9.00am to 5.00pm Calgary time. The second unit on meta-analysis will occur on November 4 and 5, 2022, and will meet 9.00am to 5.00pm Calgary time each day. The tuition for this unit of the course is $795 (Canadian dollars) until October 31, after which the tuition increases to $895.
COMPUTING
The course will concentrate on practical application, with focus on data filtering, data entry and analyses in HubMeta and R, though referencing options in other software. Most of the provided statistical materials and examples will involve R code. HubMeta is a cloud-based platform that works best in Chrome.
WHO WILL BENEFIT?
This course will be helpful for both introductory and advanced researchers in any field who are focusing on systematic review as well as mean-based or correlation-based meta-analysis. Experimental meta-analysis, though related, will be addressed only indirectly. Consequently, this course is relevant to researchers from most fields (e.g., psychology, management, sociology, education, human development, social work, public health, communication). Learners will ideally be comfortable with introductory statistics, including regression, and be able to read as well as understand the method section of basic articles in their field (i.e., necessary for data entry). Proficiency in R is desirable but not necessary. Learners should have access to academic search engines.
LEARNING OUTCOMES
Upon completing this course, you will
- be able to effectively construct search terms
- be able to train and supervise RAs for data screening
- configure machine learning algorithms for data screening
- effectively organize literature searches for later write-up
- readily meet modern systematic review requirements, such as transparency and replicability
- a systematic review in an efficient, transparent, and replicable format
- be able to produce a competitive tier-1 meta-analytic dataset, method section and framework
- tackle meta-analytic topics an order of magnitude larger than previously plausible as well as in a fraction of the time, including comprehensive meta-analytic correlation matrices
- improve your management of a dispersed international meta-analytic research team
- address journals’ theory requirements through meta-regression and MASEM
- understand the fundamentals of meta-analysis as well as the basic ideas behind more advanced techniques (e.g., One Stage MASEM)
Notes
INSTRUCTOR BIO
Piers Steel received his Ph.D. in Industrial and Organization Psychology from the University of Minnesota. He is Professor and the Brookfield Research Chair at the Haskayne School of Business at the University of Calgary. Piers’ particular areas of research interest include culture, motivation and decision-making, and he also has expertise in systematic review and meta-analysis and is a member of the Society of Research Synthesis and Methodology. He has published several methodology papers on how to improve meta-analysis and is a cofounder of the online meta-analytic platforms HubMeta and metaBUS. Piers’ work has appeared in such places as the Journal of Personality and Social Psychology, Psychological Bulletin, and Personality and Social Psychology Review, Journal of Applied Psychology, Personnel Psychology and Academy of Management Review, among others. He is a fellow of the American Psychological Association, the Society of Industrial Organizational Psychology, and the American Psychological Society. His meta-analytic work has been reported globally in thousands of news articles and produced one best selling book. More details about Dr. Steel and his work can be found at:https://haskayne.ucalgary.ca/haskayne_info/profiles/piers-steel