HSK 845 - CCRAM: Systematic Review & Meta-Analysis

Haskayne Executive Education

Course Description

Systematic Review & Meta-Analysis Course Description

One of the greatest scientific challenges and opportunities in the information age is making sense and making use of a vast sea of scholarly findings. Every time we write an introduction or acquaint or reacquaint ourselves with a field of study, we have to come to grips with the tsunami of articles that are all at our fingertips. By some estimates, scientific output doubles every nine years or so, but the hours in our days remains a constant twenty-four. Consequently, we all struggle with finding the relevant articles for our theses, introductions, meta-analyses and systematic reviews. Furthermore, we are expected often to search multiple databases, readily deduplicate them, keep them up to date, and do so all with transparency and replicability.

Once a set of articles that are relevant to one’s reviews have been relocated, the next challenge is making sense of the findings, which always vary from study to study. Reasons for such cross-study variation include differences in measurement and experimental procedures, stimuli, and rigor, populations sampled, time of study, and so forth. Meta-analysis is used to quantitatively extract the signal from the noise in this study-to-study variation in results as well as find systematic and reliable moderators of cross-study variation.

This course provides a theoretical and practical introduction to systematic reviews and meta-analysis. Using the latest machine learning algorithms incorporated into a cloud-based online platform, we can vastly reduce the time required to conduct a search and collect the data needed for a meta-analysis. Participants will learn how to conduct effective searches across multiple databases, upload them into the HubMeta platform for automatic deduplication, easily screen them using an advanced refinement of the PRISMA protocols (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), annotate or tag them for later use, train RAs in the process, keep a record of their steps, and readily update their search. Whether the project is an individual’s thesis or a comprehensive meta-analysis with a large, international team, this course will make the process radically more replicable and easier. Time reduced can be over 90%, making this a necessity for anyone with limited resources.

Participants can choose just to focus on the literature review element of the course, which includes search, screening and organizing. For those who want to extend their learning into meta-analysis, we later include how to reduce coding time by ten-fold by using the latest techniques and resources, including integrated online platforms that draw on optical character recognition, and automatic error detection. We will replicate and critique a previous tier-1 meta-analysis in real time. The goal of this unit of the course is to enable participants to readily publish their own competitive meta-analytic project.

Course Details

LEARNING STATEMENT

In this course, you will learn about the underlying principles and the practical applications of systematic review and meta-analysis.

For the systematic review unit, the topics covered include:

Conducting effective searches
Database selection
Automatic article acquisition
Establishing and training a research team
Defining and conducting a literature search iteratively
Citation chaining
Deduplication and title/abstract screening
Full-text acquisition and screening
Dealing with foreign language articles
Tagging and sorting article for later use

For meta-analysis, the topics covered include:

Creating a taxonomy and connecting measures to constructs
Converting effect sizes to a common metric
The data entry process
Dealing with dependent effect sizes and time series data
Psychometric corrections for meta-analysis
Meta-analytic models and their weighting schemes
Outlier analysis
Publication bias analysis
Fundamental meta-analytic results
Establishing potential moderators
Meta-regression and moderator analysis
Meta-analytic structural equal modeling (MASEM)
Integrating other research designs
Open Science reporting

COURSE DELIVERY AND TUITION

This course is offered online through Zoom. The first unit on systematic reviews will take place on October 28, 2022, from 9.00am to 5.00pm Calgary time. The second unit on meta-analysis will occur on November 4 and 5, 2022, and will meet 9.00am to 5.00pm Calgary time each day. The tuition for this unit of the course is $795 (Canadian dollars) until October 31, after which the tuition increases to $895.

COMPUTING

The course will concentrate on practical application, with focus on data filtering, data entry and analyses in HubMeta and R, though referencing options in other software. Most of the provided statistical materials and examples will involve R code. HubMeta is a cloud-based platform that works best in Chrome.

WHO WILL BENEFIT?

This course will be helpful for both introductory and advanced researchers in any field who are focusing on systematic review as well as mean-based or correlation-based meta-analysis. Experimental meta-analysis, though related, will be addressed only indirectly. Consequently, this course is relevant to researchers from most fields (e.g., psychology, management, sociology, education, human development, social work, public health, communication). Learners will ideally be comfortable with introductory statistics, including regression, and be able to read as well as understand the method section of basic articles in their field (i.e., necessary for data entry). Proficiency in R is desirable but not necessary. Learners should have access to academic search engines.

LEARNING OUTCOMES

Upon completing this course, you will

be able to effectively construct search terms
be able to train and supervise RAs for data screening
configure machine learning algorithms for data screening
effectively organize literature searches for later write-up
readily meet modern systematic review requirements, such as transparency and replicability
a systematic review in an efficient, transparent, and replicable format
be able to produce a competitive tier-1 meta-analytic dataset, method section and framework
tackle meta-analytic topics an order of magnitude larger than previously plausible as well as in a fraction of the time, including comprehensive meta-analytic correlation matrices
improve your management of a dispersed international meta-analytic research team
address journals’ theory requirements through meta-regression and MASEM
understand the fundamentals of meta-analysis as well as the basic ideas behind more advanced techniques (e.g., One Stage MASEM)

Notes

INSTRUCTOR BIO

Piers Steel received his Ph.D. in Industrial and Organization Psychology from the University of Minnesota. He is Professor and the Brookfield Research Chair at the Haskayne School of Business at the University of Calgary. Piers’ particular areas of research interest include culture, motivation and decision-making, and he also has expertise in systematic review and meta-analysis and is a member of the Society of Research Synthesis and Methodology. He has published several methodology papers on how to improve meta-analysis and is a cofounder of the online meta-analytic platforms HubMeta and metaBUS. Piers’ work has appeared in such places as the Journal of Personality and Social Psychology, Psychological Bulletin, and Personality and Social Psychology Review, Journal of Applied Psychology, Personnel Psychology and Academy of Management Review, among others. He is a fellow of the American Psychological Association, the Society of Industrial Organizational Psychology, and the American Psychological Society. His meta-analytic work has been reported globally in thousands of news articles and produced one best selling book. More details about Dr. Steel and his work can be found at:https://haskayne.ucalgary.ca/haskayne_info/profiles/piers-steel