what is a systematic literature review journal

How to Do a Systematic Review: A Best Practice Guide for Conducting and Reporting Narrative Reviews, Meta-Analyses, and Meta-Syntheses

Affiliations.

• 1 Behavioural Science Centre, Stirling Management School, University of Stirling, Stirling FK9 4LA, United Kingdom; email: [email protected].
• 2 Department of Psychological and Behavioural Science, London School of Economics and Political Science, London WC2A 2AE, United Kingdom.
• 3 Department of Statistics, Northwestern University, Evanston, Illinois 60208, USA; email: [email protected].
• PMID: 30089228
• DOI: 10.1146/annurev-psych-010418-102803

Systematic reviews are characterized by a methodical and replicable methodology and presentation. They involve a comprehensive search to locate all relevant published and unpublished work on a subject; a systematic integration of search results; and a critique of the extent, nature, and quality of evidence in relation to a particular research question. The best reviews synthesize studies to draw broad theoretical conclusions about what a literature means, linking theory to evidence and evidence to theory. This guide describes how to plan, conduct, organize, and present a systematic review of quantitative (meta-analysis) or qualitative (narrative review, meta-synthesis) information. We outline core standards and principles and describe commonly encountered problems. Although this guide targets psychological scientists, its high level of abstraction makes it potentially relevant to any subject area or discipline. We argue that systematic reviews are a key methodology for clarifying whether and how research findings replicate and for explaining possible inconsistencies, and we call for researchers to conduct systematic reviews to help elucidate whether there is a replication crisis.

Keywords: evidence; guide; meta-analysis; meta-synthesis; narrative; systematic review; theory.

• Guidelines as Topic
• Meta-Analysis as Topic*
• Publication Bias
• Review Literature as Topic
• Systematic Reviews as Topic*

Systematic Reviews

Aims and scope.

Systematic Reviews encompasses all aspects of the design, conduct and reporting of systematic reviews. The journal publishes high quality systematic review products including systematic review protocols, systematic reviews related to a very broad definition of human health, rapid reviews, updates of already completed systematic reviews, and methods research related to the science of systematic reviews, such as decision modelling. At this time Systematic Reviews does not accept reviews of in vitro studies.  The journal also aims to ensure that the results of all well-conducted systematic reviews are published, regardless of their outcome.

Trending articles

Click here to view which articles have been shared the most in the last month!

• Most accessed

Protocol for a systematic review of the application of the kidney failure risk equation and Oxford classification in estimating prognosis in IgA nephropathy

Authors: Michael Toal, Ruth Fergie, Michael Quinn, Christopher Hill, Ciaran O’Neill and Alexander P. Maxwell

Risk factors and long-term outcomes of oropharyngeal dysphagia in persons with multiple sclerosis: a systematic review protocol

Authors: Zahra Sadeghi, Mohamadreza Afshar, Asefeh Memarian and Heather L. Flowers

Comparative analysis of Cochrane and non-Cochrane reviews over three decades

Authors: Mikkel Zola Andersen, Philine Zeinert, Jacob Rosenberg and Siv Fonnes

Prevalence of thyroid dysfunction and associated factors among adult type 2 diabetes mellitus patients, 2000–2022: a systematic review and meta-analysis

Authors: Rishan Hadgu, Abebaw Worede and Sintayehu Ambachew

Safety and efficacy of intravenous thrombolysis before mechanical thrombectomy in patients with atrial fibrillation

Authors: Qiangji Bao, Xiaodong Huang, Xinting Wu, Hao Huang, Xiaoqiang Zhang and Mingfei Yang

Most recent articles RSS

View all articles

Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement

Authors: David Moher, Larissa Shamseer, Mike Clarke, Davina Ghersi, Alessandro Liberati, Mark Petticrew, Paul Shekelle and Lesley A Stewart

The PRISMA 2020 statement: an updated guideline for reporting systematic reviews

Authors: Matthew J. Page, Joanne E. McKenzie, Patrick M. Bossuyt, Isabelle Boutron, Tammy C. Hoffmann, Cynthia D. Mulrow, Larissa Shamseer, Jennifer M. Tetzlaff, Elie A. Akl, Sue E. Brennan, Roger Chou, Julie Glanville, Jeremy M. Grimshaw, Asbjørn Hróbjartsson, Manoj M. Lalu, Tianjing Li…

The Editorial to this article has been published in Systematic Reviews 2021 10 :117

Optimal database combinations for literature searches in systematic reviews: a prospective exploratory study

Authors: Wichor M. Bramer, Melissa L. Rethlefsen, Jos Kleijnen and Oscar H. Franco

Drug discontinuation before contrast procedures and the effect on acute kidney injury and other clinical outcomes: a systematic review protocol

Authors: Swapnil Hiremath, Jeanne Françoise Kayibanda, Benjamin J. W. Chow, Dean Fergusson, Greg A. Knoll, Wael Shabana, Brianna Lahey, Olivia McBride, Alexandra Davis and Ayub Akbari

PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews

Authors: Melissa L. Rethlefsen, Shona Kirtley, Siw Waffenschmidt, Ana Patricia Ayala, David Moher, Matthew J. Page and Jonathan B. Koffel

Most accessed articles RSS

Sign up to receive article alerts

Systematic Reviews is published continuously online-only. We encourage you to sign up to receive free email alerts to keep up to date with all of the latest articles by registering here .

Peer review mentoring

The Editors endorse peer review mentoring of early career researchers. Find out more here

Article collections

Thematic series The role of systematic reviews in evidence-based research Edited by Professor Dawid Pieper and Professor Hans Lund 

Thematic series Canadian Task Force on Preventive Health Care Evidence Reviews Edited by Dr Ivan Florez

Thematic series Automation in the systematic review process Edited by Prof Joseph Lau

Thematic series Five years of Systematic Reviews

View all article collections

Latest Tweets

Your browser needs to have JavaScript enabled to view this timeline

• Editorial Board
• Manuscript editing services
• Instructions for Editors
• Sign up for article alerts and news from this journal
• Follow us on Twitter

Annual Journal Metrics

2022 Citation Impact 3.7 - 2-year Impact Factor 3.8 - 5-year Impact Factor 1.561 - SNIP (Source Normalized Impact per Paper) 1.269 - SJR (SCImago Journal Rank)

2023 Speed 88 days submission to first editorial decision for all manuscripts (Median) 296 days submission to accept (Median)

2023 Usage  3,531,065 downloads 3,533 Altmetric mentions

• More about our metrics

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

What is a Systematic Literature Review?

A systematic literature review (SLR) is an independent academic method that aims to identify and evaluate all relevant literature on a topic in order to derive conclusions about the question under consideration. "Systematic reviews are undertaken to clarify the state of existing research and the implications that should be drawn from this." (Feak & Swales, 2009, p. 3) An SLR can demonstrate the current state of research on a topic, while identifying gaps and areas requiring further research with regard to a given research question. A formal methodological approach is pursued in order to reduce distortions caused by an overly restrictive selection of the available literature and to increase the reliability of the literature selected (Tranfield, Denyer & Smart, 2003). A special aspect in this regard is the fact that a research objective is defined for the search itself and the criteria for determining what is to be included and excluded are defined prior to conducting the search. The search is mainly performed in electronic literature databases (such as Business Source Complete or Web of Science), but also includes manual searches (reviews of reference lists in relevant sources) and the identification of literature not yet published in order to obtain a comprehensive overview of a research topic.

An SLR protocol documents all the information gathered and the steps taken as part of an SLR in order to make the selection process transparent and reproducible. The PRISMA flow-diagram support you in making the selection process visible.

In an ideal scenario, experts from the respective research discipline, as well as experts working in the relevant field and in libraries, should be involved in setting the search terms . As a rule, the literature is selected by two or more reviewers working independently of one another. Both measures serve the purpose of increasing the objectivity of the literature selection. An SLR must, then, be more than merely a summary of a topic (Briner & Denyer, 2012). As such, it also distinguishes itself from “ordinary” surveys of the available literature. The following table shows the differences between an SLR and an “ordinary” literature review.

• Charts of BSWL workshop (pdf, 2.88 MB)
• Listen to the interview (mp4, 12.35 MB)

Differences to "common" literature reviews

What are the objectives of slrs.

• Avoidance of research redundancies despite a growing amount of publications
• Identification of research areas, gaps and methods
• Input for evidence-based management, which allows to base management decisions on scientific methods and findings
• Identification of links between different areas of researc

Process steps of an SLR

A SLR has several process steps which are defined differently in the literature (Fink 2014, p. 4; Guba 2008, Transfield et al. 2003). We distinguish the following steps which are adapted to the economics and management research area:

1. Defining research questions

Briner & Denyer (2009, p. 347ff.) have developed the CIMO scheme to establish clearly formulated and answerable research questions in the field of economic sciences:

C – CONTEXT:  Which individuals, relationships, institutional frameworks and systems are being investigated?

I – Intervention:  The effects of which event, action or activity are being investigated?

M – Mechanisms:  Which mechanisms can explain the relationship between interventions and results? Under what conditions do these mechanisms take effect?

O – Outcomes:  What are the effects of the intervention? How are the results measured? What are intended and unintended effects?

The objective of the systematic literature review is used to formulate research questions such as “How can a project team be led effectively?”. Since there are numerous interpretations and constructs for “effective”, “leadership” and “project team”, these terms must be particularized.

With the aid of the scheme, the following concrete research questions can be derived with regard to this example:

Under what conditions (C) does leadership style (I) influence the performance of project teams (O)?

Which constructs have an effect upon the influence of leadership style (I) on a project team’s performance (O)?          

Research questions do not necessarily need to follow the CIMO scheme, but they should:

• ... be formulated in a clear, focused and comprehensible manner and be answerable;
• ... have been determined prior to carrying out the SLR;
• ... consist of general and specific questions.

As early as this stage, the criteria for inclusion and exclusion are also defined. The selection of the criteria must be well-grounded. This may include conceptual factors such as a geographical or temporal restrictions, congruent definitions of constructs, as well as quality criteria (journal impact factor > x).

2. Selecting databases and other research sources

The selection of sources must be described and explained in detail. The aim is to find a balance between the relevance of the sources (content-related fit) and the scope of the sources.

In the field of economic sciences, there are a number of literature databases that can be searched as part of an SLR. Some examples in this regard are:

• Business Source Complete
• ProQuest One Business
• Web of Science
• EconBiz        

Our video " Selecting the right databases " explains how to find relevant databases for your topic.

Literature databases are an important source of research for SLRs, as they can minimize distortions caused by an individual literature selection (selection bias), while offering advantages for a systematic search due to their data structure. The aim is to find all database entries on a topic and thus keep the retrieval bias low (tutorial on retrieval bias ).  Besides articles from scientific journals, it is important to inlcude working papers, conference proceedings, etc to reduce the publication bias ( tutorial on publication bias ).

Our online self-study course " Searching economic databases " explains step 2 und 3.

3. Defining search terms

Once the literature databases and other research sources have been selected, search terms are defined. For this purpose, the research topic/questions is/are divided into blocks of terms of equal ranking. This approach is called the block-building method (Guba 2008, p. 63). The so-called document-term matrix, which lists topic blocks and search terms according to a scheme, is helpful in this regard. The aim is to identify as many different synonyms as possible for the partial terms. A precisely formulated research question facilitates the identification of relevant search terms. In addition, keywords from particularly relevant articles support the formulation of search terms.

A document-term matrix for the topic “The influence of management style on the performance of project teams” is shown in this example .

Identification of headwords and keywords

When setting search terms, a distinction must be made between subject headings and keywords, both of which are described below:

• appear in the title, abstract and/or text
• sometimes specified by the author, but in most cases automatically generated
• non-standardized
• different spellings and forms (singular/plural) must be searched separately

Subject headings

• describe the content
• are generated by an editorial team
• are listed in a standardized list (thesaurus)
• may comprise various keywords
• include different spellings
• database-specific

Subject headings are a standardized list of words that are generated by the specialists in charge of some databases. This so-called index of subject headings (thesaurus) helps searchers find relevant articles, since the headwords indicate the content of a publication. By contrast, an ordinary keyword search does not necessarily result in a content-related fit, since the database also displays articles in which, for example, a word appears once in the abstract, even though the article’s content does not cover the topic.

Nevertheless, searches using both headwords and keywords should be conducted, since some articles may not yet have been assigned headwords, or errors may have occurred during the assignment of headwords. 

To add headwords to your search in the Business Source Complete database, please select the Thesaurus tab at the top. Here you can find headwords in a new search field and integrate them into your search query. In the search history, headwords are marked with the addition DE (descriptor).

The EconBiz database of the German National Library of Economics (ZBW – Leibniz Information Centre for Economics), which also contains German-language literature, has created its own index of subject headings with the STW Thesaurus for Economics . Headwords are integrated into the search by being used in the search query.

Since the indexes of subject headings divide terms into synonyms, generic terms and sub-aspects, they facilitate the creation of a document-term matrix. For this purpose it is advisable to specify in the document-term matrix the origin of the search terms (STW Thesaurus for Economics, Business Source Complete, etc.).

Searching in literature databases

Once the document-term matrix has been defined, the search in literature databases begins. It is recommended to enter each word of the document-term matrix individually into the database in order to obtain a good overview of the number of hits per word. Finally, all the words contained in a block of terms are linked with the Boolean operator OR and thereby a union of all the words is formed. The latter are then linked with each other using the Boolean operator AND. In doing so, each block should be added individually in order to see to what degree the number of hits decreases.

Since the search query must be set up separately for each database, tools such as  LitSonar  have been developed to enable a systematic search across different databases. LitSonar was created by  Professor Dr. Ali Sunyaev (Institute of Applied Informatics and Formal Description Methods – AIFB) at the Karlsruhe Institute of Technology.

Advanced search

Certain database-specific commands can be used to refine a search, for example, by taking variable word endings into account (*) or specifying the distance between two words, etc. Our overview shows the most important search commands for our top databases.

Additional searches in sources other than literature databases

In addition to literature databases, other sources should also be searched. Fink (2014, p. 27) lists the following reasons for this:

• the topic is new and not yet included in indexes of subject headings;
• search terms are not used congruently in articles because uniform definitions do not exist;
• some studies are still in the process of being published, or have been completed, but not published.

Therefore, further search strategies are manual search, bibliographic analysis, personal contacts and academic networks (Briner & Denyer, p. 349). Manual search means that you go through the source information of relevant articles and supplement your hit list accordingly. In addition, you should conduct a targeted search for so-called gray literature, that is, literature not distributed via the book trade, such as working papers from specialist areas and conference reports. By including different types of publications, the so-called publication bias (DBWM video “Understanding publication bias” ) – that is, distortions due to exclusive use of articles from peer-reviewed journals – should be kept to a minimum.

The PRESS-Checklist can support you to check the correctness of your search terms.

4. Merging hits from different databases

In principle, large amounts of data can be easily collected, structured and sorted with data processing programs such as Excel. Another option is to use literature management programs such as EndNote, Citavi or Zotero. The Saxon State and University Library Dresden (SLUB Dresden) provides an  overview of current literature management programs  . Software for qualitative data analysis such as NVivo is equally suited for data processing. A comprehensive overview of the features of different tools that support the SLR process can be found in Bandara et al. (2015).

Our online-self study course "Managing literature with Citavi" shows you how to use the reference management software Citavi.

When conducting an SLR, you should specify for each hit the database from which it originates and the date on which the query was made. In addition, you should always indicate how many hits you have identified in the various databases or, for example, by manual search.

Exporting data from literature databases

Exporting from literature databases is very easy. In  Business Source Complete  , you must first click on the “Share” button in the hit list, then “Email a link to download exported results” at the very bottom and then select the appropriate format for the respective literature program.

In the  Web of Science  database, you must select “Export” and select the relevant format. Tip: You can adjust the extracted data fields. Since for example the abstract is not automatically exported, decide which data fields are of interest for you.

Exporting data from the literature database  EconBiz  is somewhat more complex. Here you must first create a marked list and then select each hit individually and add it to the marked list. Afterwards, articles on the list can be exported.

After merging all hits from the various databases, duplicate entries (duplicates) are deleted.

5. Applying inclusion and exclusion criteria

All publications are evaluated in the literature management program applying the previously defined criteria for inclusion and exclusion. Only those sources that survive this selection process will subsequently be analyzed. The review process and inclusion criteria should be tested with a small sample and adjustments made if necessary before applying it to all articles. In the ideal case, even this selection would be carried out by more than one person, with each working independently of one another. It needs to be made clear how discrepancies between reviewers are dealt with. 

The review of the criteria for inclusion and exclusion is primarily based on the title, abstract and subject headings in the databases, as well as on the keywords provided by the authors of a publication in the first step. In a second step the whole article / source will be read.

Within the Citavi literature-management program, you can supplement title data by adding your own fields. In this regard, the criteria for inclusion can be listed individually and marked with 0 in the free text field for being “not fulfilled” and with 1 for being “fulfilled”. In the table view of all titles, you can use the column function to select which columns should be displayed. Here you can include the criteria for inclusion. By exporting the title list to Excel, it is easy to calculate how many titles remain when applying the criteria for inclusion and exclusion.

In addition to the common literature management tools, you can also use software tools that have been developed to support SLRs. The central library of the university in Zurich has published an overview and evaluation of different tools based on a survey among researchers. --> View SLR tools

The selection process needs to be made transparent. The PRISMA flow diagram supports the visualization of the number of included / excluded studies.

Forward and backward search

Should it become apparent that the number of sources found is relatively small, or if you wish to proceed with particular thoroughness, a forward-and-backward search based on the sources found is recommendable (Webster & Watson 2002, p. xvi). A backward search means going through the bibliographies of the sources found. A forward search, by contrast, identifies articles that have cited the relevant publications. The Web of Science and Scopus databases can be used to perform citation analyses.

6. Perform the review

As the next step, the remaining titles are analyzed as to their content by reading them several times in full. Information is extracted according to defined criteria and the quality of the publications is evaluated. If the data extraction is carried out by more than one person, a training ensures that there will be no differences between the reviewers.

Depending on the research questions there exist diffent methods for data abstraction (content analysis, concept matrix etc.). A so-called concept matrix can be used to structure the content of information (Webster & Watson 2002, p. xvii). The image to the right gives an example of a concept matrix according to Becker (2014).

Particularly in the field of economic sciences, the evaluation of a study’s quality cannot be performed according to a generally valid scheme, such as those existing in the field of medicine, for instance. Quality assessment therefore depends largely on the research questions.

Based on the findings of individual studies, a meta-level is then applied to try to understand what similarities and differences exist between the publications, what research gaps exist, etc. This may also result in the development of a theoretical model or reference framework.

Example concept matrix (Becker 2013) on the topic Business Process Management

7. synthesizing results.

Once the review has been conducted, the results must be compiled and, on the basis of these, conclusions derived with regard to the research question (Fink 2014, p. 199ff.). This includes, for example, the following aspects:

• historical development of topics (histogram, time series: when, and how frequently, did publications on the research topic appear?);
• overview of journals, authors or specialist disciplines dealing with the topic;
• comparison of applied statistical methods;
• topics covered by research;
• identifying research gaps;
• developing a reference framework;
• developing constructs;
• performing a meta-analysis: comparison of the correlations of the results of different empirical studies (see for example Fink 2014, p. 203 on conducting meta-analyses)

Publications about the method

Bandara, W., Furtmueller, E., Miskon, S., Gorbacheva, E., & Beekhuyzen, J. (2015). Achieving Rigor in Literature Reviews: Insights from Qualitative Data Analysis and Tool-Support.  Communications of the Association for Information Systems . 34(8), 154-204.

Booth, A., Papaioannou, D., and Sutton, A. (2012)  Systematic approaches to a successful literature review.  London: Sage.

Briner, R. B., & Denyer, D. (2012). Systematic Review and Evidence Synthesis as a Practice and Scholarship Tool. In Rousseau, D. M. (Hrsg.),  The Oxford Handbook of Evidenence Based Management . (S. 112-129). Oxford: Oxford University Press.

Durach, C. F., Wieland, A., & Machuca, Jose A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review . International Journal of Physical Distribution & Logistic Management , 46 (1/2), 118-137. doi:  https://doi.org/10.1108/IJPDLM-05-2013-0133

Feak, C. B., & Swales, J. M. (2009). Telling a Research Story: Writing a Literature Review.  English in Today's Research World 2.  Ann Arbor: University of Michigan Press. doi:  10.3998/mpub.309338

Fink, A. (2014).  Conducting Research Literature Reviews: From the Internet to Paper  (4. Aufl.). Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Fisch, C., & Block, J. (2018). Six tips for your (systematic) literature review in business and management research.  Management Review Quarterly,  68, 103–106 (2018).  doi.org/10.1007/s11301-018-0142-x

Guba, B. (2008). Systematische Literaturrecherche.  Wiener Medizinische Wochenschrift , 158 (1-2), S. 62-69. doi:  doi.org/10.1007/s10354-007-0500-0  Hart, C.  Doing a literature review: releasing the social science research imagination.  London: Sage.

Jesson, J. K., Metheson, L. & Lacey, F. (2011).  Doing your Literature Review - traditional and Systematic Techniques . Los Angeles, London, New Delhi, Singapore, Washington DC: Sage Publication.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71.

Petticrew, M. and Roberts, H. (2006).  Systematic Reviews in the Social Sciences: A Practical Guide . Oxford:Blackwell. Ridley, D. (2012).  The literature review: A step-by-step guide . 2nd edn. London: Sage. 

Chang, W. and Taylor, S.A. (2016), The Effectiveness of Customer Participation in New Product Development: A Meta-Analysis,  Journal of Marketing , American Marketing Association, Los Angeles, CA, Vol. 80 No. 1, pp. 47–64.

Tranfield, D., Denyer, D. & Smart, P. (2003). Towards a methodology for developing evidence-informed management knowledge by means of systematic review.  British Journal of Management , 14 (3), S. 207-222. doi:  https://doi.org/10.1111/1467-8551.00375

Webster, J., & Watson, R. T. (2002). Analyzing the Past to Prepare for the Future: Writing a Literature Review.  Management Information Systems Quarterly , 26(2), xiii-xxiii.  http://www.jstor.org/stable/4132319

Durach, C. F., Wieland, A. & Machuca, Jose. A. D. (2015). Antecedents and dimensions of supply chain robustness: a systematic literature review. International Journal of Physical Distribution & Logistics Management, 45(1/2), 118 – 137.

What is particularly good about this example is that search terms were defined by a number of experts and the review was conducted by three researchers working independently of one another. Furthermore, the search terms used have been very well extracted and the procedure of the literature selection very well described.

On the downside, the restriction to English-language literature brings the language bias into play, even though the authors consider it to be insignificant for the subject area.

Bos-Nehles, A., Renkema, M. & Janssen, M. (2017). HRM and innovative work behaviour: a systematic literature review. Personnel Review, 46(7), pp. 1228-1253

• Only very specific keywords used
• No precise information on how the review process was carried out (who reviewed articles?)
• Only journals with impact factor (publication bias)

Jia, F., Orzes, G., Sartor, M. & Nassimbeni, G. (2017). Global sourcing strategy and structure: towards a conceptual framework. International Journal of Operations & Production Management, 37(7), 840-864

• Research questions are explicitly presented
• Search string very detailed
• Exact description of the review process
• 2 persons conducted the review independently of each other

Franziska Klatt

[email protected]

+49 30 314-29778

Privacy notice: The TU Berlin offers a chat information service. If you enable it, your IP address and chat messages will be transmitted to external EU servers. more information

The chat is currently unavailable.

Please use our alternative contact options.

• Research Process

Systematic Literature Review or Literature Review?

• 3 minute read
• 44.6K views

Table of Contents

As a researcher, you may be required to conduct a literature review. But what kind of review do you need to complete? Is it a systematic literature review or a standard literature review? In this article, we’ll outline the purpose of a systematic literature review, the difference between literature review and systematic review, and other important aspects of systematic literature reviews.

What is a Systematic Literature Review?

The purpose of systematic literature reviews is simple. Essentially, it is to provide a high-level of a particular research question. This question, in and of itself, is highly focused to match the review of the literature related to the topic at hand. For example, a focused question related to medical or clinical outcomes.

The components of a systematic literature review are quite different from the standard literature review research theses that most of us are used to (more on this below). And because of the specificity of the research question, typically a systematic literature review involves more than one primary author. There’s more work related to a systematic literature review, so it makes sense to divide the work among two or three (or even more) researchers.

Your systematic literature review will follow very clear and defined protocols that are decided on prior to any review. This involves extensive planning, and a deliberately designed search strategy that is in tune with the specific research question. Every aspect of a systematic literature review, including the research protocols, which databases are used, and dates of each search, must be transparent so that other researchers can be assured that the systematic literature review is comprehensive and focused.

Most systematic literature reviews originated in the world of medicine science. Now, they also include any evidence-based research questions. In addition to the focus and transparency of these types of reviews, additional aspects of a quality systematic literature review includes:

• Clear and concise review and summary
• Comprehensive coverage of the topic
• Accessibility and equality of the research reviewed

Systematic Review vs Literature Review

The difference between literature review and systematic review comes back to the initial research question. Whereas the systematic review is very specific and focused, the standard literature review is much more general. The components of a literature review, for example, are similar to any other research paper. That is, it includes an introduction, description of the methods used, a discussion and conclusion, as well as a reference list or bibliography.

A systematic review, however, includes entirely different components that reflect the specificity of its research question, and the requirement for transparency and inclusion. For instance, the systematic review will include:

• Eligibility criteria for included research
• A description of the systematic research search strategy
• An assessment of the validity of reviewed research
• Interpretations of the results of research included in the review

As you can see, contrary to the general overview or summary of a topic, the systematic literature review includes much more detail and work to compile than a standard literature review. Indeed, it can take years to conduct and write a systematic literature review. But the information that practitioners and other researchers can glean from a systematic literature review is, by its very nature, exceptionally valuable.

This is not to diminish the value of the standard literature review. The importance of literature reviews in research writing is discussed in this article . It’s just that the two types of research reviews answer different questions, and, therefore, have different purposes and roles in the world of research and evidence-based writing.

Systematic Literature Review vs Meta Analysis

It would be understandable to think that a systematic literature review is similar to a meta analysis. But, whereas a systematic review can include several research studies to answer a specific question, typically a meta analysis includes a comparison of different studies to suss out any inconsistencies or discrepancies. For more about this topic, check out Systematic Review VS Meta-Analysis article.

Language Editing Plus

With Elsevier’s Language Editing Plus services , you can relax with our complete language review of your systematic literature review or literature review, or any other type of manuscript or scientific presentation. Our editors are PhD or PhD candidates, who are native-English speakers. Language Editing Plus includes checking the logic and flow of your manuscript, reference checks, formatting in accordance to your chosen journal and even a custom cover letter. Our most comprehensive editing package, Language Editing Plus also includes any English-editing needs for up to 180 days.

• Publication Recognition

How to Make a PowerPoint Presentation of Your Research Paper

• Manuscript Preparation

What is and How to Write a Good Hypothesis in Research?

You may also like.

Descriptive Research Design and Its Myriad Uses

Five Common Mistakes to Avoid When Writing a Biomedical Research Paper

Making Technical Writing in Environmental Engineering Accessible

To Err is Not Human: The Dangers of AI-assisted Academic Writing

When Data Speak, Listen: Importance of Data Collection and Analysis Methods

Choosing the Right Research Methodology: A Guide for Researchers

Why is data validation important in research?

Writing a good review article

Input your search keywords and press Enter.

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Efficacy of psilocybin...

Efficacy of psilocybin for treating symptoms of depression: systematic review and meta-analysis

Linked editorial.

Psilocybin for depression

• Related content
• Peer review

This article has a correction. Please see:

• EXPRESSION OF CONCERN: Efficacy of psilocybin for treating symptoms of depression: systematic review and meta-analysis - May 04, 2024
• Athina-Marina Metaxa , masters graduate researcher 1 ,
• Mike Clarke , professor 2
• 1 Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford OX2 6GG, UK
• 2 Northern Ireland Methodology Hub, Centre for Public Health, ICS-A Royal Hospitals, Belfast, Ireland, UK
• Correspondence to: A-M Metaxa athina.metaxa{at}hmc.ox.ac.uk (or @Athina_Metaxa12 on X)
• Accepted 6 March 2024

Objective To determine the efficacy of psilocybin as an antidepressant compared with placebo or non-psychoactive drugs.

Design Systematic review and meta-analysis.

Data sources Five electronic databases of published literature (Cochrane Central Register of Controlled Trials, Medline, Embase, Science Citation Index and Conference Proceedings Citation Index, and PsycInfo) and four databases of unpublished and international literature (ClinicalTrials.gov, WHO International Clinical Trials Registry Platform, ProQuest Dissertations and Theses Global, and PsycEXTRA), and handsearching of reference lists, conference proceedings, and abstracts.

Data synthesis and study quality Information on potential treatment effect moderators was extracted, including depression type (primary or secondary), previous use of psychedelics, psilocybin dosage, type of outcome measure (clinician rated or self-reported), and personal characteristics (eg, age, sex). Data were synthesised using a random effects meta-analysis model, and observed heterogeneity and the effect of covariates were investigated with subgroup analyses and metaregression. Hedges’ g was used as a measure of treatment effect size, to account for small sample effects and substantial differences between the included studies’ sample sizes. Study quality was appraised using Cochrane’s Risk of Bias 2 tool, and the quality of the aggregated evidence was evaluated using GRADE guidelines.

Eligibility criteria Randomised trials in which psilocybin was administered as a standalone treatment for adults with clinically significant symptoms of depression and change in symptoms was measured using a validated clinician rated or self-report scale. Studies with directive psychotherapy were included if the psychotherapeutic component was present in both experimental and control conditions. Participants with depression regardless of comorbidities (eg, cancer) were eligible.

Results Meta-analysis on 436 participants (228 female participants), average age 36-60 years, from seven of the nine included studies showed a significant benefit of psilocybin (Hedges’ g=1.64, 95% confidence interval (CI) 0.55 to 2.73, P<0.001) on change in depression scores compared with comparator treatment. Subgroup analyses and metaregressions indicated that having secondary depression (Hedges’ g=3.25, 95% CI 0.97 to 5.53), being assessed with self-report depression scales such as the Beck depression inventory (3.25, 0.97 to 5.53), and older age and previous use of psychedelics (metaregression coefficient 0.16, 95% CI 0.08 to 0.24 and 4.2, 1.5 to 6.9, respectively) were correlated with greater improvements in symptoms. All studies had a low risk of bias, but the change from baseline metric was associated with high heterogeneity and a statistically significant risk of small study bias, resulting in a low certainty of evidence rating.

Conclusion Treatment effects of psilocybin were significantly larger among patients with secondary depression, when self-report scales were used to measure symptoms of depression, and when participants had previously used psychedelics. Further research is thus required to delineate the influence of expectancy effects, moderating factors, and treatment delivery on the efficacy of psilocybin as an antidepressant.

Systematic review registration PROSPERO CRD42023388065.

• Download figure
• Open in new tab
• Download powerpoint

Introduction

Depression affects an estimated 300 million people around the world, an increase of nearly 20% over the past decade. 1 Worldwide, depression is also the leading cause of disability. 2

Drugs for depression are widely available but these seem to have limited efficacy, can have serious adverse effects, and are associated with low patient adherence. 3 4 Importantly, the treatment effects of antidepressant drugs do not appear until 4-7 weeks after the start of treatment, and remission of symptoms can take months. 4 5 Additionally, the likelihood of relapse is high, with 40-60% of people with depression experiencing a further depressive episode, and the chance of relapse increasing with each subsequent episode. 6 7

Since the early 2000s, the naturally occurring serotonergic hallucinogen psilocybin, found in several species of mushrooms, has been widely discussed as a potential treatment for depression. 8 9 Psilocybin’s mechanism of action differs from that of classic selective serotonin reuptake inhibitors (SSRIs) and might improve the treatment response rate, decrease time to improvement of symptoms, and prevent relapse post-remission. Moreover, more recent assessments of harm have consistently reported that psilocybin generally has low addictive potential and toxicity and that it can be administered safely under clinical supervision. 10

The renewed interest in psilocybin’s antidepressive effects led to several clinical trials on treatment resistant depression, 11 12 major depressive disorder, 13 and depression related to physical illness. 14 15 16 17 These trials mostly reported positive efficacy findings, showing reductions in symptoms of depression within a few hours to a few days after one dose or two doses of psilocybin. 11 12 13 16 17 18 These studies reported only minimal adverse effects, however, and drug harm assessments in healthy volunteers indicated that psilocybin does not induce physiological toxicity, is not addictive, and does not lead to withdrawal. 19 20 Nevertheless, these findings should be interpreted with caution owing to the small sample sizes and open label design of some of these studies. 11 21

Several systematic reviews and meta-analyses since the early 2000s have investigated the use of psilocybin to treat symptoms of depression. Most found encouraging results, but as well as people with depression some included healthy volunteers, 22 and most combined data from studies of multiple serotonergic psychedelics, 23 24 25 even though each compound has unique neurobiological effects and mechanisms of action. 26 27 28 Furthermore, many systematic reviews included non-randomised studies and studies in which psilocybin was tested in conjunction with psychotherapeutic interventions, 25 29 30 31 32 which made it difficult to distinguish psilocybin’s treatment effects. Most systematic reviews and meta-analyses did not consider the impact of factors that could act as moderators to psilocybin’s effects, such as type of depression (primary or secondary), previous use of psychedelics, psilocybin dosage, type of outcome measure (clinician rated or self-reported), and personal characteristics (eg, age, sex). 25 26 29 30 31 32 Lastly, systematic reviews did not consider grey literature, 33 34 which might have led to a substantial overestimation of psilocybin’s efficacy as a treatment for depression. In this review we focused on randomised trials that contained an unconfounded evaluation of psilocybin in adults with symptoms of depression, regardless of country and language of publication.

In this systematic review and meta-analysis of indexed and non-indexed randomised trials we investigated the efficacy of psilocybin to treat symptoms of depression compared with placebo or non-psychoactive drugs. The protocol was registered in the International Prospective Register of Systematic Reviews (see supplementary Appendix A). The study overall did not deviate from the pre-registered protocol; one clarification was made to highlight that any non-psychedelic comparator was eligible for inclusion, including placebo, niacin, micro doses of psychedelics, and drugs that are considered the standard of care in depression (eg, SSRIs).

Inclusion and exclusion criteria

Double blind and open label randomised trials with a crossover or parallel design were eligible for inclusion. We considered only studies in humans and with a control condition, which could include any type of non -active comparator, such as placebo, niacin, or micro doses of psychedelics.

Eligible studies were those that included adults (≥18 years) with clinically significant symptoms of depression, evaluated using a clinically validated tool for depression and mood disorder outcomes. Such tools included the Beck depression inventory, Hamilton depression rating scale, Montgomery-Åsberg depression rating scale, profile of mood states, and quick inventory of depressive symptomatology. Studies of participants with symptoms of depression and comorbidities (eg, cancer) were also eligible. We excluded studies of healthy participants (without depressive symptomatology).

Eligible studies investigated the effect of psilocybin as a standalone treatment on symptoms of depression. Studies with an active psilocybin condition that involved micro dosing (ie, psilocybin <100 μg/kg, according to the commonly accepted convention 22 35 ) were excluded. We included studies with directive psychotherapy if the psychotherapeutic component was present in both the experimental and the control conditions, so that the effects of psilocybin could be distinguished from those of psychotherapy. Studies involving group therapy were also excluded. Any non-psychedelic comparator was eligible for inclusion, including placebo, niacin, and micro doses of psychedelics.

Changes in symptoms, measured by validated clinician rated or self-report scales, such as the Beck depression inventory, Hamilton depression rating scale, Montgomery-Åsberg depression rating scale, profile of mood states, and quick inventory of depressive symptomatology were considered. We excluded outcomes that were measured less than three hours after psilocybin had been administered because any reported changes could be attributed to the transient cognitive and affective effects of the substance being administered. Aside from this, outcomes were included irrespective of the time point at which measurements were taken.

Search strategy

We searched major electronic databases and trial registries of psychological and medical research, with no limits on the publication date. Databases were the Cochrane Central Register of Controlled Trials via the Cochrane Library, Embase via Ovid, Medline via Ovid, Science Citation Index and Conference Proceedings Citation Index-Science via Web of Science, and PsycInfo via Ovid. A search through multiple databases was necessary because each database includes unique journals. Supplementary Appendix B shows the search syntax used for the Cochrane Central Register of Controlled Trials, which was slightly modified to comply with the syntactic rules of the other databases.

Unpublished and grey literature were sought through registries of past and ongoing trials, databases of conference proceedings, government reports, theses, dissertations, and grant registries (eg, ClinicalTrials.gov, WHO International Clinical Trials Registry Platform, ProQuest Dissertations and Theses Global, and PsycEXTRA). The references and bibliographies of eligible studies were checked for relevant publications. The original search was done in January 2023 and updated search was performed on 10 August 2023.

Data collection, extraction, and management

The results of the literature search were imported to the Endnote X9 reference management software, and the references were imported to the Covidence platform after removal of duplicates. Two reviewers (AM and DT) independently screened the title and abstract of each reference and then screened the full text of potentially eligible references. Any disagreements about eligibility were resolved through discussion. If information was insufficient to determine eligibility, the study’s authors were contacted. The reviewers were not blinded to the studies’ authors, institutions, or journal of publication.

The PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow diagram shows the study selection process and reasons for excluding studies that were considered eligible for full text screening. 36

Critical appraisal of individual studies and of aggregated evidence

The methodological quality of eligible studies was assessed using the Cochrane Risk of Bias 2 tool (RoB 2) for assessing risk of bias in randomised trials. 37 In addition to the criteria specified by RoB 2, we considered the potential impact of industry funding and conflicts of interest. The overall methodological quality of the aggregated evidence was evaluated using GRADE (Grading of Recommendations, Assessment, Development and Evaluation). 38

If we found evidence of heterogeneity among the trials, then small study biases, such as publication bias, were assessed using a funnel plot and asymmetry tests (eg, Egger’s test). 39

We used a template for data extraction (see supplementary Appendix C) and summarised the extracted data in tabular form, outlining personal characteristics (age, sex, previous use of psychedelics), methodology (study design, dosage), and outcome related characteristics (mean change from baseline score on a depression questionnaire, response rates, and remission rates) of the included studies. Response conventionally refers to a 50% decrease in symptom severity based on scores on a depression rating scale, whereas remission scores are specific to a questionnaire (eg, score of ≤5 on the quick inventory of depressive symptomatology, score of ≤10 on the Montgomery-Åsberg depression rating scale, 50% or greater reduction in symptoms, score of ≤7 on the Hamilton depression rating scale, or score of ≤12 on the Beck depression inventory). Across depression scales, higher scores signify more severe symptoms of depression.

Continuous data synthesis

From each study we extracted the baseline and post-intervention means and standard deviations (SDs) of the scores between comparison groups for the depression questionnaires and calculated the mean differences and SDs of change. If means and SDs were not available for the included studies, we extracted the values from available graphs and charts using the Web Plot Digitizer application ( https://automeris.io/WebPlotDigitizer/ ). If it was not possible to calculate SDs from the graphs or charts, we generated values by converting standard errors (SEs) or confidence intervals (CIs), depending on availability, using formulas in the Cochrane Handbook (section 7.7.3.2). 40

Standardised mean differences were calculated for each study. We chose these rather than weighted mean differences because, although all the studies measured depression as the primary outcome, they did so with different questionnaires that score depression based on slightly different items. 41 If we had used weighted mean differences, any variability among studies would be assumed to reflect actual methodological or population differences and not differences in how the outcome was measured, which could be misleading. 40

The Hedges’ g effect size estimate was used because it tends to produce less biased results for studies with smaller samples (<20 participants) and when sample sizes differ substantially between studies, in contrast with Cohen’s d. 42 According to the Cochrane Handbook, the Hedges’ g effect size measure is synonymous with the standardised mean difference, 40 and the terms may be used interchangeably. Thus, a Hedges’ g of 0.2, 0.5, 0.8, or 1.2 corresponds to a small, medium, large, or very large effect, respectively. 40

Owing to variation in the participants’ personal characteristics, psilocybin dosage, type of depression investigated (primary or secondary), and type of comparators, we used a random effects model with a Hartung-Knapp-Sidik-Jonkman modification. 43 This model also allowed for heterogeneity and within study variability to be incorporated into the weighting of the results of the included studies. 44 Lastly, this model could help to generalise the findings beyond the studies and patient populations included, making the meta-analysis more clinically useful. 45 We chose the Hartung-Knapp-Sidik-Jonkman adjustment in favour of more widely used random effects models (eg, DerSimonian and Laird) because it allows for better control of type 1 errors, especially for studies with smaller samples, and provides a better estimation of between study variance by accounting for small sample sizes. 46 47

For studies in which multiple treatment groups were compared with a single placebo group, we split the placebo group to avoid multiplicity. 48 Similarly, if studies included multiple primary outcomes (eg, change in depression at three weeks and at six weeks), we split the treatment groups to account for overlapping participants. 40

Prediction intervals (PIs) were calculated and reported to show the expected effect range of a similar future study, in a different setting. In a random effects model, within study measures of variability, such as CIs, can only show the range in which the average effect size could lie, but they are not informative about the range of potential treatment effects given the heterogeneity between studies. 49 Thus, we used PIs as an indication of variation between studies.

Heterogeneity and sensitivity analysis

Statistical heterogeneity was tested using the χ 2 test (significance level P<0.1) and I 2 statistic, and heterogeneity among included studies was evaluated visually and displayed graphically using a forest plot. If substantial or considerable heterogeneity was found (I 2 ≥50% or P<0.1), 50 we considered the study design and characteristics of the included studies. Sources of heterogeneity were explored by subgroup analysis, and the potential effects on the results are discussed.

Planned sensitivity analyses to assess the effect of unpublished studies and studies at high risk of bias were not done because all included studies had been published and none were assessed as high risk of bias. Exclusion sensitivity plots were used to display graphically the impact of individual studies and to determine which studies had a particularly large influence on the results of the meta-analysis. All sensitivity analyses were carried out with Stata 16 software.

Subgroup analysis

To reduce the risk of errors caused by multiplicity and to avoid data fishing, we planned subgroup analyses a priori and limited to: (1) patient characteristics, including age and sex; (2) comorbidities, such as a serious physical condition (previous research indicates that the effects of psilocybin may be less strong for such participants, compared with participants with no comorbidities) 33 ; (3) number of doses and amount of psilocybin administered, because some previous meta-analyses found that a higher number of doses and a higher dose of psilocybin both predicted a greater reduction in symptoms of depression, 34 whereas others reported the opposite 33 ; (4) psilocybin administered alongside psychotherapeutic guidance or as a standalone treatment; (5) severity of depressive symptoms (clinical v subclinical symptomatology); (6) clinician versus patient rated scales; and (7) high versus low quality studies, as determined by RoB 2 assessment scores.

Metaregression

Given that enough studies were identified (≥10 distinct observations according to the Cochrane Handbook’s suggestion 40 ), we performed metaregression to investigate whether covariates, or potential effect modifiers, explained any of the statistical heterogeneity. The metaregression analysis was carried out using Stata 16 software.

Random effects metaregression analyses were used to determine whether continuous variables such as participants’ age, percentage of female participants, and percentage of participants who had previously used psychedelics modified the effect estimate, all of which have been implicated in differentially affecting the efficacy of psychedelics in modifying mood. 51 We chose this approach in favour of converting these continuous variables into categorical variables and conducting subgroup analyses for two primary reasons; firstly, the loss of any data and subsequent loss of statistical power would increase the risk of spurious significant associations, 51 and, secondly, no cut-offs have been agreed for these factors in literature on psychedelic interventions for mood disorders, 52 making any such divisions arbitrary and difficult to reconcile with the findings of other studies. The analyses were based on within study averages, in the absence of individual data points for each participant, with the potential for the results to be affected by aggregate bias, compromising their validity and generalisability. 53 Furthermore, a group level analysis may not be able to detect distinct interactions between the effect modifiers and participant subgroups, resulting in ecological bias. 54 As a result, this analysis should be considered exploratory.

Sensitivity analysis

A sensitivity analysis was performed to determine if choice of analysis method affected the primary findings of meta-analysis. Specifically, we reanalysed the data on change in depression score using a random effects Dersimonian and Laird model without the Hartung-Knapp-Sidik-Jonkman modification and compared the results with those of the originally used model. This comparison is particularly important in the presence of substantial heterogeneity and the potential of small study effects to influence the intervention effect estimate. 55

Patient and public involvement

Research on novel depression treatments is of great interest to both patients and the public. Although patients and members of the public were not directly involved in the planning or writing of this manuscript owing to a lack of available funding for recruitment and researcher training, patients and members of the public read the manuscript after submission.

Figure 1 presents the flow of studies through the systematic review and meta-analysis. 56 A total of 4884 titles were retrieved from the five databases of published literature, and a further 368 titles were identified from the databases of unpublished and international literature in February 2023. After the removal of duplicate records, we screened the abstracts and titles of 875 reports. A further 12 studies were added after handsearching of reference lists and conference proceedings and abstracts. Overall, nine studies totalling 436 participants were eligible. The average age of the participants ranged from 36-60 years. During an updated search on 10 August 2023, no further studies were identified.

Flow of studies in systematic review and meta-analysis

After screening of the title and abstract, 61 titles remained for full text review. Native speakers helped to translate papers in languages other than English. The most common reasons for exclusion were the inclusion of healthy volunteers, absence of control groups, and use of a survey based design rather than an experimental design. After full text screening, nine studies were eligible for inclusion, and 15 clinical trials prospectively registered or underway as of August 2023 were noted for potential future inclusion in an update of this review (see supplementary Appendix D).

We sent requests for further information to the authors of studies by Griffiths et al, 57 Barrett, 58 and Benville et al, 59 because these studies appeared to meet the inclusion criteria but were only provided as summary abstracts online. A potentially eligible poster presentation from the 58th annual meeting of the American College of Neuropsychopharmacology was identified but the lead author (Griffiths) clarified that all information from the presentation was included in the studies by Davis et al 13 and Gukasyan et al 60 ; both of which we had already deemed ineligible.

Barrett 58 reported the effects of psilocybin on the cognitive flexibility and verbal reasoning of a subset of patients with major depressive disorder from Griffith et al’s trial, 61 compared with a waitlist group, but when contacted, Barrett explained that the results were published in the study by Doss et al, 62 which we had already screened and judged ineligible (see supplementary Appendix E). Benville et al’s study 59 presented a follow-up of Ross et al’s study 17 on a subset of patients with cancer and high suicidal ideation and desire for hastened death at baseline. Measures of antidepressant effects of psilocybin treatment compared with niacin were taken before and after treatment crossover, but detailed results are not reported. Table 1 describes the characteristics of the included studies and table 2 lists the main findings of the studies.

Characteristics of included studies

• View inline

Main findings of included studies

Side effects and adverse events

Side effects reported in the included studies were minor and transient (eg, short term increases in blood pressure, headache, and anxiety), and none were coded as serious. Cahart-Harris et al noted one instance of abnormal dreams and insomnia. 63 This side effect profile is consistent with findings from other meta-analyses. 30 68 Owing to the different scales and methods used to catalogue side effects and adverse events across trials, it was not possible to combine these data quantitatively (see supplementary Appendix F).

Risk of bias

The Cochrane RoB 2 tools were used to evaluate the included studies ( table 3 ). RoB 2 for randomised trials was used for the five reports of parallel randomised trials (Carhart-Harris et al 63 and its secondary analysis Barba et al, 64 Goodwin et al 18 and its secondary analysis Goodwin et al, 65 and von Rotz et al 66 ) and RoB 2 for crossover trials was used for the four reports of crossover randomised trials (Griffiths et al, 14 Grob et al, 15 and Ross et al 17 and its follow-up Ross et al 67 ). Supplementary Appendix G provides a detailed explanation of the assessment of the included studies.

Summary risk of bias assessment of included studies, based on domains in Cochrane Risk of Bias 2 tool

Quality of included studies

Confidence in the quality of the evidence for the meta-analysis was assessed using GRADE, 38 through the GRADEpro GDT software program. Figure 2 shows the results of this assessment, along with our summary of findings.

GRADE assessment outputs for outcomes investigated in meta-analysis (change in depression scores and response and remission rates). The risk in the intervention group (and its 95% CI) is based on the assumed risk in the comparison group and the relative effect of the intervention (and its 95% CI). BDI=Beck depression inventory; CI=confidence interval; GRADE=Grading of Recommendations, Assessment, Development and Evaluation; HADS-D=hospital anxiety and depression scale; HAM-D=Hamilton depression rating scale; MADRS=Montgomery-Åsberg depression rating scale; QIDS=quick inventory of depressive symptomatology; RCT=randomised controlled trial; SD=standard deviation

Meta-analyses

Continuous data, change in depression scores —Using a Hartung-Knapp-Sidik-Jonkman modified random effects meta-analysis, change in depression scores was significantly greater after treatment with psilocybin compared with active placebo. The overall Hedges’ g (1.64, 95% CI 0.55 to 2.73) indicated a large effect size favouring psilocybin ( fig 3 ). PIs were, however, wide and crossed the line of no difference (95% CI −1.72 to 5.03), indicating that there could be settings or populations in which psilocybin intervention would be less efficacious.

Forest plot for overall change in depression scores from before to after treatment. CI=confidence interval; DL=DerSimonian and Laird; HKSJ=Hartung-Knapp-Sidik-Jonkman

Exploring publication bias in continuous data —We used Egger’s test and a funnel plot to examine the possibility of small study biases, such as publication bias. Statistical significance of Egger’s test for small study effects, along with the asymmetry in the funnel plot ( fig 4 ), indicates the presence of bias against smaller studies with non-significant results, suggesting that the pooled intervention effect estimate is likely to be overestimated. 69 An alternative explanation, however, is that smaller studies conducted at the early stages of a new psychotherapeutic intervention tend to include more high risk or responsive participants, and psychotherapeutic interventions tend to be delivered more effectively in smaller trials; both of these factors can exaggerate treatment effects, resulting in funnel plot asymmetry. 70 Also, because of the relatively small number of included studies and the considerable heterogeneity observed, test power may be insufficient to distinguish real asymmetry from chance. 71 Thus, this analysis should be considered exploratory.

Funnel plot assessing publication bias among studies measuring change in depression scores from before to after treatment. CI=confidence interval; θ IV =estimated effect size under inverse variance random effects model

Dichotomous data

We extracted response and remission rates for each group when reported directly, or imputed information when presented graphically. Two studies did not measure response or remission and thus did not contribute data for this part of the analysis. 15 18 The random effects model with a Hartung-Knapp-Sidik-Jonkman modification was used to allow for heterogeneity to be incorporated into the weighting of the included studies’ results, and to provide a better estimation of between study variance accounting for small sample sizes.

Response rate —Overall, the likelihood of psilocybin intervention leading to treatment response was about two times greater (risk ratio 2.02, 95% CI 1.33 to 3.07) than with placebo. Despite the use of different scales to measure response, the heterogeneity between studies was not significant (I 2 =25.7%, P=0.23). PIs were, however, wide and crossed the line of no difference (−0.94 to 3.88), indicating that there could be settings or populations in which psilocybin intervention would be less efficacious.

Remission rate —Overall, the likelihood of psilocybin intervention leading to remission of depression was nearly three times greater than with placebo (risk ratio 2.71, 95% CI 1.75 to 4.20). Despite the use of different scales to measure response, no statistical heterogeneity was found between studies (I 2 =0.0%, P=0.53). PIs were, however, wide and crossed the line of no difference (0.87 to 2.32), indicating that there could be settings or populations in which psilocybin intervention would be less efficacious.

Exploring publication bias in response and remission rates data —We used Egger’s test and a funnel plot to examine whether response and remission estimates were affected by small study biases. The result for Egger’s test was non-significant (P>0.05) for both response and remission estimates, and no substantial asymmetry was observed in the funnel plots, providing no indication for the presence of bias against smaller studies with non-significant results.

Heterogeneity: subgroup analyses and metaregression

Heterogeneity was considerable across studies exploring changes in depression scores (I 2 =89.7%, P<0.005), triggering subgroup analyses to explore contributory factors. Table 4 and table 5 present the results of the heterogeneity analyses (subgroup analyses and metaregression, respectively). Also see supplementary Appendix H for a more detailed description and graphical representation of these results.

Subgroup analyses to explore potential causes of heterogeneity among included studies

Metaregression analyses to explore potential causes of heterogeneity among included studies

Cumulative meta-analyses

We used cumulative meta-analyses to investigate how the overall estimates of the outcomes of interest changed as each study was added in chronological order 72 ; change in depression scores and likelihood of treatment response both increased as the percentage of participants with past use of psychedelics increased across studies, as expected based on the metaregression analysis (see supplementary Appendix I). No other significant time related patterns were found.

We reanalysed the data for change in depression scores using a random effects Dersimonian and Laird model without the Hartung-Knapp-Sidik-Jonkman modification and compared the results with those of the original model. All comparisons found to be significant using the Dersimonian and Laird model with the Hartung-Knapp-Sidik-Jonkman adjustment were also significant without the Hartung-Knapp-Sidik-Jonkman adjustment, and confidence intervals were only slightly narrower. Thus, small study effects do not appear to have played a major role in the treatment effect estimate.

Additionally, to estimate the accuracy and robustness of the estimated treatment effect, we excluded studies from the meta-analysis one by one; no important differences in the treatment effect, significance, and heterogeneity levels were observed after the exclusion of any study (see supplementary Appendix J).

In our meta-analysis we found that psilocybin use showed a significant benefit on change in depression scores compared with placebo. This is consistent with other recent meta-analyses and trials of psilocybin as a standalone treatment for depression 73 74 or in combination with psychological support. 24 25 29 30 31 32 68 75 This review adds to those finding by exploring the considerable heterogeneity across the studies, with subsequent subgroup analyses showing that the type of depression (primary or secondary) and the depression scale used (Montgomery-Åsberg depression rating scale, quick inventory of depressive symptomatology, or Beck depression inventory) had a significant differential effect on the outcome. High between study heterogeneity has been identified by some other meta-analyses of psilocybin (eg, Goldberg et al 29 ), with a higher treatment effect in studies with patients with comorbid life threatening conditions compared with patients with primary depression. 22 Although possible explanations, including personal factors (eg, patients with life threatening conditions being older) or depression related factors (eg, secondary depression being more severe than primary depression) could be considered, these hypotheses are not supported by baseline data (ie, patients with secondary depression do not differ substantially in age or symptom severity from patients with primary depression). The differential effects from assessment scales used have not been examined in other meta-analyses of psilocybin, but this review’s finding that studies using the Beck depression inventory showed a higher treatment effect than those using the Montgomery-Åsberg depression rating scale and quick inventory of depressive symptomatology is consistent with studies in the psychological literature that have shown larger treatment effects when self-report scales are used (eg, Beck depression inventory). 76 77 This finding may be because clinicians tend to overestimate the severity of depression symptoms at baseline assessments, leading to less pronounced differences between before and after treatment identified in clinician assessed scales (eg, Montgomery-Åsberg depression rating scale, quick inventory of depressive symptomatology). 78

Metaregression analyses further showed that a higher average age and a higher percentage of participants with past use of psychedelics both correlated with a greater improvement in depression scores with psilocybin use and explained a substantial amount of between study variability. However, the cumulative meta-analysis showed that the effects of age might be largely an artefact of the inclusion of one specific study, and alternative explanations are worth considering. For instance, Studerus et al 79 identified participants’ age as the only personal variable significantly associated with psilocybin response, with older participants reporting a higher “blissful state” experience. This might be because of older people’s increased experience in managing negative emotions and the decrease in 5-hydroxytryptamine type 2A receptor density associated with older age. 80 Furthermore, Rootman et al 81 reported that the cognitive performance of older participants (>55 years) improved significantly more than that of younger participants after micro dosing with psilocybin. Therefore, the higher decrease in depressive symptoms associated with older age could be attributed to a decrease in cognitive difficulties experienced by older participants.

Interestingly, a clear pattern emerged for past use of psychedelics—the higher the proportion of study participants who had used psychedelics in the past, the higher the post-psilocybin treatment effect observed. Past use of psychedelics has been proposed to create an expectancy bias among participants and amplify the positive effects of psilocybin 82 83 84 ; however, this important finding has not been examined in other meta-analyses and may highlight the role of expectancy in psilocybin research.

Limitations of this study

Generalisability of the findings of this meta-analysis was limited by the lack of racial and ethnic diversity in the included studies—more than 90% of participants were white across all included trials, resulting in a homogeneous sample that is not representative of the general population. Moreover, it was not possible to distinguish between subgroups of participants who had never used psilocybin and those who had taken psilocybin more than a year before the start of the trial, as these data were not provided in the included studies. Such a distinction would be important, as the effects of psilocybin on mood may wane within a year after being administered. 21 85 Also, how psychological support was conceptualised was inconsistent within studies of psilocybin interventions; many studies failed to clearly describe the type of psychological support participants received, and others used methods ranging from directive guidance throughout the treatment session to passive encouragement or reassurance (eg, Griffiths et al, 14 Carhart-Harris et al 63 ). The included studies also did not gather evidence on participants’ previous experiences with treatment approaches, which could influence their response to the trials’ intervention. Thus, differences between participant subgroups related to past use of psilocybin or psychotherapy may be substantial and could help interpret this study’s findings more accurately. Lastly, the use of graphical extraction software to estimate the findings of studies where exact numerical data were not available (eg, Goodwin et al, 18 Grob et al 15 ), may have affected the robustness of the analyses.

A common limitation in studies of psilocybin is the likelihood of expectancy effects augmenting the treatment effect observed. Although some studies used low dose psychedelics as comparators to deal with this problem (eg, Carhart-Harris et al, 63 Goodwin et al, 18 Griffiths et al 14 ) or used a niacin placebo that can induce effects similar to those of psilocybin (eg, Grob et al, 15 Ross et al 17 ), the extent to which these methods were effective in blinding participants is not known. Other studies have, however, reported that participants can accurately identify the study groups to which they had been assigned 70-85% of the time, 84 86 indicating a high likelihood of insufficient blinding. This is especially likely for studies in which a high proportion of participants had previously used psilocybin and other hallucinogens, making the identification of the drug’s acute effects easier (eg, Griffiths et al, 14 Grob et al, 15 Ross et al 17 ). Patients also have expectations related to the outcome of their treatment, expecting psilocybin to improve their symptoms of depression, and these positive expectancies are strong predictors of actual treatment effects. 87 88 Importantly, the effect of outcome expectations on treatment effect is particularly strong when patient reported measures are used as primary outcomes, 89 which was the case in several of the included studies (eg, Griffiths et al, 14 Grob et al, 15 Ross et al 17 ). Unfortunately, none of the included studies recorded expectations before treatment, so it is not possible to determine the extent to which this factor affected the findings.

Implications for clinical practice

Although this review’s findings are encouraging for psilocybin’s potential as an effective antidepressant, a few areas about its applicability in clinical practice remain unexplored. Firstly, it is unclear whether the protocols for psilocybin interventions in clinical trials can be reliably and safely implemented in clinical practice. In clinical trials, patients receive psilocybin in a non-traditional medical setting, such as a specially designed living room, while they may be listening to curated calming music and are isolated from most external stimuli by wearing eyeshades and external noise-cancelling earphones. A trained therapist closely supervises these sessions, and the patient usually receives one or more preparatory sessions before the treatment commences. Standardising an intervention setting with so many variables is unlikely to be achievable in routine practice, and consensus is considerably lacking on the psychotherapeutic training and accreditations needed for a therapist to deliver such treatment. 90 The combination of these elements makes this a relatively complex and expensive intervention, which could make it challenging to gain approval from regulatory agencies and to gain reimbursement from insurance companies and others. Within publicly funded healthcare systems, the high cost of treatment may make psilocybin treatment inaccessible. The high cost associated with the intervention also increases the risk that unregulated clinics may attempt to cut costs by making alterations to the protocol and the therapeutic process, 91 92 which could have detrimental effects for patients. 92 93 94 Thus, avoiding the conflation of medical and commercial interests is a primary concern that needs to be dealt with before psilocybin enters mainstream practice.

Implications for future research

More large scale randomised trials with long follow-up are needed to fully understand psilocybin’s treatment potential, and future studies should aim to recruit a more diverse population. Another factor that would make clinical trials more representative of routine practice would be to recruit patients who are currently using or have used commonly prescribed serotonergic antidepressants. Clinical trials tend to exclude such participants because many antidepressants that act on the serotonin system modulate the 5-hydroxytryptamine type 2A receptor that psilocybin primarily acts upon, with prolonged use of tricyclic antidepressants associated with more intense psychedelic experiences and use of monoamine oxidase inhibitors or SSRIs inducing weaker responses to psychedelics. 95 96 97 Investigating psilocybin in such patients would, however, provide valuable insight on how psilocybin interacts with commonly prescribed drugs for depression and would help inform clinical practice.

Minimising the influence of expectancy effects is another core problem for future studies. One strategy would be to include expectancy measures and explore the level of expectancy as a covariate in statistical analysis. Researchers should also test the effectiveness of condition masking. Another proposed solution would be to adopt a 2×2 balanced placebo design, where both the drug (psilocybin or placebo) and the instructions given to participants (told they have received psilocybin or told they have received placebo) are crossed. 98 Alternatively, clinical trials could adopt a three arm design that includes both an inactive placebo (eg, saline) and active placebo (eg, niacin, lower psylocibin dose), 98 allowing for the effects of psilocybin to be separated from those of the placebo.

Overall, future studies should explore psilocybin’s exact mechanism of treatment effectiveness and outline how its physiological effects, mystical experiences, dosage, treatment setting, psychological support, and relationship with the therapist all interact to produce a synergistic antidepressant effect. Although this may be difficult to achieve using an explanatory randomised trial design, pragmatic clinical trial designs may be better suited to psilocybin research, as their primary objective is to achieve high external validity and generalisability. Such studies may include multiple alternative treatments rather than simply an active and placebo treatment comparison (eg, psilocybin v SSRI v serotonin-noradrenaline reuptake inhibitor), and participants would be recruited from broader clinical populations. 99 100 Although such studies are usually conducted after a drug’s launch, 100 earlier use of such designs could help assess the clinical effectiveness of psilocybin more robustly and broaden patient access to a novel type of antidepressant treatment.

Conclusions

This review’s findings on psilocybin’s efficacy in reducing symptoms of depression are encouraging for its use in clinical practice as a drug intervention for patients with primary or secondary depression, particularly when combined with psychological support and administered in a supervised clinical environment. However, the highly standardised treatment setting, high cost, and lack of regulatory guidelines and legal safeguards associated with psilocybin treatment need to be dealt with before it can be established in clinical practice.

What is already known on this topic

Recent research on treatments for depression has focused on psychedelic agents that could have strong antidepressant effects without the drawbacks of classic antidepressants; psilocybin being one such substance

Over the past decade, several clinical trials, meta-analyses, and systematic reviews have investigated the use of psilocybin for symptoms of depression, and most have found that psilocybin can have antidepressant effects

Studies published to date have not investigated factors that may moderate psilocybin’s effects, including type of depression, past use of psychedelics, dosage, outcome measures, and publication biases

What this study adds

This review showed a significantly greater efficacy of psilocybin among patients with secondary depression, patients with past use of psychedelics, older patients, and studies using self-report measures for symptoms of depression

Efficacy did not appear to be homogeneous across patient types—for example, those with depression and a life threatening illness appeared to benefit more from treatment

Further research is needed to clarify the factors that maximise psilocybin’s treatment potential for symptoms of depression

Ethics statements

Ethical approval.

This study was approved by the ethics committee of the University of Oxford Nuffield Department of Medicine, which waived the need for ethical approval and the need to obtain consent for the collection, analysis, and publication of the retrospectively obtained anonymised data for this non-interventional study.

Data availability statement

The relevant aggregated data and statistical code will be made available on reasonable request to the corresponding author.

Acknowledgments

We thank DT who acted as an independent secondary reviewer during the study selection and data review process.

Contributors: AMM contributed to the design and implementation of the research, analysis of the results, and writing of the manuscript. MC was involved in planning and supervising the work and contributed to the writing of the manuscript. AMM and MC are the guarantors. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: None received.

Competing interests: All authors have completed the ICMJE uniform disclosure form at https://www.icmje.org/disclosure-of-interest/ and declare: no support from any organisation for the submitted work; AMM is employed by IDEA Pharma, which does consultancy work for pharmaceutical companies developing drugs for physical and mental health conditions; MC was the supervisor for AMM’s University of Oxford MSc dissertation, which forms the basis for this paper; no other relationships or activities that could appear to have influenced the submitted work.

Transparency: The corresponding author (AMM) affirms that the manuscript is an honest, accurate, and transparent account of the study being reported; that no important aspects of the study have been omitted; and that any discrepancies from the study as registered have been explained.

Dissemination to participants and related patient and public communities: To disseminate our findings and increase the impact of our research, we plan on writing several social media posts and blog posts outlining the main conclusions of our paper. These will include blog posts on the websites of the University of Oxford’s Department of Primary Care Health Sciences and Department for Continuing Education, as well as print publications, which are likely to reach a wider audience. Furthermore, we plan to present our findings and discuss them with the public in local mental health related events and conferences, which are routinely attended by patient groups and advocacy organisations.

Provenance and peer review: Not commissioned; externally peer reviewed.

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

• ↵ World Health Organization. Depressive Disorder (Depression); 2023. https://www.who.int/news-room/fact-sheets/detail/depression .
• GBD 2017 Disease and Injury Incidence and Prevalence Collaborators
• Cipriani A ,
• Furukawa TA ,
• Salanti G ,
• Trivedi MH ,
• Wisniewski SR ,
• Mitchell AJ
• Bockting CL ,
• Hollon SD ,
• Jarrett RB ,
• Nierenberg AA ,
• Petersen TJ ,
• Páleníček T ,
• Carbonaro TM ,
• Bradstreet MP ,
• Barrett FS ,
• Carhart-Harris RL ,
• Bolstridge M ,
• Griffiths RR ,
• Johnson MW ,
• Carducci MA ,
• Danforth AL ,
• Chopra GS ,
• Kraehenmann R ,
• Preller KH ,
• Scheidegger M ,
• Goodwin GM ,
• Aaronson ST ,
• Alvarez O ,
• Bogenschutz MP ,
• Podrebarac SK ,
• Roseman L ,
• Galvão-Coelho NL ,
• Gonzalez M ,
• Dos Santos RG ,
• Osório FL ,
• Crippa JA ,
• Zuardi AW ,
• Cleare AJ ,
• Martelli C ,
• Benyamina A
• Vollenweider FX ,
• Demetriou L ,
• Carhart-Harris RL
• Timmermann C ,
• Giribaldi B ,
• Goldberg SB ,
• Nicholas CR ,
• Raison CL ,
• Irizarry R ,
• Winczura A ,
• Dimassi O ,
• Dhillon N ,
• Griffiths RR
• Castro Santos H ,
• Gama Marques J
• Moreno FA ,
• Wiegand CB ,
• Taitano EK ,
• Liberati A ,
• Tetzlaff J ,
• Altman DG ,
• PRISMA Group
• Sterne JAC ,
• Savović J ,
• Guyatt GH ,
• Schünemann HJ ,
• Tugwell P ,
• Knottnerus A
• Sterne JA ,
• Sutton AJ ,
• Ioannidis JP ,
• Higgins JPT ,
• Chandler J ,
• Borenstein M ,
• Hedges LV ,
• Higgins JP ,
• Rothstein HR
• DerSimonian R ,
• ↵ Borenstein M, Hedges L, Rothstein H. Meta-analysis: Fixed effect vs. random effects. Meta-analysis. com. 2007;1-62.
• IntHout J ,
• Rovers MM ,
• Gøtzsche PC
• Spineli LM ,
• ↵ Higgins JP, Green S. Identifying and measuring heterogeneity. Cochrane handbook for systematic reviews of interventions. 2011;5(0).
• Austin PC ,
• O’Donnell KC ,
• Mennenga SE ,
• Bogenschutz MP
• Sander SD ,
• Berlin JA ,
• Santanna J ,
• Schmid CH ,
• Szczech LA ,
• Feldman HI ,
• Anti-Lymphocyte Antibody Induction Therapy Study Group
• ↵ Iyengar S, Greenhouse J. Sensitivity analysis and diagnostics. Handbook of research synthesis and meta-analysis. Russell Sage Foundation, 2009:417-33.
• McKenzie JE ,
• Bossuyt PM ,
• ↵ Griffiths R, Barrett F, Johnson M, Mary C, Patrick F, Alan D. Psilocybin-Assisted Treatment of Major Depressive Disorder: Results From a Randomized Trial. Proceedings of the ACNP 58th Annual Meeting: Poster Session II. In Neuropsychopharmacology. 2019;44:230-384.
• ↵ Barrett F. ACNP 58th Annual Meeting: Panels, Mini-Panels and Study Groups. [Abstract.] Neuropsychopharmacology 2019;44:1-77. doi: 10.1038/s41386-019-0544-z . OpenUrl CrossRef
• Benville J ,
• Agin-Liebes G ,
• Roberts DE ,
• Gukasyan N ,
• Hurwitz ES ,
• Považan M ,
• Rosenberg MD ,
• Carhart-Harris R ,
• Buehler S ,
• Kettner H ,
• von Rotz R ,
• Schindowski EM ,
• Jungwirth J ,
• Vargas AS ,
• Barroso M ,
• Gallardo E ,
• Isojarvi J ,
• Lefebvre C ,
• Glanville J
• Sukpraprut-Braaten S ,
• Narlesky M ,
• Strayhan RC
• Prouzeau D ,
• Conejero I ,
• Voyvodic PL ,
• Becamel C ,
• Lopez-Castroman J
• Więckiewicz G ,
• Stokłosa I ,
• Gorczyca P ,
• John Mann J ,
• Currier D ,
• Zimmerman M ,
• Friedman M ,
• Boerescu DA ,
• Attiullah N
• Borgherini G ,
• Conforti D ,
• Studerus E ,
• Kometer M ,
• Vollenweider FX
• Pinborg LH ,
• Rootman JM ,
• Kryskow P ,
• Turner EH ,
• Rosenthal R
• Bershad AK ,
• Schepers ST ,
• Bremmer MP ,
• Sepeda ND ,
• Hurwitz E ,
• Horvath AO ,
• Del Re AC ,
• Flückiger C ,
• Rutherford BR ,
• Pearson C ,
• Husain SF ,
• Harris KM ,
• George JR ,
• Michaels TI ,
• Sevelius J ,
• Williams MT
• Collins A ,
• Bonson KR ,
• Buckholtz JW ,
• Yamauchi M ,
• Matsushima T ,
• Coleshill MJ ,
• Colloca L ,
• Zachariae R ,
• Colagiuri B
• Heifets BD ,
• Pratscher SD ,
• Bradley E ,
• Sugarman J ,

• Open access
• Published: 05 May 2024

Exoskeleton-based exercises for overground gait and balance rehabilitation in spinal cord injury: a systematic review of dose and dosage parameters

• Patrik Nepomuceno 1 , 2 , 5 ,
• Wagner H. Souza 1 ,
• Maureen Pakosh 1 ,
• Kristin E. Musselman 1 , 3 , 6 &
• B. Catharine Craven 1 , 4 , 5 , 6  

Journal of NeuroEngineering and Rehabilitation volume  21 , Article number:  73 ( 2024 ) Cite this article

41 Accesses

2 Altmetric

Metrics details

Exoskeletons are increasingly applied during overground gait and balance rehabilitation following neurological impairment, although optimal parameters for specific indications are yet to be established.

This systematic review aimed to identify dose and dosage of exoskeleton-based therapy protocols for overground locomotor training in spinal cord injury/disease.

A systematic review was conducted in accordance with the Preferred Reporting Items Systematic Reviews and Meta-Analyses guidelines. A literature search was performed using the CINAHL Complete, Embase, Emcare Nursing, Medline ALL, and Web of Science databases. Studies in adults with subacute and/or chronic spinal cord injury/disease were included if they reported (1) dose (e.g., single session duration and total number of sessions) and dosage (e.g., frequency of sessions/week and total duration of intervention) parameters, and (2) at least one gait and/or balance outcome measure.

Of 2,108 studies identified, after removing duplicates and filtering for inclusion, 19 were selected and dose, dosage and efficacy were abstracted. Data revealed a great heterogeneity in dose, dosage, and indications, with overall recommendation of 60-min sessions delivered 3 times a week, for 9 weeks in 27 sessions. Specific protocols were also identified for functional restoration (60-min, 3 times a week, for 8 weeks/24 sessions) and cardiorespiratory rehabilitation (60-min, 3 times a week, for 12 weeks/36 sessions).

This review provides evidence-based best practice recommendations for overground exoskeleton training among individuals with spinal cord injury/disease based on individual therapeutic goals – functional restoration or cardiorespiratory rehabilitation. There is a need for structured exoskeleton clinical translation studies based on standardized methods and common therapeutic outcomes.

Introduction

Over the past decade, lower limb robotic technologies have been increasingly applied in neurorehabilitation [ 1 , 2 ]. Essentially anthropomorphic in concept, these powered mechanical devices are used for locomotor training [ 3 ] and are classified as end-effectors or exoskeletons [ 4 ]. The first one generates movements from the distal segment through a haptic interface [ 5 ], while the latter encompass independent robot joints guided in a pre-programmed trajectory which is further classified as unilateral or bilateral [ 4 ]. Among such technologies, exoskeletons are reportedly useful to promote mobility in individuals with locomotor dysfunction, including those with complete lower extremity paralysis [ 6 ]. Exoskeletons often do so through motorized actuators that assist hip, knee, and ankle motion in dynamic orthoses capable of supporting, stabilizing and reciprocally progressing the lower limbs [ 4 ]. Newer generation devices offer training modes which allow therapists to manually trigger and control steps, in addition to adaptive and variable assistive features for individuals with incomplete injuries and a fair prognosis for voluntary active movement and functional recovery.

More recently, an alternative robotic exoskeleton classification was suggested based on four categories: end-effectors (e.g., Haptic Walker), grounded exoskeletons (e.g., Lokomat), wearable exoskeletons (e.g., Ekso and ReWalk) and soft exoskeletons (e.g., Myosuit) [ 7 ]. These devices seem especially promising as strategies to improve balance and walking abilities [ 8 , 9 ], two of the most frequent goals following subacute or chronic spinal cord injury/disease (SCI/D) [ 7 , 10 ]. The first, characterized by physiologic responses at a cellular level (e.g., glial scars), occurs within a few weeks after the injury [ 11 , 12 , 13 ]. Conversely, the latter is achieved as of 6 months after the injury. In traumatic SCI, the interval between the acute (< 30 days) and chronic (> 6 months) phases has been labelled the intermediate phase [ 14 ].

In terms of motor support, exoskeletons offer different types of assistance including active (equipment performs the movement, partially or totally, through powered assistance to the user); passive (device does not offer powered assistance to the movement, users execute by themselves); active-assisted (offers powered assistance to complete movements initiated by the user); resistive (offers resistance to movements initiated by the user); and interactive (uses feedback to correct movements based on interactions between actuators and control strategies) [ 4 , 7 , 15 ]. Understanding these different levels of assistance is important to account for the variable forms of haptic feedback involved in robotic motor training which can either enhance or degrade motor performance depending on the patient’s impairments and abilities (e.g., novice learners vs. advanced learners, subacute vs. chronic patients, those with autonomic or sensory function, presence or absence of spasticity, etc.) [ 16 ]. Prior publications with these variable assistive devices have shown that gait and balance training with exoskeletons contribute to increased energy expenditure, muscle activation/recruitment and weight bearing [ 17 , 18 , 19 , 20 ], in addition to improved independence and health-related quality of life [ 21 ]. These outcomes are often achieved in response to neurorecovery fostered by functional restoration programs[ 22 ]. Functional Restoration interventions focus on the refinement of sensorimotor function in daily living. That ability is associated with the stimulation of remaining neural connections that even in SCI/D re-enable sensorimotor function following repeated exposure to directed stimuli, hence yielding [ 23 ] greater motor and autonomic recovery [ 23 , 24 ].

Specific to SCI/D, a recent study of exoskeleton-based rehabilitation among individuals with subacute injury reported that exposure to sixteen 30-min sessions of robotic-assisted gait training led to a significant improvement in gait as measured by the Walking Index SCI II (WISCI-II), which translates to more functional gait and activities of daily living [ 25 ]. Moreover, Tamburella et al. [ 26 ] reported that individuals living with SCI/D could walk significantly faster, with longer steps and reduced gait cycles after rehabilitation with a powered exoskeleton. Similarly, Okawara et al. [ 27 ] reported gains in the 10-Meter Walk Test (10MWT), Time Up and Go (TUG) and Berg Balance Scale (BBS) after twenty 60-min sessions of body weight supported treadmill training (BWSTT) with a hybrid-assisted limb system. These results, however, were only observed in SCI/D patients with prior high walking ability as measured by the WISCI-II. In a similar population, Baunsgaard et al. [ 28 ] performed twenty-four 60-min sessions of robotic exoskeleton gait training, which resulted in improvements in the 10MWT, TUG and BBS, however with no treadmill or body weight support. The aforementioned results suggest that individuals living with subacute spinal cord lesions (< 1 year) are most likely to experience therapeutic benefits. However, individuals living with chronic SCI/D may also benefit from these interventions. While neuroplasticity is primarily expected at earlier phases after SCI/D, improvements are still attainable at later stages, specifically in response to coordinated, repeated motor stimuli as fostered by exoskeletons [ 4 , 17 , 28 ].

In response to the growing interest in exoskeletons to enhance the outcomes of neurorehabilitation, particularly in SCI/D, a significant body of literature has been published on associated topics and therapeutic benefits such as cardiovascular function [ 19 ], gait performance and training [ 17 , 19 ], spasticity and pain [ 18 ], device characteristics [ 29 ], cardiorespiratory function and fatigue [ 30 ]. Although the aforementioned evidence is based on structured rehabilitation protocols, little emphasis has been given to discussing dose and dosage parameters of the exercises used in the respective therapeutic protocols beyond feasibility, safety and the specific outcomes observed. Additionally, interventions using powered exoskeleton-based rehabilitation for gait and balance were reportedly delivered under widely variable designs [ 31 , 32 , 33 , 34 ]. Although dose and dosage parameters were reported by previous systematic review authors in adults with SCI/D who underwent lower limb powered exoskeleton rehabilitation for overground gait and balance, most did not discuss these training parameters. Instead, most authors acknowledged the absence of best practice recommendations in the field and endorsed the need to further understand rehabilitation designs aimed to restore or maintain locomotion with powered exoskeletons [ 7 , 15 , 21 , 35 ].

This systematic review addresses two main questions: (1) To what extent are dose (e.g., single session duration, and total number of sessions) and dosage (e.g., frequency of sessions per week, and total duration of the intervention) of exoskeleton-based exercises reported in the literature on overground gait and balance rehabilitation for adults with SCI/D (subacute or chronic, complete or incomplete)?; and, (2) Which outcome measures are used to inform changes in gait and balance following exoskeleton-based rehabilitation in SCI/D? We hypothesized that the investigation of dose and dosage parameters of exoskeleton-based exercises reported from interventions for overground gait and balance rehabilitation interventions among individuals with SCI/D would contribute to: (1) the identification of consistent dose and dosage parameters to inform best practice recommendations related to locomotor rehabilitation strategies; and, (2) informing the development of innovative, clinically robust protocols evaluating exoskeletons for SCI/D rehabilitation; and, (3) to driving implementation of exoskeleton based training programs within tertiary SCI/D rehabilitation settings.

This systematic review was conducted in accordance with the Preferred Reporting Items Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 36 ] and registered in the International Prospective Register of Systematic Reviews (PROSPERO) under the number CRD42022319271.

Search strategy and data sources

The search strategy was co-developed by the authors in collaboration with a local Medical Librarian and Information Specialist (MP) using the concepts contained in the PICO framework encompassing P opulation, I ntervention, C omparisons, and O utcomes. Valid subject headings for each database were utilized as appropriate, as were free text terms pertinent to each topic or concept (e.g., Spinal Cord Injuries; Paraplegia; Quadriplegia; Exoskeleton Device; Gait; Postural Balance). The search was performed from inception to 31 March 2022 using five electronic databases: CINAHL Complete (EBSCOhost), Embase (Ovid), Emcare Nursing (Ovid), Medline ALL (Ovid; includes PubMed non-Medline records), and the Web of Science Core Collection. Each concept searched was kept as broad as possible to ensure all relevant materials were identified. The Population encompassed adults with Spinal Cord Injuries. The Intervention was the use of Exoskeletons. The Outcomes included any biomechanical and/or clinical measures related to Gait or Balance. No date or language limits were applied. The full Medline search strategy is shown in Additional file 1 .

Study selection criteria

Studies were included according to the following criteria:

Participants: adults regardless of sex/gender identity (≥ 16 years of age) with subacute/chronic (≥ 30 days post injury onset) complete or incomplete SCI/D of traumatic or non-traumatic etiology; and any neurological level of injury (C1-L4 ASIA Impairment Scale A-D).

Intervention/Exposure: overground gait and balance rehabilitation with a lower limb powered exoskeleton – an anthropomorphic device worn by the participants for orthostatic passive or active (facilitated) motor training [ 3 ].

Comparison: no specific rehabilitation strategy was specified for comparison.

Outcomes: studies which included at least 3 of 4 parameters of dose (e.g., single session duration, and total number of sessions) and dosage (e.g., frequency of sessions per week, and total duration of the intervention) of exoskeleton-based exercises; and at least one measure of gait and/or balance (e.g., Mini-Balance Evaluation Systems Test, Community Balance & Mobility Scale, ABC Scale, 6-min walk test (6MWT), 10MWT or other measure of gait speed, BBS, TUG).

Publication type: Experimental studies with more than five participants in randomized clinical trials, quasi-randomized clinical trials, prospective controlled trials, pre-post studies, cross-sectional, crossover and quasi-experimental studies. Studies with mixed populations (e.g., children and adults) or mixed impairments (e. g., SCI/D, stroke, multiple sclerosis), were included when outcome separation was possible. Only peer-reviewed articles were included. Reasons for exclusion included: literature reviews, qualitative studies, case series (n < 5), grey literature (i.e., letters, editorial, white papers), studies with end-effector or grounded systems, equipment design and development studies, and with gait training carried over specialized surfaces (e.g., treadmill). The inclusion and exclusion criteria are listed in the Table  1 .

Screening criteria and study selection

After the initial search, duplicate manuscripts were excluded, and remaining references were imported into the Covidence Systematic Review Manager (Veritas Health Innovation Ltd, Australia). Articles eligible for title and abstract screening were assessed by PN and WHS independently (a third author, KEM, was assigned to resolve eventual conflicts). Prior to working independently, an initial fidelity agreement regarding the article inclusion/exclusion process was established based on the first 10 studies with a 100% agreement between raters. If titles and abstracts did not report enough information to determine article inclusion or exclusion, the full text was screened. Following the title and abstract screening, remaining citations were independently read in full by the same two authors to verify articles met inclusion criteria. Again, disagreements were resolved by the same third author.

Data charting and analysis

The authors created individual versions of a data extraction form. Their forms were compared and merged into a combined form used to abstract data from the included manuscripts. The data extraction form was pilot tested by two authors (PN and WHS), who independently extracted data from two of the included manuscripts. Following a comparison of the outcomes obtained, minor revisions were implemented towards a final, revised version of the abstraction form.

Data were extracted from the selected papers about authors; year of publication; institution and country of the study; participant demographics (age, number of participants, etiology and level of lesion,); dose (e.g., total number of sessions, and duration of each session, in minutes) and dosage (e.g., frequency of sessions per week, and duration of the complete intervention, in weeks); gait and balance outcomes measures (e.g., Mini-Balance Evaluation Systems Test, Community Balance & Mobility Scale, ABC Scale, 6MWT, 10MWT, BBS, TUG, gait speed). The data were synthesised by the authors and reported in tables and graphics. Narrative syntheses were applied.

In the case of articles with missing data (e.g., total duration of intervention), the corresponding author was contacted by e-mail. For some studies included, dose and dosage parameters were not explicitly stated, but could be estimated using available training parameters in the published article. For instance, sessions per week multiplied by the number of intervention weeks informed the total number of sessions; total number of sessions divided by weeks informed weekly frequency; and total number of sessions divided by sessions per week informed the duration of the intervention. For parameters indicated as best practice recommendations, only studies that reported statistically significant improvements (p < 0.05) and/or improvements equal or greater than the minimal clinically important difference (MCID) were considered. The MCID was observed for the 6MWT, 10MWT and TUG, with the following thresholds: 36 m [ 37 ] or 0.1 m/s [ 38 ], 0.13 m/s [ 39 ], and 10.8 s [ 40 ], respectively. For the cardiorespiratory outcomes, no MCID was set, and only statistically significant improvements (p < 0.05) were considered. Conversely, studies with dramatically large variability within the reported protocol (e.g., participants exposed to a different total number of sessions from 12 to 102, duration of intervention from 4 to 34 weeks) were excluded from the average calculation. As for studies with small variability within the protocol, the mean of the total range (e.g., weekly frequency from 4 to 5, was considered as 4.5; duration of each session from 60 to 90-min, was considered 75-min) were computed. Data regarding dose and dosage parameters were reported as mean and standard deviation (normal distribution) or median and interquartile range (non-normal distribution), to determine distribution the Shapiro–Wilk Test was used considering p < 0.05 as non-normal distribution.

The initial electronic database search identified 2,108 references. After removing the duplicates, 977 references were screened for titles and abstracts. At full text screening, 69 articles were revised (Fig.  1 ). Nineteen (n = 19) full text articles were included in the review with a total of 288 participants (214 male) who underwent exoskeleton gait and/or balance training. Five (n = 5) studies had control/comparison groups treated with conventional physical therapy (n = 2) [ 41 , 42 ], Lokomat gait training (n = 1) [ 43 ], BWSTT or no intervention (n = 1) [ 44 ] and BWSTT with overground gait training with functional electrical stimulation (FES) (n = 1) [ 45 ]. One (n = 1) study had a comparison group of individuals with acute SCI/D who underwent the same exoskeleton protocol [ 46 ]. As for the geographical distribution of study sites, five (n = 5) were developed in the United States [ 6 , 33 , 44 , 47 , 48 ], four (n = 4) in Italy [ 43 , 45 , 49 , 50 ], two (n = 2) in Canada [ 51 , 52 ], two (n = 2) in China [ 41 , 53 ] and two (n = 2) in Korea [ 54 , 55 ], one (n = 1) in France [ 56 ], one (n = 1) in Japan [ 46 ], one (n = 1) in South Africa [ 42 ], and one (n = 1) from a 7 site (Denmark, Germany, the Netherlands, Norway, Spain, Sweden and Switzerland) multicenter study in Europe [ 28 ], Fig.  2 displays the countries of origin for 18 studies, except for the multicenter study in Europe, which is the most active region investigating overground exoskeletons training for gait and balance rehabilitation among individuals with SCI/D. Six (n = 6) studies were partially or totally supported by the industry manufacturer, including equipment loan [ 6 , 57 ], trial funding [ 28 , 44 , 56 ] and employees collaborating in manuscript production [ 54 ].

PRISMA flow diagram

Frequency of study per country. Figure represents the country of origin of 18 of the 19 studies included because 1 study was a multicenter study across Europe

The refined dataset included articles describing participants with subacute (1 to 5 months post-injury) or chronic (> 6- or 12-months post-injury) SCI/D. Thirteen (n = 13) studies investigated chronic SCI/D (> 6 months [ 46 , 51 ], > 12 months [ 6 , 42 , 43 , 44 , 48 , 52 , 54 , 55 , 56 , 57 ], stated it is chronic but did not report time since injury[ 50 ]), one study investigated subacute participants (from 1 to 11 months) [ 41 ] and five studies investigated both subacute and chronic participants [ 28 , 45 , 47 , 49 , 53 ]. The participant’s age ranged from 16 to 78 years, although one study included one participant that was 15 years old, however this paper was not included in our best practice recommendation because the authors did not find significant changes. That study, however, had a mean participant age of 41.3 years [ 53 ]. Regarding the etiology of the injury, nine (n = 9) studies included individuals with SCI/D of traumatic and non-traumatic etiology (four chronic [ 6 , 43 , 44 , 51 ], four chronic and subacute [ 28 , 45 , 53 , 57 ], one subacute only [ 41 ]). Five (n = 5) studies only included individuals with traumatic lesions (four chronic [ 42 , 48 , 52 , 56 ] and one subacute and chronic [ 49 ]). One (n = 1) study focused on chronic non-traumatic participants [ 46 ]. Four (n = 4) studies did not report the etiology (three chronic [ 50 , 54 , 55 ], one chronic and one subacute [ 47 ]). As for the extent of injury, twelve (n = 12) studies were conducted in individuals with complete or incomplete SCI/D [ 28 , 41 , 43 , 45 , 47 , 48 , 50 , 51 , 52 , 53 , 54 , 57 ], four (n = 4) studies in complete SCI/D only [ 6 , 49 , 55 , 56 ] and two (n = 2) studies in incomplete SCI/D [ 42 , 44 ]. One study did not report the extent of participant injury [ 46 ]. Relative to the level of injury, one (n = 1) study included individuals with cervical lesions [ 42 ], four (n = 4) included individuals with thoracic lesions [ 6 , 48 , 55 , 56 ], four (n = 4) included individuals with cervical or thoracic lesions [ 43 , 51 , 52 , 57 ], two (n = 2) included cervical, thoracic, or lumbar [ 44 , 54 ], and six (n = 6) studies included thoracic or lumbar injuries [ 41 , 45 , 47 , 49 , 50 , 53 ]. Two (n = 2) studies did not report the level of injury [ 28 , 46 ]. A summary of participants’ characteristics and the exoskeleton device with their respective study protocols are shown in Table  2 .

Exoskeleton training dose, dosage, and outcome measures

The 19 studies included devices from seven different exoskeleton manufacturers. Seven (n = 7) studies used Ekso devices [ 28 , 42 , 43 , 44 , 45 , 50 , 51 ], six (n = 6) used ReWalk [ 6 , 48 , 49 , 52 , 55 , 57 ], two (n = 2) used AIDER [ 41 , 53 ], one (n = 1) used Indego [ 47 ], one (n = 1) H-MEX[ 54 ], one (n = 1) Hybrid Assistive Limb (HAL) [ 46 ] and one (n = 1) Atalante [ 56 ]. In 15 studies, the rehabilitation protocol included only exoskeleton gait and/or balance training [ 6 , 28 , 41 , 42 , 43 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 56 , 57 ]. Four studies included exoskeleton training associated with overground walking without body weight supported (BWS) [ 44 ], FES cycling [ 45 ], BWS [ 46 ], or knee-ankle–foot orthosis (KAFO) gait training [ 55 ]. In respect to the dose and dosage parameters, the total number of sessions reported ranged from 10 to 102 sessions. The number of sessions per week varied from 2 to 5 sessions. The duration of the total intervention ranged from 2 to 34 weeks. The duration of each gait and balance exoskeleton gait training varied from 30 to 90-min (one paper did not report [ 28 ]). The most frequent dose and dosage parameters were: 60-min sessions [ 42 , 43 , 46 , 48 , 49 , 51 , 52 , 54 , 56 ], 3 sessions a week [ 6 , 28 , 42 , 44 , 45 , 47 , 48 , 49 , 51 , 54 , 57 ], over 8 to 12 weeks [ 6 , 28 , 43 , 44 , 47 , 49 , 52 , 54 ], for a total of 20–40 sessions [ 6 , 28 , 44 , 45 , 47 , 49 , 50 , 54 , 55 ].

Overall, considering the dose and dosage parameter averages across all studies included in this review, regardless of clinically relevant change, a protocol with 60-min individual sessions, 3 times a week, for 9 weeks is suggested for a total of 27 sessions. The mean and standard deviation, or median and interquartile range for overall interventions and for protocols focused on specific therapeutic intent (e.g., functional restoration or cardiorespiratory rehabilitation) are described in Table  3 . As for the total number of sessions and the duration of interventions recommended, most studies showed variability within a range of (24–36 sessions) and (8–12 weeks), respectively [ 6 , 28 , 44 , 47 , 49 , 51 , 52 , 54 ]. Also, the duration of each session (60-min) and weekly frequency (3 times a week) were mostly consistent across the reviewed dataset, including studies with clinically relevant changes [ 6 , 28 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 51 , 52 , 54 , 56 ].

The gait and balance outcome measures used include: the 6MWT [ 6 , 41 , 42 , 44 , 45 , 47 , 48 , 49 , 50 , 52 , 53 , 54 , 55 , 57 ], 10MWT [ 6 , 28 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 56 , 57 ], TUG [ 28 , 44 , 45 , 47 , 50 , 57 ], WISCI-II [ 28 , 44 , 45 , 53 ], gait speed [ 43 , 46 , 50 , 52 ], steps taken [ 46 , 51 , 52 ], BBS [ 28 ], step length [ 46 ], stride length [ 50 ], Hoffer Walking Ability [ 53 ], and one paper adapted the 6MWT to 30-min walk test to evaluate gait function during 30-min [ 55 ], the frequency of the gait and balance outcomes across the studies is indicated in the Fig.  3 . Other non-gait related measures reported as main outcomes across different studies were categorized as either cardiorespiratory or physiologic outcomes and are listed in Fig.  3 .

Frequency of clinical outcomes reported. Heat map presenting the frequency of clinical outcomes measures reported, per studies by manufacturer. %HRR percentage of heart rate reserve, CO2 carbon dioxide, FEF forced expiratory flow, FEV1 forced expiratory volume in 1 s; LEMS Lower Extremities Motor Score, MVV maximum voluntary ventilation, NASA-TLX NASA Task Load Index, NBD neurogenic bladder dysfunction, PCI Physiological Cost Index, PEF peak expiratory flow, PGI-I Patient Global Impression of Improvement; Resp. respiratory, RPE rating of perceived exertion, SCATS Spinal Cord Assessment Tool for Spastic Reflexes, SCIM-II Spinal Cord Independence Measure II, UEMS Upper Extremities Motor Score, VAS Visual Analogue Scale, WISCI-II Walking Index for Spinal Cord Injury II

Protocol therapeutic intent

The studies included in this systematic review of overground exoskeleton training dose and dosage were classified in two groups according to the inferred therapeutic intent based on the described study design which addressed: functional restoration [ 6 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 53 , 56 , 57 , 58 ] or cardiorespiratory rehabilitation [ 41 , 42 , 43 , 48 , 54 , 55 ]. The therapeutic intent was determined based on each study’s primary research question, aim and main outcome measures in reference to motor (gait or balance) or cardiorespiratory performance, respectively. Although in recent years changes in body composition (e.g., muscle and bone mineral density) have been increasingly associated with exoskeleton training [ 31 , 59 , 60 , 61 ], none of the studies included in this review focused on anatomical adaptations in response to overground exoskeleton training.

Functional restoration

Thirteen (n = 13) studies focused on functional restoration [ 6 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 53 , 56 , 57 , 58 ]. Of those, eleven reported statistically significant improvements and/or showed improvements equal or higher than the MCID for gait and/or balance outcome measures [ 6 , 28 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 56 ]. Table 4 summarizes the individual studies’ aims and main results. The functional restoration protocols ranged from 10 to 51.5 sessions, 2 to 5 sessions a week, 3 to 12 weeks of duration for 45- to 90-min. Considering the studies with significant motor improvement (n = 11), it is suggested that a protocol aimed towards functional restoration would encompass 60-min individual sessions carried 3 times a week, over 8 weeks for a total of 24 sessions (Table  3 ).

Functional restoration interventions were shorter than cardiorespiratory interventions. They included subacute or chronic SCI/D patients, mostly with complete or incomplete thoracolumbar lesions. In this group analysis, two manuscripts did not report improvements [ 53 , 57 ]. The first one [ 53 ] reported the effects of a new robotic exoskeleton based on ten 30-min sessions over 2 weeks, that is shorter than the period suggested by our recommendation based on studies with significant functional restoration gains. The second study [ 57 ] focused on describing the protocol performed in a rehabilitation research institute, including the process of participant recruitment, fitting, donning, standing, standing balance, walking, mobility training, sitting and doffing. The functional outcomes, however, were measured only after the intervention. Additionally, among the respective study participants, individuals underwent 12 to 102 sessions over 4 to 34 weeks in remarkably variable study designs.

Regarding the therapeutic content, studies on functional restoration mainly focused on sit to stand, and stand to sit transitions, standing balance and walking training for significant changes or improvements above the MCID as per functional restoration outcome measures. The frequency of training, total number of trainings, therapy content (exercise training) and studies with significant changes are shown in Table  5 .A.

Cardiorespiratory rehabilitation

The six studies (n = 6) focused on cardiorespiratory rehabilitation [ 41 , 42 , 43 , 48 , 54 , 55 ] showed significant improvement of cardiorespiratory function. Table 6 summarizes the cardiorespiratory studies’ aims and main results. Cardiorespiratory-centered interventions ranged from 16 to 72 sessions, 2 to 5 sessions weekly, for 4 to 24 weeks. Individual sessions lasted between 55 to 90 min. Because the six protocols yielded significant improvement in cardiorespiratory function, it is suggested that interventions to that end are likely to succeed when based on 60-min sessions carried 3 times a week for 12 weeks in a total of 36 sessions (Table  3 ).

Unexpectedly, protocols focusing on cardiorespiratory outcomes were longer in average than protocols for functional restoration. Conversely, four of the referred articles also reported significant improvements in gait and balance measures [ 42 , 48 , 54 , 55 ], while two manuscripts reported improvements in cardiorespiratory outcomes alone [ 41 , 43 ]. The latter studies were based on 16 sessions over 4 weeks [ 41 ] and 17 sessions over 9 weeks [ 43 ], indicating that shorter interventions could be enough to improve cardiorespiratory function alone, that is uncoupled from significant functional restoration. In this case, the cardiorespiratory recommendation would include 60-min sessions carried out 3 times a week for 6 weeks for a total of 18 sessions.

Regarding the therapeutic content, studies on cardiorespiratory rehabilitation mainly focused on walking training and sit to stand and/or stand to sit transitions for significant improvements in cardiorespiratory outcome measures. The frequency of training, total number of trainings, therapy content and studies with significant changes in cardiorespiratory parameters are shown in Table  5 .B.

This review aimed to identify the dose and dosage parameters of exoskeleton-based exercises for overground gait and balance training in individuals with SCI/D. Although previous studies have discussed this topic in different neurological populations [ 7 , 62 ], to the best of our knowledge, this is the first review to prioritize the investigation and discussion of dose and dosage of overground exoskeleton therapy among individuals with SCI/D – a need repeatedly acknowledged in recent literature [ 7 , 15 , 21 , 35 ] yet widely overlooked as a primary research topic. We have summarized evidence from 19 manuscripts to determine current training parameters for specific therapeutic indications to inform best practice recommendations in exoskeleton-based SCI/D rehabilitation. Of 19 manuscripts, seventeen [ 6 , 28 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 54 , 55 , 56 ] reported statistically significant improvements and/or gains above the MCID in the gait, balance, cardiorespiratory and/or related physiological outcomes they assessed. The evidence gathered supports the assumption that exoskeletons are a promising therapeutic tool in SCI/D, particularly for functional restoration [ 6 , 28 , 44 , 45 , 46 , 47 , 49 , 50 , 51 , 52 , 56 ] and/or cardiorespiratory improvement [ 41 , 42 , 43 , 48 , 54 , 55 ].

Protocol design

Based on strict adherence to the systematic review inclusion criteria, several manuscripts initially screened did not fully report dose (total number of sessions, and duration of the session) and dosage (frequency per week, and duration of the intervention) parameters and were excluded. Consistent with previous reviews [ 35 , 62 ] on exoskeleton-based gait rehabilitation the lack of dose and dosage parameters ultimately limits the replication and generalizability of the outcomes reported. The absence of dose and dosage information also limits the translation of findings to evidence-based clinical practice, whereas the requirement for routine universal reports of dosing parameters in future studies would foster knowledge dissemination and implementation of precision rehabilitation approaches in the field. To support the development of future studies with structured information for better clinical translation, a checklist for reporting exoskeleton therapy is proposed in Table  7 .

We observed considerable inconsistency of protocols for exoskeleton-based gait and balance training, with substantial variability in dose and dosage parameters used. Indeed, the protocols ranged from 10 [ 46 ] to 102 [ 57 ] sessions over 2 to 34 weeks, two [ 53 ] to five [ 55 ] times a week, with individual sessions lasting from 30 [ 53 ] to 90 min [ 47 ]. Further, the systematic review results indicated that the weekly frequency and session duration are the most consistent parameters, with most protocols reporting 3 sessions a week [ 6 , 28 , 42 , 44 , 45 , 47 , 48 , 49 , 51 , 54 , 57 ] at an average of 60 min per session [ 42 , 43 , 46 , 48 , 49 , 51 , 52 , 54 , 56 ]. Another important variable across studies was the device used and the exoskeleton manufacturer, with Ekso (n = 7) [ 28 , 42 , 43 , 44 , 45 , 50 , 51 ] and ReWalk (n = 6) [ 6 , 48 , 49 , 52 , 55 , 57 ] being the most used devices.

Protocol effectiveness

Training effectiveness (e.g., changes in assessment values at or above the MCID) is shaped by multiple factors beyond dose and dosage, including but not limited to device parameters and the extent or intensity of training. Relative to exoskeleton-based gait and balance rehabilitation such factors include device assistance and resistance levels, different walking patterns (e.g., step and stride length, width of base of support, gait speed and step cadence) as well as exercise intensity. The latter is associated with a lack of specific consensus-based measures and definitions universally adopted by experts in the field of neurorehabilitation [ 63 ]. This is particularly true in the SCI/D populations among whom there is substantial heterogeneity in neurological impairment, and associated variability in prognosis and responsiveness to exoskeleton interventions. As a result, variability in prognosis and responsiveness are commonly observed and personalized prescriptions are provided in the absence of consensus-based terminology and practices [ 64 , 65 ]. In this context, the best practice recommendations derived from this systematic review are valid given the reporting of whether the participants achieved a clinically meaningful change in function/assessment parameter based on the dose and dosage reported despite the lack of data specifying exercise intensity.

Despite protocol variability, including that of device choice, therapeutic intent, and training intensity, it is possible that the significant changes reported are associated with the repeated exposure to active standing time versus non-active sitting time [ 65 ]. However, most studies included similar functional therapeutic activities (e.g., sit to stand transitions, standing and balance training and walking training). In fact, exercise intensity in robotic rehabilitation, although not standardized, is often associated with the number of repetitions (e.g., step count), step frequency and total walking distance. The modulation of intensity on a case-by-case basis likely favored the observed performance improvements across the multiple protocol designs reported in this review. Future studies reporting the therapeutic benefits of exoskeleton therapy should include the therapeutic indication, device choice and parameters, exercise intensity, and the dose and dosage parameters as means to improve precision rehabilitation – particularly among people living with a spinal cord impairment and multimorbidity [ 66 ].

Injury characteristics

In addition to exercise parameters, the influence of injury characteristics on exoskeleton-based SCI/D rehabilitation is very likely, yet controversial. Benson et al. [ 67 ] reported that individuals with complete injuries showed greater improvement in walking speed than incomplete injured pairs. That may be because participants with incomplete lesions were functional walkers before the beginning of their training, benefiting mostly from the ability to walk longer distances with exoskeletons as opposed to participants with complete injuries to whom exoskeletons allowed not only orthostatism, but gait initiation and speed improvements. In agreement with those findings, Xiang et al. [ 53 ] reported that individuals with higher spinal lesions and motor complete injuries showed greater improvement in gait and functional outcomes (gait speed and 6MWT) while using exoskeletons compared to people who were functional walkers with lower and or incomplete lesions. Conversely, it has been reported that adults living with lower neurological level of injury (complete versus incomplete) can achieve significantly faster walking speeds following exoskeleton training [ 6 , 68 , 69 ]. The explanation of these findings may be linked to the fact that people with complete SCI/D obtain more remarkable gains with training (e.g., from no standing to walking), although they still walk slower than individuals with incomplete lesions [ 53 , 67 ]. Differences in gait speed is possibly associated with the remaining neural pathways in individuals with incomplete lesions, which foster better neurorecovery in response to functional restoration strategies [ 2 ]. This assumption agrees with Louie et al.’s [ 17 ] report that walking speed with exoskeletons is positively correlated with the level of spinal injury (coded from 0 (cervical) to 17 (lumbar)) and training duration. Thus, lower injuries and longer training could, favor greater locomotor gains for individuals with SCI/D. Nevertheless, Sale et al. [ 50 ] reported that exoskeleton rehabilitation is safe and feasible across a heterogeneous sample of persons with SCI/D provided it is tailored to their personal needs. Further, it is plausible that there may be additional therapeutic benefits of longitudinal training not addressed in this review.

Exoskeleton-based therapeutic intent and physiological considerations

Upon review of the nineteen manuscripts included, consistent similarities across some of the protocols in terms of their therapeutic goals led us to classify the studies in two categories of therapeutic intent (e.g., functional restoration and cardiorespiratory rehabilitation). While the clinical purpose of individual studies seemed distinguishable enough for us to categorize them, that was not explicitly disclosed by the authors.

The current knowledge of the physiological mechanisms involved in exoskeleton-based therapies remains limited. A prior review reported that neurophysiological responses in exoskeleton recovery are linked to the exploitation of neuroplasticity, sensory stimulation, and coordination of limb and muscle activation during the training. The authors purport that functional restoration and neurorecovery are much like a relearning process where preserved sensorimotor and neural circuits are engaged to promote recovery [ 2 ]. For cardiorespiratory function, exoskeleton gait training’s rationale for the observed improvements in function associated with stimulation of the cardiorespiratory system and activation of the lower limbs is due to an increase in metabolic rate indicating this is an effective way of increasing energy expenditure with consequent improvements of cardiorespiratory fitness. Moreover, exoskeleton training contributes to the augmentation of end-systolic and end-diastolic volume, cardiac output, ventricular mass and reduces heart rate following cardiovascular conditioning [ 42 , 54 , 70 ].

Our findings suggest that different exercise exposures are needed to achieve MCID as per therapeutic intent in SCI/D rehabilitation, with cardiorespiratory changes demanding longer protocols compared to functional restoration. Nevertheless, we hypothesized that shorter interventions would be warranted for cardiorespiratory gains due to faster cardiovascular adaptation to structured exercises compared to neurological responses [ 71 , 72 ]. This unexpected outcome may be related to two cardiorespiratory-focused manuscripts in which participants underwent longer interventions (72 [ 42 ] and 60 [ 48 ] sessions) to evaluate changes over the time (early, mid and late changes), justifying the longer experimental designs. Additionally, of the six studies [ 41 , 42 , 43 , 48 , 54 , 55 ] included in cardiorespiratory rehabilitation, two [ 41 , 43 ] had significant improvement in cardiorespiratory function but not in gait, which was achieved with shorter interventions, in line with our initial hypothesis. Supporting our hypothesis, Faulkner et al. [ 73 ] reported that exoskeleton gait training associated with conventional physiotherapy in 5 sessions over a single week improved cardiovascular health, by reducing the augmentation index and mean arterial pressure. Further, Evans et al. [ 42 ] reported statistically significant increases in cardiovascular efficiency as early as 6 weeks after exoskeleton gait training. Interestingly, despite protocol duration variability, the six articles focused on cardiorespiratory training reported significant improvements in cardiorespiratory health as per increased oxygen consumption, heart rate and metabolic equivalent, in addition to reduced perception of effort and oxygen cost [ 41 , 42 , 43 , 48 , 54 , 55 ]. A prior systematic review reported that exoskeleton gait training elevates the energy expenditure, while allowing participants to exercise at moderate intensity, further indicating exoskeletons are beneficial for cardiorespiratory training [ 19 ].

In SCI/D, reduced lower-limb weight bearing and other health complications contribute to the loss of muscle mass and bone mineral density (BMD), specially below the level of injury [ 74 ]. This leads to an increased risk of fragility fractures, which should be accounted for when performing exoskeleton-based gait training. That is important due to previous reports of lower limb fragility fracture after exoskeleton use, mainly induced by the effect of gravity and pressure points created by the resistance of the equipment against the user’s body [ 75 , 76 ]. Thus, people living with SCI/D should be advised of their fracture risk, prior to using wearable exoskeletons for increased safety, regional improvements in bone strength and BMD [ 59 ]. To prevent fragility fractures, Bass et al. [ 59 ] developed a volume and progression algorithm based on BMD thresholds. Accordingly, individuals with osteoporotic profile (T-score ≤ -2.5) should be exposed to a slow-progression program, individuals with osteopenic profile (-2.5 < T-Score < -1.0) should start with moderate-progression and individuals with preserved BMD profile (T-Score ≥ -1.0) should be enrolled in a fast-progression walking program. It is worth noting that as per the position statement 4 in the International Society for Clinical Densitometry, there is no established threshold BMD value below which weight-bearing activities are absolutely contra-indicated, and that BMD and clinical risk factors should be used together on a case-to-case basis to assess risk exposure [ 74 ]. Furthermore, people living with SCI/D are in a higher risk of developing skin abrasions and tissue injury [ 77 ]. Many studies have reported skin abrasions after the use of exoskeleton in SCI/D population [ 44 , 53 , 56 , 58 ]. The reduction of physical activity levels, immobilization, changes in circulation and microcirculation, sensory loss, skin compression due to positioning and impaired venous return are aspects of injury that preclude individuals to lower extremity abrasions [ 77 , 78 ]. Also, participants with sensory impairment are at greater risk of developing skin lesions [ 79 ], and hence warrant ongoing screening for skin integrity. That is particularly true at points of higher pressure caused the interface between the skin and the exoskeleton [ 44 , 58 , 79 ].

Considerations for translation to practice

Recommendations from systematic reviews are extremely helpful at informing new research designs and guiding the translation of optimal evidence-based findings to clinical practice. However, it is also true that best practice recommendations, as identified by this review cannot always be implemented, particularly considering contextual disparities, including different countries (e.g., North America, Europe and Asia, Fig.  2 ), devices and therapeutic intent. Should a clinician find the implementation of the suggested best practice recommendations infeasible, reproducing the observed dose and dosage of therapy with a specific device can be limited to the shortest study with reported clinical effectiveness above the MCID for the outcome of interest (see the reduced dose and dosage but observed MCID with specific interventions on Table  5 ). For instance, ten 60-min sessions at a frequency of 5 sessions per week over two weeks yielded significant improvements in functional restoration [ 46 ]. Alternatively, sixteen 50–60-min sessions at a frequency of 4 times a week over four weeks yielded significant improvements in cardiorespiratory function [ 41 ]. We also suggest that patients be supported to work incrementally with healthcare providers to further implement best practice dose and dosage recommendations.

Study limitations

This study has limitations that include the relatively scarce literature available, which did not allow us to analyze the results according to the participant’s characteristics (sex, ASIA Impairment Scale, neurologic level of injury, etc.). However, the population described in this review are similar to those described in prior reviews among individuals living with SCI/D [ 17 , 62 ]. Also, it is important to state that the implementation of exoskeleton-based interventions is still limited due to the cost, availability of the equipment, equipment specifications and limitations, and the lack of highly trained staff to support exoskeleton-based therapy [ 80 , 81 , 82 ]. As for the limited study sample size, our search was broadened to identify manuscripts applying overground exoskeletons in SCI/D, but many of the identified references did not fully report dose and dosage – that is at least 3 parameters – and were excluded in a strategy that reduced the already restricted sample, but guaranteed data consistency. Additionally, the references included in this systematic review were classified according to their clinical intent by the review authors, which may not reflect the original authors’ intent. Furthermore, the study quality and risk of bias were not assessed as our search aimed to perform a comprehensive overview of dose and dosage in exoskeleton gait and balance training in SCI/D. Nevertheless, this systematic review is consistent with prior reports in the literature that did not report risk of bias in studies involving exoskeleton rehabilitation [ 7 , 21 , 35 , 62 ]. The exoskeleton device donning and doffing times were inconsistently reported across the reviewed studies, with only two of them [ 6 , 44 ] indicating that donning and doffing times were not part of the reported session duration and a single study [ 43 ] indicating that the session duration included donning and doffing. While we believe that some of the other sixteen studies included donning and doffing times in the session duration, we presume that most studies reported the time dedicated to standing/walking training apart from donning and doffing. Altogether, we encourage readers to implement the enclosed practice recommendations and to report device donning and doffing times, device parameters and therapeutic intensity in future reports. We also encourage clinicians and investigators to describe barriers and facilitators to implementation of best practices in different contexts.

Conclusions

In summary, this systematic review advances the understanding of overground exoskeleton-based gait and balance training in SCI/D and its role in facilitating functional recovery and or cardiorespiratory fitness. The review results provide evidence-based clinical practice recommendations, which are tailored to the therapeutic intent of the intervention. However, problems with inconsistent reporting of exoskeleton training dose and dosage and the heterogeneity of study designs among adults with SCI/D preclude fulsome dissemination of data and are acknowledged as important limitations. To advance the field of exoskeleton rehabilitation in SCI/D and increase research quality, there is an urgent need to standardize clinical practice recommendations and guidelines through well-structured studies with clear indications of their therapeutic intent. Finally, we highlight the need for multicentre studies, which could validate the therapeutic effectiveness of specific dose and dosage parameters for optimal gait and balance rehabilitation among adults with SCI/D based on poling of data from multiple sites and contexts.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

10-Meter Walk Test

6-Minute walk test

Berg Balance Scale

Bone mineral density

Body weight supported

Body weight supported treadmill training

Functional electrical stimulation

Hybrid Assistive Limb

Knee-ankle–foot orthosis

Minimal clinically important difference

Preferred Reporting Items Systematic Reviews and Meta-Analyses

International Prospective Register of Systematic Reviews

Spinal cord injury/disease

Time Up and Go

Walking Index SCI II

Reinkensmeyer DJ, Dietz V. Neurorehabilitation technology. New York: Springer; 2016.

Book   Google Scholar  

Gassert R, Dietz V. Rehabilitation robots for the treatment of sensorimotor deficits: a neurophysiological perspective. J Neuroeng Rehabil. 2018;15(1):46.

Article   PubMed   PubMed Central   Google Scholar  

Geonea ID, Tarnita D. Design and evaluation of a new exoskeleton for gait rehabilitation. Mech Sci. 2017;8(2):307–21.

Article   Google Scholar  

Molteni F, Gasperini G, Cannaviello G, Guanziroli E. Exoskeleton and end-effector robots for upper and lower limbs rehabilitation: narrative review. PM&R. 2018;10(9):S174–88.

Google Scholar  

Hesse S, Uhlenbrock D. A mechanized gait trainer for restoration of gait. J Rehabil Res Dev. 2000;37(6):701–8.

CAS   PubMed   Google Scholar  

Esquenazi A, Talaty M, Packel A, Saulino M. The rewalk powered exoskeleton to restore ambulatory function to individuals with thoracic-level motor-complete spinal cord injury. Am J Phys Med Rehabil. 2012;91(11):911–21.

Article   PubMed   Google Scholar  

Rodríguez-Fernández A, Lobo-Prat J, Font-Llagunes JM. Systematic review on wearable lower-limb exoskeletons for gait training in neuromuscular impairments. J Neuroeng Rehabil. 2021;18(1):22.

Russo M, Maggio MG, Naro A, Portaro S, Porcari B, Balletta T, et al. Can powered exoskeletons improve gait and balance in multiple sclerosis? A retrospective study. Int J Rehab Res. 2021;44(2):126–30.

Nam KY, Kim HJ, Kwon BS, Park J, Lee HJ, Yoo A. Robot-assisted gait training (Lokomat) improves walking function and activity in people with spinal cord injury: a systematic review. J Neuroeng Rehabil. 2017;14(1):24.

Scivoletto G, Tamburella F, Laurenza L, Torre M, Molinari M. Who is going to walk? A review of the factors influencing walking recovery after spinal cord injury. Front Human Neurosci. 2014;8:141.

Fehlings MG, Tetreault LA, Wilson JR, Kwon BK, Burns AS, Martin AR, et al. A clinical practice guideline for the management of acute spinal cord injury: introduction, rationale, and scope. Global Spine J. 2017;7(3 Suppl):84S-94S.

Hachem LD, Fehlings MG. Pathophysiology of spinal cord injury. Neurosurg Clin N Am. 2021;32(3):305–13.

Sandean D. Management of acute spinal cord injury: a summary of the evidence pertaining to the acute management, operative and non-operative management. World J Orthop. 2020;11(12):573–83.

Ahuja CS, Wilson JR, Nori S, Kotter M, Druschel C, Curt A, et al. Traumatic spinal cord injury. Nat Rev Dis Primers. 2017;3:17018.

Gandolfi M, Valè N, Posteraro F, Morone G, Dell’orco A, Botticelli A, et al. State of the art and challenges for the classification of studies on electromechanical and robotic devices in neurorehabilitation: a scoping review. Eur J Phys Rehabil Med. 2021;57(5):831–40.

Basalp E, Wolf P, Marchal-Crespo L. Haptic training: which types facilitate (re) learning of which motor task and for whom? Answers by a review. IEEE Trans Haptics. 2021;14(4):722–39.

Louie DR, Eng JJ, Lam T. Gait speed using powered robotic exoskeletons after spinal cord injury: a systematic review and correlational study. J Neuroeng Rehabil. 2015;12:82.

Fang C, Tsai J, Li G, Lien AS, Chang Y. Effects of robot-assisted gait training in individuals with spinal cord injury: a meta-analysis. Biomed Res Int. 2020;2020:2102785.

Duddy D, Doherty R, Connolly J, McNally S, Loughrey J, Faulkner M. The effects of powered exoskeleton gait training on cardiovascular function and gait performance: A systematic review. Sensors. 2021;21(9):3207.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Cheung EY, Ng TK, Kevin KK, Kwan RL, Cheing GL. Robot-assisted training for people with spinal cord injury: a meta-analysis. Arch Phys Med Rehabil. 2017;98(11):2320-2331.e12.

Mekki M, Delgado AD, Fry A, Putrino D, Huang V. Robotic rehabilitation and spinal cord injury: a narrative review. Neurotherapeutics. 2018;15(3):604–17.

Jacobson PB, Goody R, Lawrence M, Mueller BK, Zhang X, Hooker BA, et al. Elezanumab, a human anti-RGMa monoclonal antibody, promotes neuroprotection, neuroplasticity, and neurorecovery following a thoracic hemicompression spinal cord injury in non-human primates. Neurobiol Dis. 2021;155: 105385.

Article   CAS   PubMed   Google Scholar  

Lin A, Shaaya E, Calvert JS, Parker SR, Borton DA, Fridley JS. A review of functional restoration from spinal cord stimulation in patients with spinal cord injury. Neurospine. 2022;19(3):703–34.

James ND, McMahon SB, Field-Fote EC, Bradbury EJ. Neuromodulation in the restoration of function after spinal cord injury. Lancet Neurol. 2018;17(10):905–17.

Yildirim MA, Öneş K, Gökşenoğlu G. Early term effects of robotic assisted gait training on ambulation and functional capacity in patients with spinal cord injury. Turk J Med Sci. 2019;49(3):838–48.

Tamburella F, Tagliamonte NL, Masciullo M, Pisotta I, Arquilla M, van Asseldonk E, et al. Gait training with Achilles ankle exoskeleton in chronic incomplete spinal cord injury subjects. J Biol Regul Homeost Agents. 2020;34(5 Suppl. 3):147–64.

Okawara H, Sawada T, Matsubayashi K, Sugai K, Tsuji O, Nagoshi N, et al. Gait ability required to achieve therapeutic effect in gait and balance function with the voluntary driven exoskeleton in patients with chronic spinal cord injury: a clinical study. Spinal Cord. 2020;58(5):520–7.

Bach Baunsgaard C, Vig Nissen U, Katrin Brust A, Frotzler A, Ribeill C, Kalke Y, et al. Gait training after spinal cord injury: safety, feasibility and gait function following 8 weeks of training with the exoskeletons from Ekso Bionics. Spinal Cord. 2018;56(2):106–16.

Holanda LJ, Silva PM, Amorim TC, Lacerda MO, Simão CR, Morya E. Robotic assisted gait as a tool for rehabilitation of individuals with spinal cord injury: a systematic review. J Neuroeng Rehabil. 2017;14(1):126.

Lefeber N, Swinnen E, Kerckhofs E. The immediate effects of robot-assistance on energy consumption and cardiorespiratory load during walking compared to walking without robot-assistance: a systematic review. Disab Rehabil Assist Technol. 2017;12(7):657–71.

Karelis AD, Carvalho LP, Castillo MJ, Gagnon DH, Aubertin-Leheudre M. Effect on body composition and bone mineral density of walking with a robotic exoskeleton in adults with chronic spinal cord injury. J Rehabil Med. 2017;49(1):84–7.

Choi H, Kim G, Chai JH, Ko C. Effect of gait training program with mechanical exoskeleton on body composition of paraplegics. J Multidiscip Healthc. 2020;13:1879.

Asselin P, Cirnigliaro CM, Kornfeld S, Knezevic S, Lackow R, Elliott M, et al. Effect of Exoskeletal-Assisted Walking on Soft Tissue Body Composition in Persons With Spinal Cord Injury. Arch Phys Med Rehabil. 2021;102(2):196–202.

Gorgey AS, Poarch H, Harnish C, Miller JM, Dolbow D, Gater DR. Acute effects of locomotor training on neuromuscular and metabolic profile after incomplete spinal cord injury. NeuroRehabilitation. 2011;29(1):79–83.

Tan K, Koyama S, Sakurai H, Teranishi T, Kanada Y, Tanabe S. Wearable robotic exoskeleton for gait reconstruction in patients with spinal cord injury: A literature review. J Orthop Translat. 2021;28:55–64.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Int J Surg. 2021;88: 105906.

Academy of Neurologic Physical Therapy. 6 minute walk test (6MWT). 2019. https://www.neuropt.org/docs/default-source/cpgs/core-outcome-measures/6mwt-pocket-guide-proof9.pdf?sfvrsn=9ee25043_0 . Accessed 12 Mar 2023.

Ryan S. 6 minute walk test. 2013. https://www.sralab.org/rehabilitation-measures/6-minute-walk-test . Accessed 12 Mar 2023.

Physiopedia. 10 metre walk test. 2022. https://www.physio-pedia.com/10_Metre_Walk_Test . Accessed 12 Mar 2023.

Ryan S. Time up and go. 2013. https://www.sralab.org/rehabilitation-measures/timed-and-go . Accessed 12 Mar 2023.

Xiang X, Zong H, Ou Y, Yu X, Cheng H, Du C, et al. Exoskeleton-assisted walking improves pulmonary function and walking parameters among individuals with spinal cord injury: a randomized controlled pilot study. J Neuroeng Rehabil. 2021;18(1):86.

Evans RW, Shackleton CL, West S, Derman W, Laurie Rauch H, Baalbergen E, et al. Robotic locomotor training leads to cardiovascular changes in individuals with incomplete spinal cord injury over a 24-week rehabilitation period: a randomized controlled pilot study. Arch Phys Med Rehabil. 2021;102(8):1447–56.

Corbianco S, Cavallini G, Dini M, Franzoni F, D’Avino C, Gerini A, et al. Energy cost and psychological impact of robotic-assisted gait training in people with spinal cord injury: effect of two different types of devices. Neurol Sci. 2021;42(8):3357–66.

Edwards DJ, Forrest G, Cortes M, Weightman MM, Sadowsky C, Chang S, et al. Walking improvement in chronic incomplete spinal cord injury with exoskeleton robotic training (WISE): a randomized controlled trial. Spinal Cord. 2022;60(6):522–32.

Stampacchia G, Olivieri M, Rustici A, Avino C, Gerini A, Mazzoleni S. Gait rehabilitation in persons with spinal cord injury using innovative technologies: an observational study. Spinal Cord. 2020;58(9):988–97.

Puentes S, Kadone H, Kubota S, Abe T, Shimizu Y, Marushima A, et al. Reshaping of gait coordination by robotic intervention in myelopathy patients after surgery. Front Neurosci. 2018;12:99.

Tefertiller C, Hays K, Jones J, Jayaraman A, Hartigan C, Bushnik T, et al. Initial outcomes from a multicenter study utilizing the indego powered exoskeleton in spinal cord injury. Top Spinal Cord Inj Rehabil. 2018;24(1):78–85.

Knezevic S, Asselin PK, Cirnigliaro CM, Kornfeld S, Emmons RR, Spungen AM. Oxygen uptake during exoskeletal-assisted walking in persons with paraplegia. Arch Phys Med Rehabil. 2021;102(2):185–95.

Guanziroli E, Cazzaniga M, Colombo L, Basilico S, Legnani G, Molteni F. Assistive powered exoskeleton for complete spinal cord injury: correlations between walking ability and exoskeleton control. Eur J Phys Rehabil Med. 2019;55(2):209–16.

Sale P, Russo EF, Scarton A, Calabro RS, Masiero S, Filoni S. Training for mobility with exoskeleton robot in spinal cord injury patients: a pilot study. Eur J Phys Rehabil Med. 2018;54(5):745–51.

Gagnon DH, Escalona MJ, Vermette M, Carvalho LP, Karelis AD, Duclos C, et al. Locomotor training using an overground robotic exoskeleton in long-term manual wheelchair users with a chronic spinal cord injury living in the community: Lessons learned from a feasibility study in terms of recruitment, attendance, learnability, performance and safety. J Neuroeng Rehabil. 2018;15(1):12.

Khan AS, Livingstone DC, Hurd CL, Duchcherer J, Misiaszek JE, Gorassini MA, et al. Retraining walking over ground in a powered exoskeleton after spinal cord injury: a prospective cohort study to examine functional gains and neuroplasticity. J Neuroeng Rehabil. 2019;16:145.

Xiang X, Ding M, Zong H, Liu Y, Cheng H, He C, et al. The safety and feasibility of a new rehabilitation robotic exoskeleton for assisting individuals with lower extremity motor complete lesions following spinal cord injury (SCI): an observational study. Spinal Cord. 2020;58(7):787–94.

Park JH, Kim HS, Jang SH, Hyun DJ, Park SI, Yoon J, et al. Cardiorespiratory responses to 10 weeks of exoskeleton-assisted overground walking training in chronic nonambulatory patients with spinal cord injury. Sensors (Basel, Switzerland). 2021;21(15):5022.

Kwon SH, Lee BS, Lee HJ, Kim EJ, Lee JA, Yang SP, et al. Energy efficiency and patient satisfaction of gait with knee-ankle-foot orthosis and robot (ReWalk)-assisted gait in patients with spinal cord injury. Ann Rehabil Med. 2020;44(2):131–41.

Kerdraon J, Previnaire JG, Tucker M, Coignard P, Allegre W, Knappen E, et al. Evaluation of safety and performance of the self balancing walking system Atalante in patients with complete motor spinal cord injury. Spinal Cord Ser Cases. 2021;7(1):71.

Asselin PK, Avedissian M, Knezevic S, Kornfeld S, Spungen AM. Training persons with spinal cord injury to ambulate using a powered exoskeleton. J Vis Exp. 2016;112:54071.

Baunsgaard CB, Nissen UV, Brust AK, Frotzler A, Ribeill C, Kalke Y, et al. Exoskeleton gait training after spinal cord injury: an exploratory study on secondary health conditions. J Rehabil Med. 2018;50(9):806–13.

Bass A, Aubertin-Leheudre M, Morin SN, Gagnon DH. Preliminary training volume and progression algorithm to tackle fragility fracture risk during exoskeleton-assisted overground walking in individuals with a chronic spinal cord injury. Spinal Cord Ser Cases. 2022;8(1):29.

Bass A, Morin S, Guidea M, Lam J, Hammad R, Aubertin-Leheudre M, et al. Effects of an exoskeleton-assisted walking program on bone strength in wheelchair individuals with spinal cord injury: a preliminary study using imaging and serum biomarkers. Physiology. 2023;38(S1):5796080.

Chen S, Wang Z, Li Y, Tang J, Wang X, Huang L, et al. Safety and feasibility of a novel exoskeleton for locomotor rehabilitation of subjects with spinal cord injury: a prospective, multi-center, and cross-over clinical trial. Front Neurorobot. 2022;16: 848443.

Contreras-Vidal JL, Bhagat NA, Brantley J, Cruz-Garza JG, He Y, Manley Q, et al. Powered exoskeletons for bipedal locomotion after spinal cord injury. J Neural Eng. 2016;13(3): 031001.

Goikoetxea-Sotelo G, van Hedel HJ. Defining, quantifying, and reporting intensity, dose, and dosage of neurorehabilitative interventions focusing on motor outcomes. Front Rehabil Sci. 2023;4:1139251.

Hutchinson MJ, Goosey-Tolfrey VL. Rethinking aerobic exercise intensity prescription in adults with spinal cord injury: time to end the use of “moderate to vigorous” intensity? Spinal Cord. 2022;60(6):484–90.

Craven BC, Souza WH, Jaglal S, Gibbs J, Wiest MJ, Sweet SN et al. Reducing endocrine metabolic disease risk in adults with chronic spinal cord injury: strategic activities conducted by the Ontario-Quebec RIISC team. Disabil Rehabil. 2023:1–13. Ahead of print.

Bierman AS, Tinetti ME. Precision medicine to precision care: managing multimorbidity. Lancet. 2016;388(10061):2721–3.

Benson I, Hart K, Tussler D, van Middendorp JJ. Lower-limb exoskeletons for individuals with chronic spinal cord injury: findings from a feasibility study. Clin Rehabil. 2016;30(1):73–84.

Zeilig G, Weingarden H, Zwecker M, Dudkiewicz I, Bloch A, Esquenazi A. Safety and tolerance of the ReWalk™ exoskeleton suit for ambulation by people with complete spinal cord injury: a pilot study. J Spinal Cord Med. 2012;35(2):96–101.

Hartigan C, Kandilakis C, Dalley S, Clausen M, Wilson E, Morrison S, et al. Mobility outcomes following five training sessions with a powered exoskeleton. Top Spinal Cord Inj Rehab. 2015;21(2):93–9.

Escalona MJ, Brosseau R, Vermette M, Comtois AS, Duclos C, Aubertin-Leheudre M, et al. Cardiorespiratory demand and rate of perceived exertion during overground walking with a robotic exoskeleton in long-term manual wheelchair users with chronic spinal cord injury: a cross-sectional study. Ann Phys Rehabil Med. 2018;61(4):215–23.

Hellsten Y, Nyberg M. Cardiovascular adaptations to exercise training. 2015;6 1:1-32.

Škarabot J, Brownstein CG, Casolo A, Del Vecchio A, Ansdell P. The knowns and unknowns of neural adaptations to resistance training. Eur J Appl Physiol. 2021;121(3):675–85.

Faulkner J, Martinelli L, Cook K, Stoner L, Ryan-Stewart H, Paine E, et al. Effects of robotic-assisted gait training on the central vascular health of individuals with spinal cord injury: a pilot study. J Spinal Cord Med. 2021;44(2):299–305.

Morse LR, Biering-Soerensen F, Carbone LD, Cervinka T, Cirnigliaro CM, Johnston TE, et al. Bone mineral density testing in spinal cord injury: 2019 ISCD official position. J Clin Densitom. 2019;22(4):554–66.

Bass A, Morin SN, Vermette M, Aubertin-Leheudre M, Gagnon DH. Incidental bilateral calcaneal fractures following overground walking with a wearable robotic exoskeleton in a wheelchair user with a chronic spinal cord injury: is zero risk possible? Osteoporosis Int. 2020;31(5):1007–11.

Article   CAS   Google Scholar  

Van Herpen F, Van Dijsseldonk RB, Rijken H, Keijsers N, Louwerens J, van Nes I. Case report: description of two fractures during the use of a powered exoskeleton. Spinal Cord Ser Cases. 2019;5(1):99.

Sunn G. Spinal cord injury pressure ulcer treatment: an experience-based approach. Phys Med Rehabil Clin. 2014;25(3):671–80.

Huiming G, Yuming W, Mingliang Y, Changbin L, Qiuchen H, Jianjun L. Study on the characteristics of microcirculation in the site of pressure ulcer in patients with spinal cord injury. Sci Prog. 2021;104(3):00368504211028726.

Ozdemir RA, Perez MA. Afferent input and sensory function after human spinal cord injury. J Neurophysiol J Neurophysiol. 2018;119(1):134–44.

Hill D, Holloway CS, Ramirez DZM, Smitham P, Pappas Y. What are user perspectives of exoskeleton technology? A literature review. Int J Technol Assess Health Care. 2017;33(2):160–7.

Van Dijsseldonk RB, Van Nes IJ, Geurts AC, Keijsers NL. Exoskeleton home and community use in people with complete spinal cord injury. Sci Rep. 2020;10(1):15600.

Mertz L. The next generation of exoskeletons: lighter, cheaper devices are in the works. IEEE Pulse. 2012;3(4):56–61.

Download references

Acknowledgements

Patrik Nepomuceno acknowledges receipt of a scholarship through the Emerging Leaders in the Americas Program with the support of Global Affairs Canada and Government of Canada. Dr. Wagner H. Souza acknowledges fellowship support from the University Health Network (UHN) Foundation and Spinal Cord Injury Ontario, and inspiration from Mr. Robert MacDonald. Dr. Craven acknowledges support from the UHN Foundation for her UHN/University of Toronto Chair in SCI Rehabilitation.

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and affiliations.

KITE Research Institute, University Health Network, Toronto, ON, Canada

Patrik Nepomuceno, Wagner H. Souza, Maureen Pakosh, Kristin E. Musselman & B. Catharine Craven

Graduate Program in Health Promotion, Department of Health Sciences, University of Santa Cruz do Sul, Santa Cruz do Sul, RS, Brazil

Patrik Nepomuceno

Department of Physical Therapy, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada

Kristin E. Musselman

Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada

B. Catharine Craven

Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, Canada

Patrik Nepomuceno & B. Catharine Craven

Rehabilitation Sciences Institute, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada

Kristin E. Musselman & B. Catharine Craven

You can also search for this author in PubMed   Google Scholar

Contributions

PN, WHS, KEM and BCC contributed to conceptualization, methodology, design, and development of the study. PN and WHS share first authorship and equally contributed to data curation, acquisition, formal analysis and to the writing of the manuscript. MP contributed to data curation, acquisition and methodology. KEM and BCC contributed to supervision, review and editing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to B. Catharine Craven .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1:.

Medline Search Strategy. Contain the Medline Search Strategy.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Nepomuceno, P., Souza, W.H., Pakosh, M. et al. Exoskeleton-based exercises for overground gait and balance rehabilitation in spinal cord injury: a systematic review of dose and dosage parameters. J NeuroEngineering Rehabil 21 , 73 (2024). https://doi.org/10.1186/s12984-024-01365-2

Download citation

Received : 28 November 2023

Accepted : 22 April 2024

Published : 05 May 2024

DOI : https://doi.org/10.1186/s12984-024-01365-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Exoskeleton
• Neurorehabilitation
• Overground training
• Spinal cord injury

Journal of NeuroEngineering and Rehabilitation

ISSN: 1743-0003

• Submission enquiries: [email protected]

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• For authors
• New editors
• BMJ Journals More You are viewing from: Google Indexer

You are here

• Online First
• Best practices for the dissemination and implementation of neuromuscular training injury prevention warm-ups in youth team sport: a systematic review
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0009-0005-0529-0398 Destiny Lutz 1 ,
• http://orcid.org/0000-0001-6429-4333 Carla van den Berg 1 ,
• http://orcid.org/0000-0003-3056-8169 Anu M Räisänen 1 , 2 ,
• Isla J Shill 1 , 3 ,
• Jemma Kim 4 , 5 ,
• Kenzie Vaandering 1 ,
• Alix Hayden 6 ,
• http://orcid.org/0000-0002-0427-2877 Kati Pasanen 1 , 7 , 8 , 9 ,
• http://orcid.org/0000-0002-5951-5899 Kathryn J Schneider 1 , 3 , 8 , 9 , 10 ,
• http://orcid.org/0000-0002-9499-6691 Carolyn A Emery 1 , 3 , 8 , 9 , 11 , 12 , 13 ,
• http://orcid.org/0000-0002-5984-9821 Oluwatoyosi B A Owoeye 1 , 4
• 1 Sport Injury Prevention Research Centre, Faculty of Kinesiology , University of Calgary , Calgary , Alberta , Canada
• 2 Department of Physical Therapy Education - Oregon , Western University of Health Sciences College of Health Sciences - Northwest , Lebanon , Oregon , USA
• 3 Hotchkiss Brain Institute , University of Calgary , Calgary , Alberta , Canada
• 4 Department of Physical Therapy & Athletic Training , Doisy College of Health Sciences, Saint Louis University , Saint Louis , Missouri , USA
• 5 Interdisciplinary Program in Biomechanics and Movement Science , University of Delaware College of Health Sciences , Newark , Delaware , USA
• 6 Libraries and Cultural Resources , University of Calgary , Calgary , Alberta , Canada
• 7 Tampere Research Center for Sports Medicine , Ukk Instituutti , Tampere , Finland
• 8 McCaig Institute for Bone and Joint Health , University of Calgary , Calgary , Alberta , Canada
• 9 Alberta Chilrden's Hopsital Research Institute , University of Calgary , Calgary , Alberta , Canada
• 10 Sport Medicine Centre , University of Calgary , Calgary , Alberta , Canada
• 11 O'Brien Institute for Public Health , University of Calgary , Calgary , Alberta , Canada
• 12 Department of Community Health Sciences , Cumming School of Medicine, University of Calgary , Calgary , Alberta , Canada
• 13 Department of Paediatrics , Cumming School of Medicine, University of Calgary , Calgary , Alberta , Canada
• Correspondence to Ms Destiny Lutz, Sport Injury Prevention Research Centre, Faculty of Kinesiology, University of Calgary, Calgary, Alberta, Canada; destiny.lutz{at}ucalgary.ca

Objective To evaluate best practices for neuromuscular training (NMT) injury prevention warm-up programme dissemination and implementation (D&I) in youth team sports, including characteristics, contextual predictors and D&I strategy effectiveness.

Design Systematic review.

Data sources Seven databases were searched.

Eligibility The literature search followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Inclusion criteria: participation in a team sport, ≥70% youth participants (<19 years), D&I outcomes with/without NMT-related D&I strategies. The risk of bias was assessed using the Downs & Black checklist.

Results Of 8334 identified papers, 68 were included. Sport participants included boys, girls and coaches. Top sports were soccer, basketball and rugby. Study designs included randomised controlled trials (RCTs) (29.4%), cross-sectional (23.5%) and quasi-experimental studies (13.2%). The median Downs & Black score was 14/33. Injury prevention effectiveness (vs efficacy) was rarely (8.3%) prioritised across the RCTs evaluating NMT programmes. Two RCTs (2.9%) used Type 2/3 hybrid approaches to investigate D&I strategies. 19 studies (31.6%) used D&I frameworks/models. Top barriers were time restrictions, lack of buy-in/support and limited benefit awareness. Top facilitators were comprehensive workshops and resource accessibility. Common D&I strategies included Workshops with supplementary Resources (WR; n=24) and Workshops with Resources plus in-season Personnel support (WRP; n=14). WR (70%) and WRP (64%) were similar in potential D&I effect. WR and WRP had similar injury reduction (36–72%) with higher adherence showing greater effectiveness.

Conclusions Workshops including supplementary resources supported the success of NMT programme implementation, however, few studies examined effectiveness. High-quality D&I studies are needed to optimise the translation of NMT programmes into routine practice in youth sport.

Data availability statement

Data are available in a public, open access repository. Not Applicable.

https://doi.org/10.1136/bjsports-2023-106906

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

WHAT IS ALREADY KNOWN ON THE TOPIC

Neuromuscular training (NMT) injury prevention warm-up programmes are effective at preventing injury rates in youth sports. However, for proper dissemination and implementation (D&I) by multiple stakeholders, barriers such as low adoption, adherence and lack of time must be addressed.

WHAT THIS STUDY ADDS

There are limited high-quality research studies to facilitate the widespread adoption of, and improved adherence to, NMT programmes. Few studies used D&I theories, frameworks or models. Programme flexibility is a common barrier to implementation; adaptation of NMT programmes to fit local contexts is imperative. Comprehensive workshops and supplementary resources currently support the success of NMT programme implementation.

HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY

Promotion of NMT programmes as the standard of practice is essential to increase practical D&I of these programmes, and thus reduce the burden of youth sport injuries. This work provides some directions for stakeholders, including researchers, implementation support practitioners and youth sport policymakers, on current best practices for the delivery of NMT programmes in local youth sport settings. This work also provides the evidence base for more translational research efforts in youth sport injury prevention, a much-needed next step to optimise NMT programmes into youth sport practice.

Introduction

Youth (<19 years) sport participation provides numerous benefits, positively impacting physical and mental health. 1 Youth sport participation rates are high, with up to 90% of youth participating in sport globally. 2–5 However, with increased sport participation comes increased injury risk. One-in-three youth sustain a sport-related injury each year, leading to a significant public health burden with high healthcare costs. 3 6–8 Sport-related injuries may also result in long-term health consequences (eg, poor mental health, reduced physical activity, post-traumatic osteoarthritis). 7–9 Implementing injury prevention strategies is critical to mitigate the injury risk associated with youth sport participation.

Neuromuscular training (NMT) injury prevention warm-up programmes in youth team sport are effective in reducing injury rates by up to 60% and decreasing costs associated with injury based on randomised controlled trials (RCT) and systematic reviews. 10–21 NMT programmes include exercises that can be categorised across aerobic, balance, strength and agility components 22 23 and typically take 10–15 min. 24 25 Originally implemented with the intention of reducing non-contact lower extremity injury risk, 26–28 the effectiveness of NMT programmes has since been evaluated across numerous sports, age groups and levels of play and are associated with lower extremity and overall injury rates compared with standard of practice warm-ups. 12 20 21 25 In youth team sports, a protective effect has been demonstrated in soccer, handball, basketball, netball, rugby and floorball. 11 16 29–31 The International Olympic Committee Consensus Statement on Youth Athletic Development recommends multifaceted NMT warm-up programmes in youth sport. 32

Despite being a primary injury prevention strategy across youth sports, NMT programme adoption remains low. 33–38 For evidence-informed interventions to be successful and have a practical impact, pragmatic approaches derived from dissemination and implementation (D&I) science are necessary across multiple socioecological levels including organisation, coach and player. 36 Dissemination is defined as ‘the active process of spreading evidence-based interventions to a target population through determined channels and using planned strategies’. Implementation is ‘the active process of using strategies across multiple levels of change to translate evidence-based interventions into practice and prompt corresponding behaviour change in a target population’. 36

The aim of this systematic review was to evaluate current best practices for the D&I of NMT programmes in a youth team sport. The specific objectives of this systematic review were to: (1) describe the characteristics of identified D&I-related studies (studies with at least one D&I outcome directly or indirectly assessed as primary, secondary or tertiary outcome); (2) evaluate factors associated with the D&I of NMT warm-up programme across socioecological levels, including barriers and facilitators; (3) examine the effect of D&I strategies in delivering NMT warm-ups across multiple socioecological levels; and (4) examine the influence of D&I strategies on injury rates. Our protocol was registered in PROSPERO (CRD42021271734), and the review is reported in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines ( supplemental appendix S1 ).

Supplemental material

Search strategy and data sources.

A comprehensive search was developed with a librarian (KAH) in MEDLINE, incorporating four main concepts: child/youth, injury prevention, implementation/compliance/adherence and sports. The author team reviewed the final search strategy which was then piloted against the known key studies to ensure that the search was capturing relevant studies. Finally, the MEDLINE search was translated to the other databases. Searches were conducted 25 August 2021 (updated 16–18 August 2022; 5 September 2023). Search strategies are available in Supplemental Appendix S2 . Studies were identified by searching seven databases: MEDLINE(R) and EPUB Ahead of Print, In-Process & Other Non-Indexed Citations and Daily, Embase, Cochrane Central Register of Controlled Trials, Cochrane database of Systematic Reviews (all Ovid); CINAHL Plus with Full Text, SPORTDiscus with Full Text (EBSCO) and ProQuest Dissertations & Thesis Global.

Study selection and eligibility

All database search results were uploaded and duplicates were removed in Covidence (Veritas Health Innovation, Melbourne, Australia). Records were independently reviewed by authors in pairs (DL/IJS, CV/JK, KV/DL), starting with a screening of 50 randomly selected citations to assess inter-rater agreement with a threshold set at 90%. Each pair of reviewers performed title/abstract screening and full-text screening independently, providing reasons for exclusion at full-text stage ( figure 1 ). Any disagreements for exclusion, where a consensus could not be reached within pairs, were resolved by a senior author (OBAO). A secondary evaluation of included manuscripts was performed by senior authors (OBAO and CAE) to ensure appropriate inclusion. Study inclusion criteria were: (1) Participation in a team sport (male and female); (2) a minimum of 70% of participants as a youth (<19 years) or coaches of these youth teams; (3) reported dissemination and/or implementation outcomes (eg, self-efficacy, adherence, intention); (4) reported D&I strategies related to NMT warm-up programmes (ie, NMT delivery strategies, where applicable eg, in RCTs). Exclusion criteria were: (1) Studies evaluating rehabilitation programmes, non-team-based or physical education programmes; (2) non-peer-reviewed; (3) not English. The screening process was reported using the PRISMA flow diagram. 39

• Download figure
• Open in new tab
• Download powerpoint

Study identification Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow diagram.

Risk of bias

To assess the risk of bias, three sets of paired reviewers independently used the Downs & Black (D&B) quality assessment tool. 40 The tool consists of a 27-item checklist (total score/33). A third senior reviewer (OBAO or CAE or AMR) resolved any disagreements. The rating of evidence and strength of recommendations were assessed using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) guidelines. 41–43

Frameworks/models

The proportion of studies that used D&I research theories/frameworks/models, including behaviour change frameworks/models, was examined to identify commonly used frameworks/models.

Efficacy-effectiveness orientation in RCTs

We assessed the components of 12 RCTs using the Rating Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool, as adapted by Maddox et al 44 RITES scores RCTs in systematic reviews based on a continuum of efficacy-effectiveness across four domains: Participant characteristics, trial setting, flexibility of intervention(s) and clinical relevance of experimental and comparison intervention(s) ( online supplemental table S1 ). 45 We modified the Likert grading system to classify studies depending on whether their emphasis was more on efficacy or effectiveness or balanced for both. Given that different aspects of each trial may fall in different places along the efficacy-effectiveness continuum, each RITES domain is scored independently and a composite score is not applicable. To minimise subjectivity, the RITES evaluation for included RCTs was completed by two reviewers (AMR and OBAO). Any disagreements were resolved through discussion to reach a consensus.

Study typologies and assessment of study relevance to D&I

The level of relevance of individual studies (RCTs and quasi-experiments) to D&I was determined based on the implementation-effectiveness hybrid taxonomy: Type 1 (primarily focused on clinical/intervention outcomes), Type 2 (balance focused on both clinical/intervention outcomes and D&I outcomes) and Type 3 (primarily or ‘fully’ (our adaptation) focused on D&I outcomes) studies. 36 46 For ease of interpretation of results, studies were rated considering three broad traditional research design categories (ie, hierarchy of evidence): RCTs, quasi-experimental and observational studies, including cohort, cross-sectional, pre-experimental, qualitative, mixed-methods and ecological studies. Observational studies were categorised as ‘fully focused’ observational-implementation (if only D&I outcomes were evaluated) or ‘partially focused’ observational-implementation (if a combination of clinical and D&I outcomes were evaluated’ D&I studies. 47 RCTs and quasi-experimental studies with Type 2 or Type 3 hybrid approaches were indicated as ‘highly relevant’ towards informing D&I best practices. Furthermore, observational-implementation studies that are fully focused on D&I were also indicated as ‘highly relevant’.

Data extraction

The extracted data included: study design, author, journal, year, population (eg, 13–17 years old female soccer players), participant demographics, D&I intervention strategies (eg, workshops, supplementary resources), D&I framework/model, control group strategies, D&I outcomes (eg, adoption, adherence, intention, fidelity, self-efficacy) and injury outcomes. Study design classification was completed based on data extracted and the process taken by authors, 48 which may have differed from the original classification. Furthermore, prospective, and retrospective cohort studies were consolidated into ‘cohort’ to improve ease of readability. D&I outcomes indicated as compliance were included in the appropriate adherence category as defined in Owoeye et al and described as ‘adherence-related’ outcomes, to maintain unified language across results; the full list is provided in online supplemental table S2 . 36 49–51 Based on the dose-response thresholds reported for NMT programmes within current literature, measures of adherence were used to indicate potential D&I effect ( online supplemental table S5 ). 24 36 52–54 Studies with cumulative utilisation (sessions completed/total possible) of ≥70%, utilisation frequency of ≥1.5 sessions/week or a significant association between D&I exposures and outcomes were defined as moderate-to-highly relevant and identified as having a potential D&I effect (ie, yes). Studies presenting cumulative utilisation <70%, utilisation frequency <1.5 sessions/week or no association between D&I exposures and outcomes were defined as low-to-no relevance (ie, no). Studies reporting both frequency utilisation and cumulative utilisation must both reach the established dose-response thresholds to be considered as having a potential D&I effect. D&I barriers and facilitators, factors influencing injury prevention implementation success and the identification of any frameworks used were also extracted and categorised into themes. Measures of potential effect for these results were summarised using OR, proportions and mean differences in D&I outcomes (eg, adoption, adherence). Injury-specific results were reported as incidence rate ratios, risk ratios, ORs or prevalence. D&I strategies were classified into various categories, including workshops, supplementary resources, personnel support, supervision and combinations of these strategies.

Equity, diversity and inclusion statement

Our author team is comprised of student and senior researchers across various disciplines with representation from low-to-middle-income countries. A variety of demographic, socioeconomic and cultural backgrounds were included in our study populations.

PRISMA flow, characteristics of included studies and risk of bias assessment

A total of 68 relevant studies were included from our initial and updated search yield of 9021 studies ( figure 1 ). Across included studies, 13 included only male youth participants, 13 included only female youth participants, 26 included both and 16 reported coach-focused findings. Sports represented were soccer (n=33), rugby (n=8), basketball (n=7), multisport (n=7), handball (n=5), floorball (n=3), field hockey (n=3), volleyball (n=1) and futsal (n=1).

Details of study characteristics and risk of bias are presented in online supplemental table S3 . D&B scores ranged from 4/33 to 24/33 (median=14/33) from a variety of study designs, including 20 RCTs, 16 cross-sectional, 9 quasi-experimental, 8 cohort, 6 qualitative, 3 ecological, 3 mixed-methods and 3 pre-experimental. The D&B scores for the two top D&I-related relevant studies—an RCT Type 2 study (n=1) was 21/33 and an RCT Type 3 hybrid study (n=1) was 17/33. Using the GRADE guidelines for the process of rating the quality of evidence available and interpreting the quality assessment, the strength of recommendations was ‘low’ given the multiplicity of designs. 42 43

Characteristics of current D&I-related studies

23 studies (33.8%) reported using a D&I /behaviour change framework/model. D&I frameworks included Reach, Effectiveness, Adoption, Implementation and Maintentance (RE-AIM) Framework (n=7), Consolidation Framework for Implementation Research (n=1), Precede-Proceed Model (n=1), Translating Research into Injury Prevention Practice (n=1) and Promoting Action on Research Implementation of Health Sciences (n=1) and the Adherence Optimisation Framework (n=1). Behaviour change models included the Health Action Process Approach (HAPA) (n=8), Theory of Planned Behaviour (n=1) and the Health Belief Model (n=1).

Assessment of study relevance to D&I

Two RCTs of 68 included D&I-related studies (2.9%) were identified as highly relevant to D&I best practices (ie, Type 2 or 3 hybrid approach). 55 56 18 (27.9%) RCTs reported a secondary analysis of D&I strategies 12 16 19 30 53 57–69 ; classified as Type 1 hybrids. Five (8.3%) quasi-experimental studies used Type 2 or Type 3 hybrid approach 22 70–73 ; the remaining studies (n=4; 5%) were classified as quasi-experimental Type 1 hybrids. 74–77 Many observational studies (n=17; 26.7%), 78–94 were highly relevant based on being fully-focused observational-implementation studies; 5 (6.7%) were partially-focused observational-implementation studies. 52 95–98 The remaining observational studies (n=17; 23.3%) were observational-implementation studies, 35 99–114 reporting D&I outcomes from a qualitative lens using interviews and surveys.

The RITES scores for the 14 D&I-related RCTs that examined injuries as primary outcome and D&I outcomes as secondary (Type 1 hybrid approach) are presented in table 1 . Almost all (13 of 14; 92.9%) of the RCTs focused mainly on intervention efficacy (as opposed to effectiveness) regarding the flexibility of NMT warm-up programmes. Cumulatively, effectiveness was rarely (7.1%) prioritised as a primary focus across all the 56 possible ratings of the RITES domains for all 14 studies. 50% of the domain ratings demonstrated efficacy as a priority and 42.9% of the ratings were indicated for a balance between efficacy and effectiveness.

• View inline

RITES domain scores for included trials

Contextual predictors of NMT warm-up programme implementation

50 (73.5% of total) studies identified ≥1 barrier or facilitator within their findings, with 10 (14.7% of total) specifically examining barriers/facilitators as their main objectives. Full list is provided in online supplemental table S4 . The most common barriers identified were time restrictions (n=30), 30 35 59 62 69 70 73 74 78 79 81 82 84 87–91 93 96 98 101 102 105 107 108 112 114 reduced buy-in/support (n=8) 62 75 84 87 105 110–112 and limited awareness of preventative effects of programmes (n=8). 74 84 103 104 107 109 113 Facilitators included comprehensive workshops from trained instructors (n=11), 53 71 78–80 84 90 96 99 100 112 accessibility of supplementary resources (n=10) 82 84 87 89 90 105 114 and uptake/support from multiple stakeholders (n=7). 56 67 84 101 103 105 112 Moreover, suggestions from multiple socioecological levels indicated that increasing programme education and support, increased sport-specific activities and improved awareness of preventive effects, influence NMT implementation success. 36 88 89 115 116 Figure 2 , adapted from Basow et al 117 illustrates the contextual factors reported in the literature. 117 This evidence-informed model shows the important barriers and facilitators that influence the end-user implementation of NMT warm-up programmes across the three key socioecological levels of change.

Contextual predictors of NMT implementation across multiple socioecological levels. (Adapted from Basow et al (2021)).(116). Notes. SE, self-efficacy. NMT, neuromuscular training. Bold represents top barrier(s)/facilitator(s).

51 (75%) studies used implementation strategies for NMT warm-up programmes. The most frequently used strategies were Workshops with supplementary Resources (WR; n=24), followed by Workshops with supplementary Resources, plus in-season Personnel support (WRP; n=14). Three studies employed both WR and WRP strategies. Other methods for implementation included only workshops (n=9), only supplementary resources (n=4), supplementary resources and personnel support (n=2), workshops with personnel support (n=1) and supervision (n=1). Note, some studies are duplicated throughout the table when multiple D&I strategies are compared. 22 53 56 86

The key D&I concepts that were reported within the included studies were adherence or adherence-related (eg, self-efficacy, translation and perception). Specific outcomes within these concepts were further examined from the individual study results. We did not have enough evidence to present a meta-analysis of the effect of D&I strategies on D&I outcomes. Therefore, online supplemental table S5 presents a qualitative summary of the relationships between reported D&I exposure and D&I outcomes. 40 studies reported adherence-related outcomes, of which 32 (80%) were indicated to have potential D&I effect. Studies using WRP (n=14) reported completing between 1.4 and 2.6 sessions/week and cumulative utilisation of 39–85.6%; 9 of these 14 studies have potential D&I effect. Studies using WR (n=24) presented utilisation frequency ranging from 0.8 to 3.2 sessions/week and cumulative utilisation of 55–98% of sessions; 16 of these 24 have potential D&I effect. In studies evaluating workshops only (n=9; 22%), frequency utilisation was reported between 1 and 2 sessions/week across eight of the nine studies and one study had 52% cumulative utilisation; two have potential D&I effect.

Effects of D&I strategies on injury outcomes

Three RCTs specifically examined the effects of the D&I strategies used to deliver NMT programmes on injury outcomes ( table 2 ). Two studies that compared both WR and WRP to supplementary resource only found no significant differences between strategies, 53 56 they reported reduced injury rates in the highest adherence groups by 56% and 72%, respectively. Another study comparing WR and WRP to a standard of practice warm-up found a 36% reduction of ankle and knee injuries when using WR and a 38% reduction in ankle and knee injuries without supervision. 22 There were no significant differences in injury rates between groups.

Injury Outcomes by D&I strategies and adherence

This study evaluated current literature to inform evidence-based best practices for the D&I of NMT programmes in youth team sport. To our knowledge, this is the first systematic review evaluating the D&I of NMT programmes in youth sport. To improve the practical implementation of NMT warm-ups, factors associated with implementation success and current best practices for delivering context-specific NMT programmes are required to be evaluated. 118 In this review, we found few D&I-related studies use D&I or behaviour change frameworks, theories or models to guide their research questions. We discovered the number of RCTs examining the effectiveness of D&I strategies for NMT programme delivery is limited. Common barriers to NMT implementation include programme flexibility and time restrictions; and the use of coach workshops and supplementary resources are currently the primary strategy in NMT programme D&I facilitation.

One-third of the included studies used a D&I framework or behaviour change model in their research work. The HAPA and RE-AIM models were the most frequently used. These models are a conceptual and organised combination of theories required to direct the design, evaluation and translation of evidence-based interventions (NMT programmes) and the context in which they are being implemented. 36 71 119 It is imperative for D&I studies to use these frameworks/models to fully understand specific implementation processes and contexts. Future D&I studies should consider using appropriate frameworks or models, including adaptations and combination of models to guide their specific aims.

Relevance to D&I

Across the relevant literature, a variety of designs and levels of evidence were included.

Of 68 studies, 7 (10.3%) were found to be ‘highly relevant’ toward informing D&I best practice (2 (2.9%) RCTs, 5 (7.4%) quasi-experimental). Other ‘relevant’ studies evaluated implementation as secondary objectives (Type 1 hybrid designs) and/or were of lower level of evidence. 33 observational studies were ‘highly relevant’ to D&I, assessing D&I outcomes and barriers and facilitators from a qualitative lens. While these studies are important for understanding D&I context, more high-quality and highly relevant studies such as RCTs and quasi-experimental designs using the Type 3 hybrid approach, or non-hybrid approach focused on solely evaluating the effectiveness of D&I strategies, are needed to advance the widespread adoption and continued use of NMT programmes in youth team sport.

Effectiveness versus efficacy

Effectiveness is indicative of an evidence-informed intervention’s readiness for practical implementation. 36 Findings from our RITES scores evaluation indicate that the majority of the RCTs had a primary focus on efficacy and not effectiveness. Although many RCT studies had a fair balance between efficacy and effectiveness for participant characteristics, trial settings and clinical relevance domains (≥50% of RCTs), there is a lack of flexibility in the development and evaluation of the evidence supporting current NMT warm-up programmes. These disparities regarding practical implementation have implications for D&I research and practice in this field. Current NMT programmes may need to be modified or adapted to the local context and evaluated further to improve implementation in youth sport settings.

Contextual considerations

In our Adapted Socioecological Model ( figure 2 ), we demonstrate that the utilisation of NMT programmes by individual players within youth team sport can depend on their coach adopting and implementing the warm-up, which may also be dependent on larger organisational systems. Barriers related to end-users’ success in wide-spread adoption and long-term maintenance can be moderated; however, researchers and implementers have to be intentional about tackling these recognised barriers and associated challenges 25 87 104 115 ; integrating the facilitators of successful implementation intending to reduce and address these obstacles is essential. The barriers and facilitators identified in this systematic review provide insight into the combination of D&I strategies that should be formulated and tested by D&I researchers in the sports injury prevention field.

Within the current review, lack of time, whether it be learning, instructing and/or practicing the programme, is a common barrier that plays a significant role in implementation. A recent narrative review focused solely on the barriers and facilitators associated with exercise-based warm-up programmes showed similar conclusions regarding time restrictions. 115 Collective themes within this literature for players, coaches and organisations found that reduced buy-in and support at different levels impacted the adoption of NMT warm-up programmes. The lack of awareness and knowledge of the injury prevention benefits of NMT warm-up programmes also presented major barriers to buy-in, leading to reduced implementation success. Future interventions should ensure that education about evidence-informed injury prevention outcomes associated with programme adherence is integrated within their D&I strategies.

D&I science is a growing field of study. A variety of D&I outcomes were identified such as self-efficacy, intention, reach, outcome expectancy and most commonly, adherence or adherence-related outcomes. These outcomes were evaluated using different D&I strategies for NMT warm-up programmes. The most commonly reported strategies were Workshops with supplementary Resources with/without in-season Personnel support. Evaluation of D&I outcomes showed that adherence or adherence-related outcomes were most frequently reported across studies. Various measures of adherence as defined by Owoeye et al (2020) were identified, including cumulative utilisation, utilisation frequency, utilisation fidelity, duration fidelity and exercise fidelity. 36

Adherence remains the most common D&I outcome in the sport injury prevention literature. 36 120 In this review, we defined adherence and adherence-related thresholds for a moderate-to-high dose-response to be ≥70% cumulative utilisation and/or ≥1.5 session/week to achieve the desired protective effects. This was done with consideration of pragmatism and a practical balance between programme efficacy and effectiveness given the existing literature. 24 91 32 of 40 studies (80%) from those with adherence or adherence-related outcomes had a potential D&I effect based on a moderate-to-high adherence or adherence-related outcome level. The use of WR and WRP was the most common D&I strategies for delivering NMT warm-up programmes. While there are several areas for improvement for the practical D&I of NMT warm-up programmes in youth sport settings, the use of comprehensive workshops and supplementary resources at various levels, particularly with coaches, appears to be the optimal best practice. However, only two ‘highly relevant’ D&I studies (RCTs) from the current systematic review presented conclusions based on the effectiveness of D&I strategies and outcomes specifically.

Many studies (n=26/68; 38.2%) included both male and female participants; however, no sex-differences were described. When examining D&I outcomes, only 7/26 (26.9%) had moderate-to-high adherence when looking at both male and female youth players. In total, 84.6% of the female-only (11/13) and 72.7% of the male-only studies (8/11) reported moderate-to-high adherence levels. These findings suggest greater attention and efforts for adherence and implementation of NMT programmes in the male youth team sport setting may need further consideration compared with the female youth sport context.

Of the preliminary evidence for Type 2 and 3 hybrid designs, the literature highlighted in the synthesis of this data that WR are effective strategies in injury prevention and showed more moderate-to-high adherence levels. Given that most studies are doing some form of WR, adding in-season personnel support does not increase the protective effect and may be less sustainable given that resources, time and support are significant barriers to the D&I of these programmes.

Additionally, greater implementation and programme buy-in were found in studies where uptake of these NMT programmes was supported across multiple stakeholders, particularly at the organisation level. 19 67 90 103 112 Catering to programme deliverers (coaches, organisations, parents) and evaluating their awareness, perception and self-efficacy may help further inform our understanding of D&I and how we can best work to promote programme uptake further.

D&I strategies and injury outcomes

The findings from this systematic review suggest that while various D&I intervention strategies are effective at reducing injuries in youth team sports, the ranges of injury rate ratios are similar across studies employing different strategies (32–88% lower injury rates across WR strategy studies and 41–77% lower injury rates across WRP strategy studies). 22 53 56 Although this was not the proposed evaluation of these studies, our findings demonstrate that the use of workshops may influence D&I success and the availability of supplementary resources alone may not be efficacious. Future evaluation of the influence of delivery strategies should be considered.

Future directions

Using facilitators to reduce barrier burden.

Regarding NMT strategy evaluation, our findings show that most of the current programmes focus on efficacy over effectiveness, particularly in the aspect of intervention flexibility; this suggests a need for the adaptation of NMT programmes to fit local contexts. NMT programme developers should consider more enjoyable and user-friendly exercises that include sport-specific activities (eg, ball work, partner drills, tags). Increasing variations also improves player buy-in and increases intrinsic motivation. At a coaching level, workshops on NMT programmes should include evidence-informed education on the injury prevention benefits and should incorporate content addressing coach self-efficacy to enhance implementation quality. 16 100 121 An ongoing pragmatic evaluation of NMT programme effectiveness is warranted as they undergo adaptation to local contexts.

Organisations have expressed limited knowledge and education for implementation as a significant barrier to successful NMT programme use. 90 99 101 105 112 115 122 Implementers should look to provide accessible resources and encourage further support from multiple stakeholders, including the governing bodies. This could lead to policy changes within the club and result in greater uptake of these programmes long-term. Collaborations among stakeholders (researchers, youth sport administrators, coaches and players) in programme development, evaluation, D&I are necessary to improve efforts for impactful practical translation of programmes.

Research recommendations

The support for NMT programmes within youth sport is extensive. 28 Although these programmes have been shown to be effective for injury prevention in many sports, 10 11 sport representation across D&I studies in our review was limited. Scaling up of NMT programmes and supporting continued research into other sports is vital for increased context-specific D&I of these programmes to reduce the overall burden of youth sport injuries.

Compliance and adherence were often used interchangeably, despite having distinct definitions. Although their mathematical calculations are similar, these two constructs are contextually different. Compliance refers to individuals conforming to prescribed recommendations in controlled intervention settings, 123 while adherence refers to the agreement of an individual’s behaviour to recommended evidence-based interventions in uncontrolled settings. 36 Standardised definitions should be considered more frequently by researchers to build on current knowledge and inform future D&I research.

Using D&I frameworks/models can improve NMT programme implementation success in a practical setting. 71 124 Application of D&I frameworks/models, including behaviour change models, 124 is limited in injury prevention and this is reflected in the current systematic review. Future studies should use D&I frameworks/models to help guide the implementation of these NMT programmes. In doing so, researchers can gain a better understanding of the contextual and behaviour change aspects related to youth sport injury prevention. 115

Limitations

Given the broad nature of our research question, specific results were required for inclusion. Despite being specific to our objectives, our limitations set for participant age range, team sport settings and English language studies only, may have resulted in missing other studies that evaluated D&I interventions and outcomes related to NMT programmes.

Due to the heterogeneous nature of studies, meta-analysis was not possible for any of our objectives. Inclusion of various study designs, although comprehensive, impeded this process and resulted in inconsistent injury and adherence definitions across our population of interest. Furthermore, the subjective nature of many qualitative studies included may have resulted in variability within the data extracted. With the varied definitions used for each specific outcome and design, we looked to consolidate the terminology used into more succinct and unified language and we encourage this to be employed by researchers.

Methodological flaws existed in the included studies that may warrant caution about the interpretation of our conclusions. For example, many of the included studies did not include power calculations or reported low power, increasing the chance of Type 2 error. Further, many studies did not consider confounding or effect modification in their analyses or failed to report the validity of measurement tools used for injury data collection. We also acknowledge that publication bias may have favoured the inclusion of studies demonstrating significant findings (eg, effectiveness, efficacy). By considering quality assessment as an objective, we aimed to account for these limitations.

There was limited evidence supporting the effect of D&I strategies on D&I-specific outcomes. There were only two high-level evidence (RCTs) studies in this review that directly discussed the matter of D&I strategies on D&I outcomes. 55 56 D&I-related outcomes were evaluated as secondary objectives in other high-level evidence studies, therefore, we could only examine the relationship between D&I strategy and outcome to assess if the strategy used resulted in moderate-to-high adherence levels, given our pre-established thresholds.

Conclusions

This systematic review demonstrates that: (1) Few D&I-related studies are based on D&I or behaviour change theories, frameworks or models; (2) few RCTs have examined the effectiveness of D&I strategies for delivering NMT programmes; (3) programme flexibility and time restrictions are the most common barriers to implementation and; (4) a combination of coach workshops and supplementary resources are currently the primary strategy facilitating NMT programme D&I; however their effectiveness is only evaluated in a few studies. This systematic review provides foundational evidence to facilitate evidence-informed knowledge translation practices in youth sport injury prevention. Transitioning to more high-quality D&I research RCTs and quasi-experimental designs that leverage current knowledge of barriers and facilitators, incorporates Type 2 or Type 3 hybrid approaches and uses behaviour change frameworks are important next steps to optimise the translation of NMT programmes into routine practice in youth team sport settings.

Ethics statements

Patient consent for publication.

Not applicable.

Ethics approval

• Cumming SP ,
• Australian Sports Commission
• Meeuwisse DW ,
• Eliason PH , et al
• Child and Adolescent Health Measurement Initiative
• Emmonds S ,
• Weaving D ,
• Lara-Bercial S , et al
• Merkel DL ,
• Raysmith BP ,
• Charlton PC
• Brunner R ,
• Friesenbichler B ,
• Casartelli NC , et al
• Whittaker JL , et al
• Meeuwisse WH
• Gianotti S ,
• Jacquet C ,
• Verhagen E , et al
• Marshall DA ,
• Lopatina E ,
• Lacny S , et al
• Owoeye OBA ,
• Akinbo SRA ,
• Tella BA , et al
• Palacios-Derflingher LM ,
• Pasanen K ,
• Parkkari J ,
• Pasanen M , et al
• Soligard T ,
• Myklebust G ,
• Steffen K , et al
• Thorborg K ,
• Krommes KK ,
• Esteve E , et al
• Webster KE ,
• Räisänen AM , et al
• Räisänen AM ,
• van den Berg C ,
• Owoeye OBA , et al
• Pfeifer K , et al
• Richmond SA ,
• Doyle-Baker PK , et al
• Barengo N ,
• Meneses-Echávez J ,
• Ramírez-Vélez R , et al
• Bizzini M ,
• Sadigursky D ,
• De Lira DNL , et al
• Åkerlund I ,
• Sonesson S , et al
• Hislop MD ,
• Stokes KA ,
• Williams S , et al
• Hopper AJ ,
• Joyce C , et al
• Bergeron MF ,
• Mountjoy M ,
• Armstrong N , et al
• De Ste Croix M ,
• Sanchez SH , et al
• Creech MJ ,
• Peterson DC , et al
• Rauvola RS ,
• Brownson RC
• Owoeye OB ,
• Olawale OA , et al
• Niederer D ,
• Vogt L , et al
• Ardern CL ,
• Büttner F ,
• Andrade R , et al
• Guyatt GH ,
• Vist GE , et al
• Kunz R , et al
• Siemieniuk R ,
• Maddox CD ,
• Subialka JA ,
• Young JL , et al
• Wieland LS ,
• Berman BM ,
• Altman DG , et al
• Landes SJ ,
• McBain SA ,
• Schwartz S ,
• Duncan DT , et al
• Ranganathan P
• Proctor E ,
• Silmere H ,
• Raghavan R , et al
• Schwarzer R
• Hägglund M ,
• Atroshi I ,
• Wagner P , et al
• Steffen K ,
• Romiti M , et al
• Sugimoto D ,
• Barber Foss KD , et al
• Schneider B , et al
• Meeuwisse WH ,
• Al Attar WSA ,
• Alzahrani H , et al
• Alizadeh MH ,
• Shahrbanian S , et al
• Waldén M , et al
• Akasaka K ,
• Otsudo T , et al
• Leppänen M ,
• Vasankari T , et al
• LaBella CR ,
• Huxford MR ,
• Grissom J , et al
• Loppini M ,
• Berton A , et al
• Parsons JL ,
• Carswell J ,
• Nwoba IM , et al
• Rössler R ,
• Bizzini M , et al
• Slauterbeck JR ,
• Choquette R ,
• Tourville TW , et al
• Olsen OE , et al
• Mattacola CG ,
• Bush HM , et al
• Hancock MV ,
• Stokes KA , et al
• RÄisÄnen AM , et al
• Barboza SD ,
• Emery C , et al
• Hellquist E ,
• Ahlqvist K , et al
• Tomsovsky L ,
• Whatman C , et al
• Verhagen E ,
• Gouttebarge V , et al
• Celebrini RG ,
• Miller WC , et al
• Franchina M ,
• Tercier S , et al
• Register-Mihalik J ,
• Choquette R , et al
• Lindblom H ,
• Carlfjord S ,
• Ljunggren G ,
• Perera NKP ,
• Merrett CK ,
• Linnéll J , et al
• Myklebust G , et al
• O’Brien J ,
• Santner E ,
• Befus K , et al
• Räisänen A ,
• Black AM , et al
• Sonesson S ,
• Lindblom H , et al
• Nilstad A ,
• Thein-Nissenbaum J ,
• Ageberg E ,
• Lucander K , et al
• Brodin EM ,
• Watkins R ,
• Cornelissen MH ,
• Baan A , et al
• Donaldson A ,
• Callaghan A ,
• Taylor JR ,
• Novak MA , et al
• MacFarlane AJ ,
• Weiss-Laxer NS , et al
• McGuine TA ,
• Pennuto A , et al
• Munoz-Plaza C ,
• Davis A , et al
• Norcross MF ,
• Johnson ST ,
• Bovbjerg VE , et al
• Jimenez P ,
• Oliver JL , et al
• Denegar CR , et al
• Lambert M , et al
• Withall AL , et al
• Minnig MC ,
• Hawkinson L ,
• Root HJ , et al
• Macpherson A , et al
• Fenelon D , et al
• Bruinsma A , et al
• Verhagen EALM , et al
• Brown JC , et al
• Bogardus RL ,
• Martin RJ ,
• Richman AR , et al

Supplementary materials

Supplementary data.

This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

• Data supplement 1
• Data supplement 2

X @carlavdb_, @amraisanen, @KatiPasanen, @Kat_Schneider7, @CarolynAEmery, @owoeye_oba

Contributors DL, CE and OBAO contributed to development of study proposal and design. DL, CvdB, AMR, IJS, KV, JK, AH, CE and OBAO conducted search, study selection and screening, data extraction and synthesis and quality assessment. DL led the writing of the manuscript and was the guarantor for the project. All authors contributed to drafting and revising the final manuscript. All authors approved the submitted version of the manuscript.

Funding This study was funded by Canadian Institutes for Health Research Foundation Grant Program (PI CAE).

Competing interests OBAO is a Deputy Editor for the British Journal of Sports Medicine. CE, KJS and KP are Associate Editors for the British Journal of Sports Medicine.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: