case study results synthesis

A Guide to Evidence Synthesis: 11. Synthesize, Map, or Describe the Results

• Meet Our Team
• Our Published Reviews and Protocols
• What is Evidence Synthesis?
• Types of Evidence Synthesis
• Evidence Synthesis Across Disciplines
• Finding and Appraising Existing Systematic Reviews
• 0. Develop a Protocol
• 1. Draft your Research Question
• 2. Select Databases
• 3. Select Grey Literature Sources
• 4. Write a Search Strategy
• 5. Register a Protocol
• 6. Translate Search Strategies
• 7. Citation Management
• 8. Article Screening
• 9. Risk of Bias Assessment
• 10. Data Extraction
• 11. Synthesize, Map, or Describe the Results
• Evidence Synthesis Institute for Librarians
• Open Access Evidence Synthesis Resources

Synthesizing, Mapping, or Describing the Results

Synthesize, map, or describe the results.

In the data synthesis section, you need to present the main findings of your evidence synthesis . As an evidence synthesis  summarizes existing research, there are a number of ways in which you can synthesize results from your included studies.

If the studies you have included in your evidence synthesis are sufficiently  similar , or in other words homogenous, you can synthesize the data from these studies using a process called “meta-analysis”. As the name suggests, a meta-analysis uses a statistical approach to bring together results from multiple studies. There are many advantages to undertaking a meta-analysis. Cornell researchers can consult with the Cornell University Statistical Consulting Unit or attend a meta-analysis workshop for more information on this quantitative analysis. 

If the studies you have included in your evidence synthesis are not similar (e.g. you have included different research designs due to diversity in the evidence base), then a meta-analysis is not possible. In this instance, you can synthesize the data from these studies using a process called “narrative or descriptive synthesis”.

A word of caution here   – while the process underpinning meta-analysis is well established and standardized, the process underpinning narrative or descriptive synthesis is subjective and there is no one standard process for undertaking this.

In recent times, evidence syntheses of qualitative research is gaining popularity. Data synthesis in these studies may be termed as “meta-synthesis”. As with narrative or descriptive synthesis, there are a myriad of approaches to meta-synthesis.

(The above content courtesy of  University of South Australia Library .)

Regardless of whether an evidence synthesis presents qualitative or quantitative information, reporting out using the PRISMA flow diagram is recommended. The PRISMA website and its many adaptations can be very helpful in understanding components of systematic reviews, meta-analyses and related evidence synthesis methods 

Librarians can help write the methods section  of your review for publication, to ensure clarity and transparency of the search process. However, we encourage evidence synthesis teams to engage statisticians to carry out their data syntheses.  

• << Previous: 10. Data Extraction
• Next: Evidence Synthesis Institute for Librarians >>
• Last Updated: May 28, 2024 3:12 PM
• URL: https://guides.library.cornell.edu/evidence-synthesis

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Anderson R, Booth A, Eastwood A, et al. Synthesis for health services and policy: case studies in the scoping of reviews. Southampton (UK): NIHR Journals Library; 2021 Aug. (Health Services and Delivery Research, No. 9.15.)

Synthesis for health services and policy: case studies in the scoping of reviews.

Chapter 3 results.

The eight case studies feature the scoping stages of evidence syntheses conducted by the HSDR Evidence Synthesis Centres between 2017 and 2020: three cases from the University of Exeter, three from the University of Sheffield and two from the University of York. Table 2 shows the considerable diversity of topics and types of synthesis method that these case studies covered. Please note that for ‘synthesis type’ we have used the terminology used in the source project and report, and that there is some variation and overlap in use of terms. In particular, evidence maps and mapping reviews are essentially the same in purpose and final product, and scoping reviews (as a defined product) should be distinguished from the scoping stage and processes that are the focus of all of the case studies. In addition, most of these named subtypes of evidence synthesis are also systematic reviews (in the sense that they had clearly defined questions and explicit methods for identifying, assessing and summarising included evidence sources). We have decided to retain the terms as used in each case study review, rather than to retrospectively impose a standardised typology. Further details of the specific synthesis methods used in each case study review, the rationale for their choice and any patient and public involvement (PPI) are in Appendix 1 , Table 13 .

Main PICOSS [population (or patient type), intervention, comparator, outcome, study types, synthesis method] characteristics of the case study evidence syntheses presented in this report

The case studies that started with a clearly known policy customer/decision-maker and with a clearly stated review question or evidence gap/need are presented first, through to those that had neither a clearly stated policy customer nor a clear initial question. In this way, the order of presentation of the case studies should, in principle, move from those with clearer predefined initial scopes to those with more open-ended and uncertain scopes. The classification of case studies according to these two criteria is presented in Table 3 .

Grouping of case studies according to initial clarity of question and whether or not main policy customer was known

Case study 1: rapid evidence synthesis of ‘Digital-First Primary Care’

This topic was given to the HSDR Evidence Synthesis Centre at York, having been identified as an urgent topic from an NHS England primary care workshop that focused on digital aspects of care. The York team had not been involved in the workshop and so sought further information from NHS England about evidence requirements and scope. Following initial clarification and discussion with representatives from NHS England in June 2018, the team undertook a rapid, responsive evidence synthesis between July and December 2018. Throughout the project, the York team maintained e-mail and telephone contact with a senior policy lead in the new business models team of NHS England’s Strategy and Innovation Directorate. This person co-ordinated the involvement of representatives from other teams (e.g. primary care).

The protocol was posted on the team’s webpage, 17 as it was not eligible for registration in PROSPERO. The final report was published in full 18 and a brief evidence summary produced. 19

The original questions articulated at the workshop were as follows:

• What are the most effective automated systems management approaches that result in high levels of general practitioner (GP) engagement?
• How do you present data to ensure change in practice?
• What are the barriers to and motivators of using digital technology that drive cultural and behavioural change within primary care practitioners?

Following initial discussions with NHS England, which identified a broad and far-reaching list of themes and questions, an iterative production process was agreed to undertake the work in stages. We presented the findings after each stage to discuss progression onto the next stage. After the discussions that followed the second stage, the work was concluded and the report completed.

The rapid responsive evidence synthesis was undertaken to inform NHS England policy in ‘Digital-First Primary Care’. The principles and some aspects of systematic review methodology were applied to ensure transparency and reproducibility. The two stages of the synthesis consisted of (1) scoping the published review evidence and (2) addressing a refined set of questions produced by NHS England from the evidence retrieved during the scoping stage. Given that ‘a full systematic review was not possible, given the time and resources available’, the team ‘conducted a rapid synthesis of the most relevant evidence identified during the scoping exercise (stage 1) to establish if and to what extent these questions can be answered by the identified research’. 18 Patient and public representatives were not directly involved in the development of the synthesis aims, methods or interpretation.

The following questions were addressed in the second stage:

• As GP workload and workforce is the main threat to primary care, how do we use these innovations to alleviate this, rather than only increasing patient convenience and improving their experience?
• Which patients can benefit from digital (online) modes and models of engagement between patients and primary care?
• What channels work best for different patient needs and conditions?
• Are there differences in synchronous and asynchronous models?
• How do you integrate ‘digital first’ models of accessing primary care within wider existing face-to-face models?
• How do you contract such models and how do you deliver them (what geography size, population size)?

The final inclusion criteria were as follows:

• population – any primary care medical staff and patients (or their caregivers) of any age and/or other medical professionals
• interventions – as the known literature rarely conceptualised interventions as ‘digital primary care’, any form of non-face-to-face interaction, including e-mail, online/video, messaging and artificial intelligence-led systems or triage (or any of these alongside telephone consultation)
• outcomes – impact on care in terms of effectiveness and safety, patient access/convenience (including which patients are able to use digital consultations and what conditions are appropriate for non-face-to-face engagement), system-level efficiencies and related issues, such as workforce retention, training and satisfaction
• study design – systematic reviews, meta-analyses and other forms of evidence syntheses (any related primary studies encountered were included where relevant, although primary research evidence was not systematically searched).

Summary of key challenges/choices and scoping decisions

Challenge 1: clarifying the questions of interest and deciding the most appropriate method of synthesis.

The brief topic outline we were given at the start of the project was broadened and expanded during the initial communications with NHS England. They were interested in patient-focused digital innovation in primary care and identified four broad themes and nearly 20 separate questions for which they wanted answers. The questions were wide-ranging, covering issues of contracting and implementation, as well as effectiveness, cost-effectiveness, benefits and risks. They reflected the interests and priorities of the various sections of NHS England involved in implementing ‘Digital-First Primary Care’.

It became clear in these discussions that it was not possible, at that stage, to identify a clear and focused research question (or questions) of high priority to NHS England that was amenable to evidence synthesis. Instead, we identified the need for a responsive and iterative approach to support the needs of the policy-makers.

We adopted a multistage approach to the work. We began by searching the research literature to scope available evidence syntheses. We extracted the key characteristics of all included documents and produced an interactive database of published and ongoing evidence that could be ordered or filtered according to these characteristics, incorporating links to the full-text versions where available. We produced an interim report that summarised the key evidence identified in this scoping exercise, along with an annotated bibliography and the interactive spreadsheet.

We presented the interim report to NHS England and asked them to decide whether it provided the information they needed or if a gap remained to be addressed by further rapid evidence synthesis. NHS England responded with seven questions that reflected specific ‘live’ policy areas in which they were most interested. Therefore, we undertook a second stage and conducted a rapid synthesis of the relevant evidence identified from the stage 1 scoping exercise with the seven research questions identified by NHS England, forming the basis of a thematic framework. Critical appraisal of included evidence was facilitated by relevant assessment tools and reporting standards used to inform judgements about the internal and external validity of included research results presented in the thematic synthesis.

We produced a report that combined both the initial scoping exercise and the rapid synthesis undertaken at stage 2. Again, we presented the findings through a summary report and teleconference to representatives from various NHS England teams (e.g. new business models, primary care, digital and workforce). Following this presentation, and the subsequent discussions, the topic was concluded.

Summary of response to challenge

We needed to adopt an iterative, responsive approach to this topic and revise our methodology accordingly. The initial scoping exercise was undertaken to provide a high-level overview of the available evidence, followed by a rapid evidence synthesis. A full systematic review was not possible, but aspects of systematic review research methodology (such as a priori inclusion criteria, critical appraisal of included evidence, and process measures to avoid bias and errors) were applied to introduce a level of transparency and reproducibility.

Challenge 2: identifying and incorporating recent and ongoing research

As a result of the initial scoping exercise and discussions with other researchers working in the field, we became aware of recent and ongoing projects, as well as two open NIHR calls for a proposal relevant to the topic. Although we searched for evidence syntheses, we incidentally identified some recent or ongoing primary research studies.

We were keen to ensure that we did not duplicate effort and also to alert NHS England to academic groups actively researching the topic. Therefore, we made contact with the researchers who were eager to engage, providing early sight of their draft reports in confidence to be included in our work. They were also keen to engage further with NHS England to inform ongoing policy work.

To avoid duplication of effort and to ensure that policy was informed by the most recent research, we highlighted the ongoing work alongside review evidence and facilitated contact between NHS England and the research authors.

Reflections and lessons

Multiple stakeholders from different areas within the same organisation identified a broad and far-reaching list of themes and questions that reflected the differing remits and priorities of the stakeholders. This required an iterative production process, undertaking the work in stages and presenting the findings at each stage to support their needs. We identified a tension between ongoing engagement with busy and changing stakeholders and fast-moving policy in an area of rapid and ongoing innovation. This required pragmatic adaptation of methods to meet the needs of stakeholders to balance methodological rigour with the usefulness of outputs, while maintaining transparency. In addition, in an area of rapid and ongoing innovation where peer-reviewed evidence may not be currently available, it was helpful to highlight ongoing work and academic colleagues working in the area for further direct dialogue with the stakeholder.

Disseminating the outputs from this multistage project was problematic. To respond to the stakeholder needs, we produced an interactive database of published and ongoing evidence (i.e. enabling sorting and filtering of the evidence base on key characteristics, accessing direct links to full publications and/or contact details of researchers). Reducing this to a final textual HSDR report stripped away this functionality and obscured much of the underlying work.

Case study 2: review of experiences of the ‘nearest relative’ provisions of the Mental Health Act 1983

Origin and context of the review topic.

This evidence review topic was initially notified by the NIHR HSDR programme in direct response to an urgent request for help in gathering evidence for the independent review of the Mental Health Act 1983 (MHA), 16 which was being conducted during 2018. In the UK, the MHA 16 is the central piece of legislation that determines the circumstances and processes for when and how people experiencing mental distress can be compulsorily detained for assessment or treatment.

For the Exeter centre, it was the most rapid of the rapid systematic reviews that we had conducted during the contract period, as well as our first review project that was specifically about a piece of legislation (rather than a health-care intervention or model of health service delivery). We got our first indication of the review topic on 3 January 2018 and submitted the final report by the end of March 2018. The scoping process was, therefore, compressed and pragmatic, for example relying closely on the knowledge and stated needs of the main policy contact within the team conducting the independent review. Although we could identify a time point when the review questions and inclusion criteria were agreed, in other respects the scoping stage was less distinguishable from the main searches and review. In effect, it was a ‘live’ review protocol and the review was shaped and delivered through regular ongoing contact with the policy customer.

The scoping and systematic review was conducted by the HSDR evidence synthesis team at Exeter, and the full review protocol was published on the PROSPERO database (CRD42018088237). 20

The review topic was initially stated as one of several very brief issues or aspects of the MHA 16 sent to the team by NIHR. The subtopics that the policy leads initially prioritised as ‘the most pressing’ were:

• the rights of relatives
• consent and capacity
• the criminal justice system (tribunals and restricted patients)
• legal clarifications.

Our work was steered towards the first topic and, specifically, the ‘nearest relative’ provisions of the MHA. 16 These are the legal requirements and associated practices that govern the involvement of the spouse or close biological relatives (i.e. the ‘nearest relative’) of a person who is being detained compulsorily for mental health reasons under the MHA 16 (sometimes referred to as ‘being sectioned’). These persons are involved primarily as an advocate and support for the person in mental distress, especially in relation to decisions about care.

Although we had no specific review question, the remit of the independent review of the MHA 16 included specific questions that it wanted to answer ( Box 3 ) and these usefully shaped the possible directions of our rapid evidence review.

Questions to be addressed by the independent review of the MHA

Our contact person in the independent review team expressed the purpose of the needed review as the following main review question:

• Are the powers under the MHA 16 relating to the appointment and involvement of the nearest relative appropriate (i.e. working)?

By the time we completed scoping the topic and finalised our review protocol, the planned systematic review aimed to address the following question:

• What are the experiences of services users, family members, carers and professionals of the use of the ‘nearest relative’ provisions in the compulsory detention and ongoing care of people under the MHA?

However, the remit and context of the independent review, together with guidance from the policy customer about the main perceived problems with the current legislation, revealed specific stages or aspects of these provisions that were of particular interest. These included the processes of identification of the nearest relative, displacement (i.e. replacement) of the assigned nearest relative, decisions about care, service users having access to support from those carers and loved ones who they wanted support from, and issues of patient confidentiality and information sharing. These stages/aspects were expressed in the PROSPERO protocol under the ‘outcomes’ of interest (see Table 8 ). 20

Summary of final inclusion criteria: interventions to reduce the length of stay of older people with planned hospital admissions

We aimed to answer this question by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria. Table 4 summarises how the scoping process refined review questions and defined the inclusion criteria, compared with the initial, briefly stated review topic. The ultimate choice of a rapid form of qualitative evidence synthesis (i.e. framework synthesis) was mainly dictated by the review question’s focus on people’s experiences, combined with the very short time frame. Although there was some PPI in interpreting and writing up this rapid evidence synthesis, people or carers who had been directly affected by the nearest relative provisions were not involved in scoping the review or its questions.

Summary of final inclusion criteria: nearest relative provisions of the MHA

Key challenge 1: uncertainty about the type of evidence that would best answer the questions

The first question was whether or not any prior research had focused on patient and carer experiences of the nearest relative provisions within UK legislation. The questions of interest seemed so specific that we doubted if research would address them. Our review group’s contract and work remit extended to the potential for primary research or surveys, where such methods best answer policy-maker or NIHR HSDR questions. However, two early discoveries enabled us to focus on the evidence review component, rather than primary research. First, we identified a 2017 survey of people’s experiences of the MHA by the Mental Health Alliance and Rethink Mental Illness (London, UK). 21 Second, we learned that the independent review team were, themselves, conducting a survey of carer experiences.

It became important to know early on whether the policy customer expected the review team to summarise research evidence or, given the distinct possibility of finding no or very little research, to extend coverage to other relevant non-research forms of evidence in the public domain (e.g. blogs, online discussion forums). Our first teleconference with the policy customer revealed a clear preference for research-based evidence, if available. This was to ‘reduce possible bias’ and achieve a breadth of evidence in the review process, given the other information sources and consultation on which the independent review processes would be drawing.

Therefore, the policy customer gave us a clear and early steer to search for and synthesise research evidence only. Nevertheless, at that time, we were not sure if there would be sufficient, or even any, good-quality UK-based research evidence on patient and carer experiences of the nearest relative provisions. Therefore, we also noted that, later, we might need to extend the scope to grey literature and less research-based sources of evidence, or include evidence from beyond the UK. This highlights pervasive uncertainty about the available evidence for answering alternative potential questions, and the motivation to make best use of a review team’s core skills and resources in responsive, commissioned policy-informing reviews.

Decision 1: to limit to research evidence only

The response of the team was to commit to reviewing evidence from research only. However, we also made some aspects of the review scope and protocol conditional (i.e. on the amount and quality of research evidence found). For such reviews, the scope and protocol are live and adjustable plans that are expected to change in response to the evidence that is found. They often lay out sequences of possible scopes, iteratively refined, realigned and renegotiated. Had the team found only one or two includable research studies, this decision may have been revisited. This kept options open within a very rapid review timescale.

Key challenge 2: deliverability within 8 weeks

This was the overarching challenge and a non-negotiable constraint, against which the other challenges and decisions were all weighed. The deliverability of any review would critically depend on the number, richness/quality and, therefore, amenability to formal synthesis of the studies found, and this would not be known until 4–6 weeks before the report submission deadline. Concerns about this were compounded by our awareness that synthesising qualitative research, usually being an interpretative process, takes time and requires a team approach. We had team discussions about the value of also searching for and including survey research (i.e. which may not be qualitative). However, having found enough qualitative research, we did not ultimately search for survey studies. (In any case, such a review would be unlikely to provide more recent and relevant evidence than a reanalysis of the 2017 Mental Health Alliance survey responses.)

Decision 2: manage expectations of the policy customer through regular contact and review of progress

We had teleconferences with the policy customer every 1 or 2 weeks, at a minimum, during the first half of the review. This was not so much a decision, but a direct response to our uncertainty (i.e. to keep open communications with the policy customer about whether or not we were finding evidence and what evidence we were finding). Had there been too much evidence to synthesise, there would be discussions during these teleconferences as to what evidence we should focus on. The short time frame of the review, undoubtedly, also played a part in us choosing not to look for evidence from beyond the UK or prior to 1998 (see Key challenge 3: how old and from which other jurisdictions would evidence be relevant to experiences of the UK Mental Health Act? and Decision 3: to include only UK evidence and evidence after 1998 ).

Key challenge 3: how old and from which other jurisdictions would evidence be relevant to experiences of the UK Mental Health Act?

As mentioned, this was our first review of an aspect of legislation. Although people in other countries might have experience of equivalent legislation and practices for including family members of close friends in decision-making for people experiencing severe mental distress, only people from the UK (i.e. the jurisdiction of application of the MHA 16 ) would have directly relevant experience of the nearest relative provisions.

Decision 3: to include only UK evidence and evidence from after 1998

This was an early and relatively easy decision (i.e. to not include evidence from beyond the UK). The legislation and the nearest relative provisions within it were so specific to the jurisdiction of the UK that evidence from elsewhere would have very little applicability to the UK legislative and mental health-care context. It was also because the broad context of this review – and the scope of the interim report of the independent review of the MHA 16 that it was to inform – was the creation of a fuller understanding of the current and past problems with legislation and its use. Had the review context required a stronger understanding of potential alternatives and legislative solutions, then perhaps an international and comparative review of equivalent provisions in mental health legislation in other countries may have been useful. Another factor that may have informed this decision was that we knew that some research evidence would originate from Scotland, where an alternative to the nearest relative provisions was already current practice. It was, therefore, likely that this evidence, and its comparison with studies in England and Wales, would be more likely to inform the independent review than studies from mainland Europe or North America.

The cut-off date of 1998 was partly to limit the size of the screening task (i.e. to address deliverability in 8 weeks) and partly because our stakeholders confirmed that the experience of the nearest relative provisions would have differed before and after the adoption of the Human Rights Act 1998 22 in the UK. Therefore, only evidence about experience of the mental health legislation after 1998 was judged as relevant to future possible legislation.

Key challenge 4: conducting rapid synthesis of qualitative research evidence

Given the extremely tight timelines, and the researcher-intensive nature of qualitative evidence synthesis, we were very nervous about the possibility of finding too many studies that met our inclusion criteria. This would have risked making the review undeliverable, rushed and of reduced quality, or having to retrospectively exclude studies.

Decision 4: using ‘framework synthesis’ and prioritising conceptually richer studies within the synthesis

Although a full thematic synthesis would have been the preferred approach, this would not have been possible within the limited time frame of this review. However, committing to conducting a rapid synthesis of qualitative research was less risky and more feasible because a streamlined, pragmatic approach to qualitative evidence synthesis was available, which could be applied to a modest number of studies. Fortunately, we found 35 papers from 20 studies; however, 22 papers provided only half a page of qualitative evidence relevant to the five study objectives.

This meant that we could, with some adaptations and innovations to the method, conduct a pragmatic, rapid, best-fit framework synthesis of the qualitative studies within the 6-week time frame. Within the method, a three-stage approach was used. First, relevant data were extracted according to the research objectives of our review from the 22 papers with half a page of relevant qualitative evidence. This process identified the six richest studies that contained the most data relevant to our research questions. Themes were selected from these studies to further refine the framework. In the third and final stage of the synthesis, thematic synthesis of the data enabled the corroboration and extension of the framework. 23

Reflection and lessons

In this review, where review scoping and the conduct of the review overlapped almost completely, regular contact with the policy customer was an important way of staying responsive to the state of the evidence as it emerged. Fine-tuning of the review in response to the evidence needed to be in keeping with the overall protocol and fit with the available time and resources. These were decisions that could largely be manage from our side. However, the review also needed to be relevant to the policy customer’s needs, which demanded a close working relationship in such a rapidly evolving review.

The development of search strategies to scope the literature and identify pockets of evidence that were amenable to review underpinned many of the decisions that were made in the early stages. Scoping searches using Google Search and Google Scholar (Google Inc., Mountain View, CA, USA), and the initial bibliographic search development, played a key role in the decisions to exclude experiences of the MHA 16 via criminal justice proceedings and to focus on research published after the 1998 Human Rights Act. 22 Throughout this process, the integration of the information specialist in the review team was crucial. The pressing requirement to rapidly scope the available literature meant that our bibliographic database search strategy was developed and ready to run as soon as the final focus of the review was agreed. Supplementary searches were iterative and responsive to the emerging literature, within the prespecified boundaries set out in the protocol. In particular, this included iterative citation searching or ‘snowballing’ of key primary studies. Indeed, this approach to searching is a core part of qualitative searching, but the rapid nature of this review brought to the foreground the importance of this type of searching.

The searches also determined that there were no available frameworks or theories directly relevant to our research objectives that we could use to inform the initial framework for our framework synthesis. This informed our decision to structure our initial framework based on our initial research objectives, and then refine the framework using the themes from the primary studies included in our reviews, followed by thematic synthesis.

• Case study 3: regulating and inspecting integrated health and social care in the UK

In 2018, the NIHR conducted a topic identification exercise within the broad area of professional regulation in UK health care. The exercise generated approximately 30 possible research topics, with some articulated as research questions and others as stated areas thought to be lacking in evidence. The NIHR assessed each topic and prioritised the following questions for referral to the York HSDR review team:

• What factors enable delivery of an effective system of regulation and inspection in an environment where services are increasingly being provided on a multiagency (including third sector) and local basis in or close to people’s own homes?
• How can we overcome the barriers to delivering effective joint regulation and inspection in a way that makes sense from the perspective of the individual accessing the care and services? To what extent is it possible to achieve this without the need for major legislative or structural change?

The research team conducted extensive stakeholder consultation to shape these questions into research questions, define terms and adapt the scope where necessary (see Summary of key challenges/choices and scoping decisions ). There was some PPI in the review project, but at the report-writing and dissemination stages only. Ultimately, we planned to conduct a rapid scoping review to identify and classify published material that could potentially address four key questions:

• What models of regulation and inspection of integrated care have been proposed (including approaches taken in other countries)?
• What evidence is available on the effectiveness of such models?
• What are the barriers to and enablers of effective regulation and inspection of integrated care?
• Can barriers to effective regulation and inspection be overcome without legislative change?

The rationale for conducting a scoping review was based on these mainly descriptive, rather than evaluative, review questions (especially questions 1–3). The protocol was posted on the team’s webpage, 24 as it was not eligible for registration in PROSPERO. The final report was published in full 25 and a brief evidence summary produced. 26

As can be seen by the above contrasting sets of questions, consulting with diverse stakeholders resulted in a broader scope, with a series of research questions focusing on the extent and nature of the evidence that were best suited to a scoping review. A map of the evidence was created, underpinned by the following inclusion criteria:

• Publication type – both empirical and non-empirical publications were eligible for inclusion. Empirical studies could be of a qualitative or quantitative design. Non-empirical publications could include discussion or theory papers, as well as other descriptive pieces, such as editorials. Letters or news articles were excluded, as were publications that reported the findings from inspections of care services.
• Setting – publications were primarily focused on the integration of health and social care provision, for example services delivered jointly by NHS providers and local authorities. However, publications could also focus on care provision that is delivered across other settings/sectors by different professional groups working together, for example across primary or secondary care. Care providers could be in the public, private or third sector, and services could be aimed at both adults and children.
• Focus – publications needed to have a primary focus on the regulation and/or inspection of integrated care. Reference to the governance of services more broadly was not sufficient for inclusion. Integration could be either horizontal or vertical in type and be at a macro-, meso- or micro-level.
• Outcomes – empirical studies could report on any outcome relevant to the regulation and/or inspection of integrated care. This could include issues related to implementation, such as views about barriers and enabling factors. Non-empirical publications could focus on any relevant issue, including proposed models of regulation or outcome frameworks.

Each included publication was coded on various key characteristics, including topic (i.e. regulation or inspection), country, population/setting and document type (e.g. empirical research, models or frameworks, or theoretical). This information was used to produce a high-level descriptive overview, which characterised the nature of current literature on the regulation and inspection of integrated health and social care in the UK, as well as identifying research gaps.

Key challenge 1: reconciling multiple independent stakeholder consultations

The original questions prioritised by the NIHR originated from Health Inspectorate Wales (Merthyr Tydfil, Wales). A teleconference held between the York team and representatives of both Health Inspectorate Wales and Care Inspectorate Wales (Merthyr Tydfil, Wales), provided background and context to their proposed questions. However, before this teleconference could be arranged, the research team had already consulted other key stakeholders about the proposed questions.

Initial contact with representatives from the Professional Standards Authority (London, UK) and Care Quality Commission (London, UK) provided the team with an overview of health-care regulation in the UK and associated research. They expressed a willingness to assist with the proposed work and arranged for researchers to attend (1) the Professional Standards Authority’s Policy and Research Forum and (2) a meeting of the health and social care regulators. The former included representatives from the Professional Standards Authority and various regulatory organisations [e.g. the General Pharmaceutical Council (London, UK), the General Chiropractic Council (London, UK), the Health and Care Professions Council (London, UK), the General Optical Council (London, UK), the General Osteopathic Council (London, UK), the General Medical Council (London, UK), the General Dental Council (London, UK) and the Nursing and Midwifery Council (London, UK)]. The latter included senior managers from the Care Quality Commission, DHSC, the General Dental Council, the Health and Care Professions Council, the Local Government and Social Care Ombudsman (Coventry, UK), the Nursing and Midwifery Council, the General Pharmaceutical Council, the Parliamentary and Health Service Ombudsman (London, UK), the Professional Standards Authority, the Medicines and Healthcare products Regulatory Agency (London, UK), Social Work England (Sheffield, UK) and the General Medical Council. The research team also held a separate teleconference with a representative of the General Medical Council.

In each case, different stakeholders emphasised slightly different areas of interest, deviating to a greater or lesser extent from the original questions. The Health and Social Care Regulators suggested that interdisciplinary regulation of online primary care was the topic of interest. The diverse disciplines and regulatory organisations providing online care provoked questions about effective regulatory oversight and complaints related to care. The potential relevance and utility of the international evidence was particularly emphasised.

The General Medical Council agreed that, even given the differences in the regulatory architecture and frameworks across countries, there was scope to learn from other health and social care systems that face similar issues and potential risks. Of particular interest was multidisciplinary team working: understanding the barriers to, enablers of and issues around responsibility, given joint working and multidisciplinary collaboration.

The representatives of Healthcare Inspectorate Wales and Care Inspectorate Wales described recent policy initiatives to promote the integration of health and social care in Wales and the implications of such policies for regulators. They elaborated on their initial questions to pose a series of fundamental questions about the regulation and inspection of integrated health and social care provision, including what models exist, their effectiveness, the barriers to their implementation and non-legislative means of overcoming these barriers.

Decision 1: share all scoping conversations among the stakeholders and encourage ongoing engagement

It can be seen that – when given exactly the same information to scope a rapid evidence review – different stakeholders proposed differing but inter-related research questions. As these stakeholders are the likely audience for the review, we felt it crucial that they remain engaged with its development. Consequently, as part of the protocol development process, we shared our individual stakeholder discussions with the wider group. An introduction to the draft protocol outlined how these shared discussions informed the basis of a draft scope, which the stakeholders were then invited to further comment on and amend, as appropriate. Each of the stakeholder groups responded positively to this invitation.

Beginning with two detailed questions derived from a prioritisation exercise may have been a barrier to initial engagement from some stakeholders. Some stakeholders would defer to the questions as worded, focusing on interpreting the authors’ intentions rather than expressing their own perception of research priorities. The research team, therefore, encouraged stakeholders to use the initial questions as a starting point for the discussion about research priorities.

Key challenge 2: not all stakeholders are familiar with rapid evidence synthesis

It was clear that stakeholders’ experience and knowledge of rapid evidence synthesis varied widely, particularly in relation to formulating implementable research questions. We wanted to avoid the potential hazard of arriving at a scope that was simply amenable to evidence synthesis rather than reflecting a genuine stakeholder knowledge need.

Decision 2: empower stakeholders to take advantage of the available resource

The research team began each stakeholder group consultation with a presentation to ensure a common level of understanding before beginning the scoping work. This presentation covered:

• the nature of the work conducted by the HSDR Evidence Synthesis Centres
• the types of evidence synthesis and the related research processes
• the role of stakeholder involvement in evidence synthesis
• required stakeholder input for this specific project [e.g. helping develop the scope of the research question(s)].

The presentation aimed to create conditions in which stakeholders could lead the discussion, speak with confidence about their evidence needs and describe how rapid evidence synthesis might serve those needs. The research team could outline the types of research method that might be feasible or suitable for proposed research questions, but were careful not to influence topic-specific discussions. It became clear that stakeholders’ information needs were best served by quickly identifying and organising the apparently diffuse and disparate literature on regulation and inspection of integrated care. Consequently, a rapid scoping review was considered the most appropriate starting point.

Successful scoping of rapid evidence synthesis questions requires the involvement of stakeholders who are knowledgeable, enthusiastic and engaged. However, it also requires researchers to developa framework in which stakeholders can lead conversations about the scope while remaining within the parameters of what can be achieved by a rapid evidence synthesis. This requires researchers to focus on listening and facilitation in the early stages of consultation, moving towards providing stronger guidance on possible methodologies once a level of consensus among stakeholders has been achieved. The time and effort required to develop these processes and practise these skills should not be underestimated.

Case study 4: social care access for BAME and LGBT+ populations – a rapid realist review

This evidence review topic was initially proposed by the Research Programmes Branch – Health and Care Section within the UK’s DHSC. The scoping and subsequent rapid realist synthesis was conducted by the HSDR Evidence Synthesis Centre at the School of Health and Related Research (ScHARR), University of Sheffield. The protocol was published on PROSPERO (CRD42019158250) and was also made publicly accessible on the NIHR’s published Sheffield Evidence Synthesis Centre web page (URL: https://njl-admin.nihr.ac.uk/document/download/2031791 ; accessed 21 June 2021). The review was conducted between November 2019 and the end of June 2020 and was published in 2021. 27

The review focus was initially stated as the following topic:

• addressing diversity and inequalities in access to social care services.

Access to social care services had featured as a dominant theme within the James Lind Alliance’s Adult Social Work Top 10 , 28 reflecting a recent priority setting exercise. Rather than representing a specific priority from the James Lind Alliance exercise, this theme encapsulated several priorities. It was, therefore, felt by the NIHR and the DHSC that coverage of these priorities could be interpreted and understood by targeting the review at groups for whom access to social care could be particularly challenging. Initial discussion with the contacts at the DHSC focused on two particular groups of interest: ethnic minorities, and lesbian, gay, bisexual, transgender plus (LGBT+) people with social care needs. This process emphasises how a broad theme may be refined to particular populations, both as a specific target for policy initiatives and in the assumption that by addressing issues in access for specific groups, this can have wider implications for other population subgroups.

Rather untypically, the review customer (the DHSC) had already conducted a preliminary literature scoping around race disparities in the use of social care. The review team, therefore, faced two particular challenges: (1) how to add value to the initial literature review by complementary activity and (2) how to inform the social care access literature from the extensive work completed in health care. Both of these considerations shaped our subsequent approach to scoping the literature. In addition, the review scope and the perceived relevance of the review was discussed with members of the Sheffield Evidence Synthesis Centre’s PPI Group.

Although classic scoping of effectiveness review questions involves population (or patient type), intervention, comparator and outcomes, with the subsequent addition of study types (i.e. PICOS), once the quantity and quality of available literature has been established by scoping, we have identified the need for addition of a further ‘S’ [population (or patient type), intervention, comparator, outcome, study types, synthesis method (PICOSS)]. This additional ‘S’ relates to the type of ‘synthesis’ (i.e. a preliminary assessment of the literature allows us to identify what type of synthesis will be possible and useful to match the requirements of the topic to the needs of the commissioner). In this instance, we identified that realist methods would extend the evidence base beyond the descriptive literature review that the DHSC team had already explored to gain an understanding of how ‘access’ worked (or did not work) for different groups.

As part of the scoping, our jointly trained lead reviewer/information specialist identified a realist review on the topic of access in primary care. 29 Our team was interested in the extent to which the conceptual models from that study might transfer to social care, particularly in relation to a trajectory or pathway of access.

Our initial scoping identified a significant number of data relating to the use of social care services by refugees and asylum seekers, particular by those with poor mental health. Checking with policy colleagues, via our direct point of contact, revealed that these groups would be included among the identified populations of interest. It further revealed a particular interest in the intersectionality between our two populations: ethnic minorities and LGBT+ people.

Following these explorations, discussions with the DHSC resolved the specifics of the review to the following research questions:

• What are the barriers to and facilitators of accessing social care for (1) ethnic minorities and (2) LGBT+ populations?
• Using ‘if–then–leading to’ statements (i.e. context–mechanism–outcome configurations), can we map access to social care against access pathways to health care (Ford et al. 29 ) to provide additional explanations for what influences access to social care?

We aimed to obtain answers to these questions by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria, using a realist review approach. The rationale for conducting a realist review was to address the primary interest ‘for whom’ (or for which groups) access to social care works, as rapid realist methods have been specifically developed for work with policy-makers and realist methods also allow us to look at the contexts of access to social care and the role of mechanisms in determining the outcomes around access (see also table 13 in Booth et al. 27 ). In addition, the DHSC acknowledged that the characteristics of their topic, which the team matched to the required review type (i.e. limited existing explanatory literature, mainly descriptive), were previously ‘reviewed’ in the UK literature and they were looking for added value from the evidence.

A key feature of realist approaches, however, involves prioritising studies or sources that can inform and develop candidate programme theories (i.e. provisional explanations of how interventions work). In this sense, even though the original research questions were suitably specified for developing the protocol, we continued to refine the focus of the scope as we started to interpret the evidence. An interim remote meeting was arranged between the ScHARR team and the DHSC to prioritise four or five programme theories to be addressed by the review. A notable feature of the review was the need to specify certain concepts early in the process. Table 5 summarises how the scoping process defined the inclusion criteria and shaped the scope.

Summary of operational definitions: access to social care

Key challenges/choices and scoping decisions

Key challenge 1: the role of definitions in specifying relevant concepts.

Dixon-Woods et al. 34 make the distinction between an integrative synthesis ‘where the focus is on summarising data, and where the concepts (or variables) under which data are to be summarised are assumed to be largely secure and well specified’ and an interpretive synthesis with its concern with ‘the development of concepts, and with the development and specification of theories that integrate those concepts’. The reviews specified by the NIHR HSDR programme typically include at least one concept that has to be ‘discovered’ during the course of the review, whether that be unpacking the constituents of an intervention, the diversity of outcomes or, as in this case, in exploring if a health-oriented concept of access and candidacy could usefully be translated to social care. Elsewhere, Dixon-Woods et al. 31 cite the distinction between the question as an ‘anchor’ [i.e. where four elements of population (or patient type), intervention, comparator and outcomes are prescribed and pinned down] and where the question is a ‘compass’ [i.e. where one element is pinned down (e.g. ‘adult social care’) and the implications of variation in the remaining concepts is explored during the course of the review].

These archetypal uses of review questions are, of course, end points of a continuum of approaches between having highly fixed inclusion criteria and questions and very fluid/flexible criteria and questions. Although the concepts of compass and anchor continue to offer useful heuristics at the question-generation stage, additional conceptual development could usefully explore their further utility and application, as they clearly also link to the core trade-off between the importance of the question(s) and the availability of relevant research to answer it (them).

In reality, when working on a review, we can recognise that all concepts are potentially either ‘compasses’ or ‘anchors’. Furthermore, definitions may be semisecure, such as the working definition provided in Thorlby et al. 32 (see Table 5 ), or bound by statute, such as the Equality Act 30 definitions associated with protected characteristics. Definitions may come from one of many different types of source ( Table 6 ). Our lead reviewer for this project leads for diversity and inclusion at a faculty level and so was already familiar with the scope and variety of terms and concepts within this topic.

Types and sources of definitions used during a scoping process

Key challenge 2: adding to the value of previous review work

Frequently, the review work that we undertake involves updating or complementing a previous review. In this case, the DHSC had already undertaken a rapid descriptive literature review. Part of our PICOSS challenge was to convey the added value of our preferred form of synthesis for this particular question. This can be conceived as a two-way dynamic flow between review commissioner and review team, in which the commissioner shares details of topic and research question and, in return, the review team seeks to communicate the synthesis methodology. Scoping, therefore, involved identifying existing evidence syntheses (such as Ford et al. 29 ) that could communicate the potential of our planned review, act as templates for our own methods and outputs, and introduce the methods to be used.

Key to this process is the requirement to be familiar with diverse review methods so that matching between topic/question and methodology can be managed effectively. The emerging science of reviewing research (‘reviewology’) involves familiarisation with different review choices and engaging in an informed negotiation with the review commissioner. Evidence Synthesis Centres, therefore, need to include or have good access to experienced synthesis methodologists to perform this integral function.

Key challenge 3: scope and protocol as a work in progress

As mentioned above, the review team had a final opportunity to determine the scope of the rapid realist synthesis following completion of a process of sensitisation and rapid data extraction that preceded the interim remote meeting with the DHSC staff. The review team prepared and shared 13 candidate programme theories in a technical document, with accompanying illustrative extracts. At a subsequent meeting, the team at the DHSC indicated the relative priority of these programme theories and then confirmed these with policy colleagues. Engaging with such a vivid exemplification of the review scope in practice helped in finally determining the limits of the work. From January 2020 onwards, the review team worked within the scope of each of the five prioritised individual programme theories, selected by DHSC from the initial list, to organise analyse and interpret data within what are essentially ‘mini-reviews’ based on each programme theory. As befits an interpretative review, the parameters of the review product were determined relatively late and from within a wider sampling frame. Furthermore, the limits of the mini-reviews continued to emerge as decisions were made on the scope of each programme theory and their degree of complementarity, synergy or overlap.

The scoping process involves extended negotiations, involving transactions and exchange of ideas, knowledge, references and topical and methodological artefacts between review commissioner and review team in moving to a shared understanding of the problem and its potential for resolution. The choice of review synthesis approach is not a decision to be made once the scope has been finalised. It represents an integral part of the scoping process (i.e. as an additional ‘S’ to the traditional PICOS framework).

More fundamentally, scoping also includes determining the extent to which concepts should be predefined or the extent to which they should be left to emerge from the review process or studies found. Concepts can themselves be viewed individually as ‘anchors’ or ‘compasses’ and definitions may be prespecified (i.e. temporarily fixed for the purposes of the review) or emerge from and give direction to an interpretive review method as part of the ‘findings’.

• Case study 5: review of the effectiveness and implementation of strengths-based approaches to adult social work

This evidence review topic was initially proposed by the Chief Social Worker for Adults within the UK’s DHSC, in August 2019. The scoping and systematic review was conducted by the HSDR Evidence Synthesis team at Exeter and the full review protocol was published on the PROSPERO database (CRD42020166870). 35 The review was conducted between November 2019 and the end of June 2020.

The review topic was initially stated as the following brief question:

• Different models of social work practice: what social work practice works best?

This directly translates into the more precise, typical language of systematic review questions as ‘What is the effectiveness of different models of social work practice?’

By the time we completed scoping the topic and finalised our review protocol, the planned systematic review aimed to address the following two questions:

• What is the effectiveness of different strengths-based approaches used within adult social work?
• What factors enable or inhibit the implementation of different strengths-based approaches in adult social work within the UK?

In other words, we had expanded the initial focus on effectiveness (i.e. ‘what works’) to include evidence on the implementation or embedding of strengths-based approaches into routine social work practice, and this case study mainly explores how that expanded focus arose. Ultimately, the review asked people with lived experience of using adult social care services to comment on research summaries and no service users were involved early enough in the process to shape the review questions and scope.

We aimed to obtain answers to these two questions by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria. Table 7 summarises how the scoping process refined the review questions and defined the inclusion criteria, compared with the initial, briefly stated review topic. Ultimately, as no effectiveness studies that met our inclusion criteria were found, it became a rapid synthesis of qualitative studies of implementation and (as with the review in case study 2) used framework synthesis because of its pragmatic aims and proven value for applied health care and informing policy.

Summary of final inclusion criteria: strengths-based approaches to social work

Key challenge 1: specifying the phenomenon of interest

Although the initially stated topic made no mention of specific models of social work practice, the earliest communications with the Chief Social Worker for Adults usefully and quickly clarified that one broad ‘model’ of social work practice was of most interest, that is strengths-based approaches. Such approaches had been enshrined in policy as part of the Care Act 2014, 36 and were already strongly supported and promoted by the DHSC, the Social Care Institute for Excellence (SCIE) and NICE.

However, from expert advice and reading key reports, it quickly became clear that a strengths-based approach is not a single, well-defined ‘intervention’ or model of practice. Rather, it is a holistic way of working, with a multidimensional set of core ideas and principles about how social care can be effectively and positively provided in a way that respects people’s rights, desires, life and family circumstances, and draws on the capabilities and strengths of the individuals being supported, their families and the communities in which they live. At the same time, we had also been sent a table by the Chief Social Worker for Adults (i.e. the policy customer), which listed various subsidiary approaches, theories and models of practice, on which she had marked ‘those that more clearly fit under strengths-based approaches’.

Therefore, by 3 October 2019, when corresponding with NICE (about key challenge 2), we wrote:

We are aware from our previous scoping searches that there will be few, if any, comparative empirical evaluations of using strengths-based approaches vs. using traditional/usual approaches to social work practice. So we have provisionally, instead, decided to conduct a systematic review of some of the component therapeutic approaches or specific tools that are commonly seen as ways of delivering social work within a strengths-based approach. Rob Anderson, University of Exeter Medical School, 2019, personal communication

In the likely absence of comparative, empirical evidence of the effectiveness of the overall strengths-based approach (based on our own scoping searches), the team had briefly considered conducting a theory-driven review (e.g. a realist review) to better specify the supposed underlying principles and mechanisms of strengths-based approaches (i.e. how the strengths-based approach improves social work outcomes). However, rather than using a review to better understand the effectiveness of strengths-based approaches from a theoretical perspective, we instead chose a more pragmatic approach to meeting the user’s stated evidence need.

Decision 1: focusing on subsidiary, named approaches aligned to strengths-based principles and practice

Mindful of the relatively short (5-month) timeline, we chose to conduct a systematic review of 17 named subsidiary approaches (i.e. models of social work practice and service change that are widely believed to be closely aligned with or to exemplify a strengths-based approach). As well as reflecting the reality of how strengths-based approaches were being fostered and adopted in different local authority adult social care teams, basing our review around such a list of named strengths-based approaches would enable targeted literature searches and give us flexibility at the evidence synthesis stage to not conduct syntheses where studies were either too few in number or too poor in quality. Our review protocol explicitly stated that the rapid review:

. . . will ultimately only include studies evaluating a selection of the approaches which are seen as a priority by the policy customer [DHSC], and [which] have an adequate number of studies for the synthesis to be more valuable. Anderson et al. 35

The Chief Social Worker for Adults initially sent the team a table of named approaches and theories of social work (that had been produced by one local authority), and they had marked those that they saw as most aligned to a strengths-based approach. We supplemented this with an appraisal of named approaches within recent presentations/webinars, government reports and a roundtable meeting report about strengths-based approaches to social work. The final list of 17 strengths-based approaches that formed the basis of our searches and review were based on a table produced from these sources and, subsequently, corroborated by social work/social care expert stakeholders who had commented on the review protocol. Although acknowledging that the list was not exhaustive, the team believed it to capture the main subsidiary approaches that foster a strengths-based approach to practice or organisational change.

In summary, the nebulous definition of strengths-based approaches, being derived from various principles and an approach of how to work with vulnerable people rather than a single distinct model, offered two competing alternatives. We needed to focus on these underlying defining principles and, therefore, how strengths-based approaches improve outcomes (e.g. using a realist review), or focus on selected, named subsidiary approaches that are believed to foster or encapsulate a strengths-based approach, and effectively conduct a systematic review of each of these. For the reasons described above, we chose the latter approach.

Key challenge 2: rapidly becoming familiar with a new domain of policy and practice

The review team have primarily worked within health care and health policy research, and so the world of adult social care and social workers as a specific professional group was unfamiliar to all members of the review team at the beginning of this review. This meant that more time than usual had to be invested in talking to experts and reading key background reports to get to grips with the language, key concepts and basic context, such as what is adult social care? What is social work and social work practice? What are the types of adults and families in the UK who typically need support from social workers?

Decision 2: recruiting a wider group of stakeholders and involving experienced social workers in the review team

Rather than being an identifiable decision, this was a conscious ongoing effort to reach out to academics, practitioners and service leads in the social work field and adult social care services. It was boosted by two significant examples of collegial generosity and luck. First, a professor of social work who had co-authored one of the key recent government reports of strengths-based working was keen to help us in our work. The professor provided detailed commentary and e-mail feedback on emerging sections of the report and synthesis. Second, a health researcher from the Institute of Health Research (University of Exeter Medical School, Exeter, UK) who was a former social worker and had also been a leader and manager of adult social care services in two different local authorities assisted with the review.

Key challenge 3: assuring that we would not duplicate ongoing or recent evidence reviews by others

By following various leads in the literature and a tip-off from our policy customer about ‘a similar piece of work’, we became aware of (1) a recent policy document from NICE and the SCIE, 37 which could have been based on a review of research evidence similar to our emerging topic, and (2) another university-based group conducting a review of the evidence relating to strengths-based approaches to social work. We, therefore, investigated both possibilities, through e-mails and telephone calls, to ensure that our planned review of the evidence relating to strengths-based approaches had not already been conducted.

In 2019, NICE and SCIE had jointly published Evidence for Strengths and Asset-Based Outcomes: A Quick Guide for Social Workers . 37 Knowing that NICE evidence products are typically based on commissioned or internally produced systematic reviews, the review team contacted the Quick Guide authors at NICE to find out what evidence this guide was based on. Ultimately, e-mails and telephone conversations with two members of the social care and leadership team at NICE confirmed that no new or specific systematic reviews had been conducted to inform the Quick Guide. Rather, the evidence comprised a collation of evidence-based recommendations drawn from NICE, SCIE and other authoritative sources.

The second potential duplication of our planned review involved a small team based at the University of Kent (Kent, UK) that we had been told was ‘involved in developing a research proposal with others on strengths-based social work, through NIHR’. We spoke with both the principal investigator and the main researcher on this work to gain a clearer picture of their planned work. They confirmed that an ongoing project, conducted by the Adults Social Care Outcomes Unit (a DHSC policy research unit), involved exploring ‘strengths-based practice in social care’.

However, discussions with the lead researcher clarified that this was effectively a scoping review of relevant literature. We concluded that a broader focus and different methods (e.g. no specific study designs or inclusion criteria, etc.) meant that, notwithstanding shared inclusion of some studies, our systematic review would have a sufficiently different focus and methods (i.e. on effectiveness studies, and on specific, selected subapproaches within a strengths-based approach) to be both distinctive and valuable.

Key challenge 4: avoiding an empty review

A frequent criticism of systematic reviews is the apparently high prevalence of so-called ‘empty reviews’. Although empty reviews may give research commissioners impetus to fund more primary research to fill research gaps and give service commissioners greater freedom to encourage service innovations based on experiential knowledge and theory, they typically have less value in the immediate decision-making context of users of rapid reviews.

Decision 4: to extend the focus of the review to include the implementation of strengths-based approaches

The decision to add a second focus and review question about the implementation of strengths-based approaches was informed by three considerations. These were as follows:

• Our scoping searches and advice from others with much greater experience in the field of social care research, indicated there would be no or, at best, very few comparative effectiveness studies of different strengths-based approaches. We judged that a review addressing the effectiveness question only would likely be an ‘empty review’ (i.e. no studies meeting our inclusion criteria).
• Several of our expert stakeholders indicated that the research needs of adult social care teams were now less to do with whether or not strengths-based approaches are effective and more about how they can be effectively implemented by adult social care teams or individual social workers.
• The view, shared by stakeholders and present in recent publications, that most social workers and other professionals working in social care ‘fundamentally supported a strengths-based approach within adult social work and social care, but often found it difficult to demonstrate, evidence and practice such an approach in practice’. Initial indications that local evaluations of models of strengths-based practice had included a focus on implementation (e.g. to identify enablers of or barriers to the adoption of the particular strengths-based approach).

The fact that strengths-based approaches were already fully promoted by government and other agencies, that they are also widely assumed by social workers to be a positive and beneficial approach (reflecting the core goals and values of the profession) and the likelihood of there being no high-quality effectiveness studies of the overall approach led to us scoping a more useful review in two ways.

First, we added the second focus on the implementation of strengths-based approaches. This meant that if the effectiveness review was empty, we would have still provided a valuable evidence synthesis. Second, and relatedly, we avoided the challenge of defining what comprises or defines a strengths-based approach to social work practice by asking other stakeholders to identify named initiatives and subsidiary approaches that they deemed to be reflecting or fostering a strengths-based approach.

Scoping searches, therefore, played a critical role in defining what potentially synthesisable evidence existed, which fed back into the specification of productive review questions. Owing to the rapid timelines of this review, our main stakeholder at the beginning was the policy customer who was engaged with the process and setting the review goals. However, we were able to corroborate and expand our understanding of the evidence needs and which named initiatives were deemed to foster or exemplify strengths-based working by working with other social care and social work experts during the review, and by building on recent roundtable reports and webinars on strengths-based working. Without this broader engagement, we would have been less confident in the choice of the 17 strengths-based approaches that we ultimately included and less aware of the important differences between them (and, therefore, also less aware of limitations in the ultimate applicability of our findings). The lesson from this is that, even when you have a very engaged and knowledgeable policy customer, it is important to form a balanced view of the evidence needs using a broader range of experts and documentary sources.

• Case study 6: review of hospital-led interventions to reduce the length of hospital stay for planned admissions of older people

This case example was conducted by the Exeter HSDR Evidence Synthesis Centre. The review protocol was published on the PROSPERO database (CRD42017080637). 38 The review was conducted between September 2017 and October 2018 and published in 2020 (with a pause of 3 months in early 2018 to address an urgent policy question) (see Case study 2: review of experiences of the ‘nearest relative’ provisions of the Mental Health Act 1983 ). 39

The scope of the review was initially discussed at a teleconference between the review team and the scientific adviser at NIHR HSDR. The aims of the meeting were to obtain a preliminary understanding of (1) the background to the decision to prioritise the topic for review, (2) the main areas of uncertainty and (3) the purpose of the proposed review.

Two key aspects of the discussion informed our next steps:

• Although the decision to focus on this area had partly been informed by key published references, the topic was considered a priority for the HSDR programme, with no other identifiable policy customer.
• admission avoidance and non-emergency (hospital) care
• organisation of hospitals (i.e. hospital elements of control for planned care)
• population groups for which reducing length of stay may be particularly relevant (e.g. people receiving palliative care, people with dementia or delirium and people with learning disabilities).

Following a teleconference with the NIHR commissioners, a member of the review team ‘suggested that intervention or programme ideas could be looked at for [intended impacts on] length of stay – although narrow’ and it was agreed that this focus could extend to evidence on other outcomes of initiatives with this aim. This helped confirm the focus for our further exploration and scoping searches (i.e. a focus on interventions that explicitly aimed to reduce length of stay).

Given considerable uncertainty about the most useful review topic, our next steps were to familiarise ourselves with the topic and the body of evidence available. We (1) conducted broad scoping searches and (2) contacted local clinical experts with knowledge and expertise of caring for patients during planned hospital admission. The review team also met on three occasions with a group of four adults aged > 60 years, including one meeting on checking the review’s planned focus. Each individual had experience of being admitted to hospital overnight for a planned procedure.

On completion of scoping the topic and finalising the review protocol, the planned systematic review aimed to address the following two questions:

• What is the effectiveness of hospital-led multicomponent interventions to reduce length of inpatient stay in hospitals for older adults following planned admission?
• What is the cost-effectiveness of hospital-based multicomponent interventions to reduce length of inpatient stay in hospitals for older adults following planned admission?

We aimed to obtain answers to these two questions by identifying, summarising and synthesising evidence from studies that met specific inclusion criteria that are typical of systematic reviews of effectiveness and cost-effectiveness, seeking quantitative outcome data from rigorous comparative evaluations [such as randomised controlled trials (RCTs)]. 9 Table 8 summarises how the scoping process refined the review questions and defined the inclusion criteria, compared with the initially stated review topic.

Summary of key challenges and decisions during scoping

Key challenge 1: absence of an explicit policy customer or question.

With no explicit policy customer, it was not easy to identify a clear and specific clinical or policy need or evidence gap, within a given population. Our initial consultations with the HSDR programme provided insight into specific populations of interest; however, we did not know what evidence already existed for these populations or the scale and importance of the problems relating to their planned hospital admissions. Conversely, several relatively recent, high-quality systematic reviews indicated where there was an abundance of research and also which evidence had already been synthesised. We were also informed by our research commissioners about another review that was being conducted, which focused on ‘early discharge’-style interventions and mainly encompassed hospital-at-home and social care interventions after patients left hospital.

Decision 1: conduct broad and targeted literature searches to try and identify an unsynthesised body of relevant primary research

With a broad domain of health services only (planned hospital admissions) and a general perceived need for better evidence relating to it, we decided to conduct scoping searches to identify what questions, patient groups and hospital procedures had been the focus of primary research, and which evidence had already been effectively summarised and synthesised by others. This very quickly led to a second challenge.

Key challenge 2: lots of previous research and quite a few previous systematic reviews of it – but was there a gap?

Owing to the substantial cost of hospital admissions within all modern health systems, there has been a focus for decades on implementing and evaluating various types of service changes that aim to avoid hospital admissions in the first place (i.e. admissions avoidance) or shorten them by enabling people to be discharged earlier and supported at home or in other non-hospital settings. Increasingly, such intermediate care, hospital-at-home and early supported discharge schemes have also been seen as ways to further policy goals of person-centred care and, relatedly, providing care closer to or in people’s homes.

This has produced a wealth of research and also systematic reviews of that research. However, our research commissioners had felt that an evidence need remained in relation to this topic, but it was not clear if that need had already been met by primary research or syntheses. Early scoping activity centred on three population groups believed, by our commissioner, to exhibit greater uncertainty: (1) people with learning disabilities, (2) palliative care and (3) people with dementia or delirium.

Initial scoping searches of Ovid MEDLINE and Google Scholar identified several systematic reviews of the effectiveness of enhanced recovery after surgery and enhanced recovery programme interventions. However, we found no reviews that specifically focused on the three population groups of interest. We followed up this finding in two different ways. First, we searched for primary studies to ascertain whether or not a review of interventions focusing on these population groups would be feasible. Second, because we were unsure whether or not primary research would have been conducted specifically on these population groups, we considered the possibility of conducting a review of reviews, using subgroup analyses of the population groups of interest within systematic reviews of primary studies of general population groups. We also initiated discussions with external experts, colleagues and stakeholders about our proposed question.

Decision 2: focusing on hospital admissions of older people

These discussions guided us towards our final decision to focus on older people in our review, enabling us to consider evidence relating to at least two of the three population groups of interest: (1) people with dementia or delirium and (2) palliative care:

• A research associate at the Nuffield Trust (London, UK), and author of their related report, 40 thought that achieving shorter hospital stays for older people was highly important because they are the most vulnerable to hospital-acquired harms.
• A professor of ageing and rehabilitation at the University of Exeter emphasised that older people would not necessarily receive an intervention unless it was deemed effective enough and, for this reason, enhanced recovery programmes could be an important option not being considered often enough for older people.
• A consultant geriatrician at Royal Devon and Exeter NHS Foundation Trust (Exeter, UK) suggested that we consider the need for evidence to inform ‘doing the right thing for the person’ in the treatment of older people with comorbidities, where interventions may not add much to quality of life. This was based on the assumption that enhanced recovery programmes are generally accepted as safe and effective, based on research evidence in younger and less complex patients.

Therefore, although the effectiveness of enhanced recovery programmes and similar initiatives to enable people to leave hospital earlier seemed to be accepted in general, there was shared uncertainty about whether or not such programmes would always be appropriate and effective in older people and, if not, for whom they would be effective. Returning to our scoping activity, we confirmed a gap in the synthesised research literature around organisational (i.e. hospital-based or hospital-led) interventions aiming to reduce the length of stay for older people following planned procedures.

Key challenge 3: defining and operationalising hospital-led ‘organisational interventions’ that aimed to reduce length of stay

Searching for evaluations of hospital-based interventions and hospital-led interventions to reduce length of stay revealed numerous studies that compared specific treatments or procedures. For example, many evaluations compared different drugs or protocols for anaesthesia, laparoscopic surgery with open surgery or different types of postoperative physiotherapy. Therefore, the team needed to distinguish ‘treatment interventions’ from broader ‘organisational interventions’ and changes in how whole teams work to support earlier discharge. In line with Medical Research Council definitions of complex interventions (as comprising multiple components 41 ), checklists for describing components of enhanced recovery after surgery protocols 42 and Cochrane Effective Practice and Organisation of Care group checklists, 43 we decided to distinguish our interventions of interest as multicomponent interventions, as follows:

• Intervention has multiple components. As defined in the guidance for using the Intervention Complexity Assessment Tool for systematic reviews (iCAT_SR), commissioned by Cochrane, an intervention component is ‘a discrete, active element of an intervention that could be implemented independently of other elements’ (p. 6). 43 Therefore, for inclusion, interventions should contain more than one component (e.g. education and exercise).

Therefore, organisational (multicomponent) hospital-based or hospital-led interventions in older people became the ultimate focus for our systematic review, and distinguished this review from those focusing explicitly on alternatives to hospital care or those focusing on single-component interventions (e.g. laparoscopic vs. open surgery or comparing different regimes for anaesthesia and analgesia for planned surgery). We also excluded strategies that were focused on discharge planning or pre-treatment assessment only (e.g. comprehensive geriatric assessment alone), as these do not affect other stages of the hospital stay.

In the absence of reliable intervention labels, the second challenging aspect of defining our included interventions was how to decide when an intervention aimed to reduce length of hospital stay (and, ideally, in a way that could be plausibly identified from a journal paper’s abstract). Our two main options were (1) to include only those papers that explicitly stated that its intention was to reduce length of stay (i.e. including papers in which length of hospital stay was the stated primary outcome), or (2) to also include those in which the aims implied that a reduction in length of stay was one of the aims [e.g. if the paper stated that it aimed to ‘improve’ or ‘enhance’ recovery (or equivalent language, e.g. ‘accelerate rehabilitation’)].

The scoping of this systematic review exemplified many of our HSDR reviews in requiring iteration between the ‘technical’ (or data-driven, bottom-up) process of defining and refining searches to quantify research on different potential questions, and the ‘collective learning’ of a small group of reviewers working with clinical or research commissioner stakeholders to ‘home in’ on the most useful review questions (as a top-down process).

From both the searches and suggestions from the research commissioners, key published sources informed our understanding of the context of the review (e.g. the possible specification of organisational interventions), highlighted key pre-existing evidence syntheses on reducing patient length of stay and helped to develop our own search strategy. However, it was the combined views of several clinical and policy stakeholders that, on top of our learning from the searches, provided the critical decision and narrower focus (i.e. on older people). This decision allowed the scope to really take shape and helped to identify the main evidence gap to be addressed by our review. As one of our reviewers later reflected, such suggestions ‘stopped it becoming an academic exercise’. Our clinical stakeholders also aided our understanding of the stages of the patient journey through hospital and how these may differ according to age group and urgency of procedure. Again, this insight supported our definition of multicomponent, hospital-led interventions.

In conclusion, after reflection by the team, this scoping exercise was considered to have been ‘evidence’ rather than question driven. It started without a clearly defined question and the final scope and questions were based on scoping searches that showed a considerable body of unsynthesised research evidence related to stated evidence needs. In the absence of a clearly specified question at the beginning or a well-defined service delivery or policy area of uncertainty, it represented a general hunch about a lack of clear evidence relating to interventions that aimed to shorten the length of hospital stays for planned hospital admissions. This hunch was explored through considerable iteration between scoping searches and consulting clinical stakeholders. The ‘breakthrough’ decision, which provided an answerable review question that would likely yield useful evidence without too many included studies, was to focus on older people. All other decisions, including to focus on multicomponent and comprehensive models of care, such as enhanced recovery programmes, can be seen as following this main decision on the patient group. As a bonus, the focus on older people enabled the review to seek evidence in relation to people with dementia or delirium (another priority group mentioned by our commissioners).

Case study 7: implementation of interventions to reduce preventable hospital admissions for cardiovascular or respiratory conditions – an evidence map and realist synthesis

This evidence review topic was initially proposed by the HSDR programme, with no direct identifiable policy customer. Negotiations about the scope, therefore, took place with the HSDR technical adviser. Scoping and the subsequent evidence map and realist synthesis was conducted by the HSDR Evidence Synthesis team at ScHARR. The review was conducted in multiple stages between September 2017 and April 2019 and published in 2020. 44

The review topic was initially stated in broad terms:

• interventions for preventing unnecessary hospital admissions.

Given that comparatively little primary research had focused on how to support the implementation of interventions to reduce preventable hospital admissions, an evidence map and realist synthesis was designed to address the following research question:

• With regard to the implementation of interventions to reduce preventable hospital admissions for cardiovascular and respiratory conditions, what works, for whom, how and in what circumstances?

The rationale for conducting a realist review was that the review’s commissioners (i.e. the NIHR HSDR programme) asked the team to consider the interventions as ‘proven interventions’ and, therefore, to provide greater understanding of how interventions that have been shown to reduce admissions for cardiovascular and respiratory conditions work in practice. The preceding mapping review was primarily to inform the sampling frame for the realist review.

Considerable effort and energy was expended at the beginning of the review to define the term ‘unnecessary’ (hospital admissions) from the originally stated review topic. Eventually, the team decided that conceptualising ‘unnecessary’ in terms of appropriateness was the realistic approach. The team and HSDR adviser agreed to focus on conditions characterised as ambulatory or primary care sensitive conditions (ACSCs). Ambulatory services are those services, like hospital outpatient services, where people attend and leave the appointments during the same day, and patients are, therefore, not classed as admitted patients and do not need an overnight stay. The logic is that where the severity or risk of acute episodes of a chronic condition is sensitive to the amount or quality of primary care or ambulatory care, then escalations and admissions to hospital can be prevented.

Table 9 summarises how the commissioned review sought to broaden and interpret the earlier review by Purdy et al. 45

Comparison of previously published review and our final inclusion criteria

Key challenge 1: targeting a representative part from a larger ‘whole’

An initial challenge was to select, from a long list of ACSCs on the DHSC website, a suitably productive focus to exemplify and, hopefully, to encapsulate issues for conditions generally. A frequent criticism is that realist syntheses fail to capitalise on the inherent analytical capacity of ‘what works for whom under what circumstances’ by failing to link qualitative insights to ‘what works’ (i.e. the effectiveness information). To avoid this pitfall, the team explicitly constructed a table ( Table 10 ) that mapped the interim conclusions on effectiveness to different ACSCs. It immediately became apparent that cardiovascular or respiratory conditions had been sufficiently explored and adequately supported as effective to make these an appropriate focus for a realist inquiry. A telephone conversation with the technical adviser at the HSDR programme examined the case for each of the candidate conditions, in turn, to finally resolve the review scope. This approach to scoping, which resolves the coverage of a review by optimising both relevance and the likely explanatory power of the evidence, represents a risk-averse strategy when commissioning rapid reviews.

Conclusions on intervention effectiveness from Purdy et al.

Realist synthesis typically requires prioritisation of programme theory to explore the phenomenon of interest in sufficient depth. The team had to ensure that, having identified review capacity to explore up to three chosen candidate programme theories, they optimised the analytical and explanatory power of their chosen foci. In consultation with the technical adviser from HSDR and the ScHARR Standing Advisory Group on Public Participation, the team selected foci to represent patient, carer and GP perspectives, respectively, within the candidate programme theory.

Key challenge 2: establishing comparability with the target population across diverse study populations

In a realist synthesis, evidence is excluded only if it does not relate to or inform the development of the programme theory. The practical focus of this review required that we exclude evidence with limited transferability to the NHS, such as avoidable admissions in low- and middle-income countries. In line with this decision, we prespecified five countries from which to derive direct evidence to the review, namely the UK, USA, Australia, Canada and New Zealand. These ‘big five’ were selected because of comparable health-care systems to the UK setting. The team continued to engage with the wider evidence base through systematic reviews, opinion pieces and direct reference to individual study reports, particularly where authors connected interventions to the UK context.

Key challenge 3: maximising the utility of review outputs

Much of the emphasis of published descriptions of scoping focuses on determining the conceptual boundaries, methodology options and logistic constraints of an individual review. In comparison, little attention has focused on the role of scoping in shaping the knowledge end product from the very beginning of the process. The scoping process offers an opportunity for commissioners and stakeholders, through their engagement, to exert an early influence on tailoring the final product to their requirements. So, for example, the focus on cardiovascular and respiratory conditions was not determined by evidence on effectiveness only, but also from the commissioner perspective, as articulated by the NIHR programme staff. For the review team, the resultant risk-averse strategy minimises the chances of producing a product that is neither useful nor appropriate. Scoping different options is one way of modelling different review configurations without embarking too far down a particular route.

Unlike other review processes that typically prioritise rigour and reproducibility (e.g. the internationally regarded NICE evidence synthesis products, with their explicit interest in effectiveness and cost-effectiveness), the emphasis of the HSDR programme has generally been to optimise a third ‘R’ of relevance (e.g. encompassing feasibility of implementation, acceptability and sustainability in different organisational contexts). This must continue to be underpinned by the preceding considerations of rigour/reproducibility. Scoping offers an early opportunity to sensitise the context within which the review is eventually situated. In this connection, subsequent attempts by the centres to further improve their working relationships with the patients, public and commissioners is particularly germane. Necessarily, this process of shaping review outputs does not simply involve determining the content, but also entails stimulating receptivity for the form of the final review. Review teams spend a significant amount of time, both in conversations and through the written protocol, cultivating the expectations of the commissioners with regard to less-standard review products, such as realist syntheses.

In a research arena that is increasingly populated by systematic reviews and other types of evidence synthesis, this review and case study has demonstrated the increasingly frequent call to build on the foundations provided by a previous review. Previous reviews can be used in numerous ways (likened to the choice between bulldozing and reusing materials from an existing structure, building an extension or constructing a separate and adjacent annexe). In this particular case, the extension model was most evident. The review shaped the choice of a specific subset of studies that were extended and examined closely. Data on the viability of the chosen conditions, and the likelihood of their achieving an effect, drove the subsequent realist inquiry into how these interventions work. A long list of potential conditions to be examined was slimmed down rapidly into a short list of ambulatory sensitive conditions of potential interest. Conversely, this ‘extension’ approach required good familiarity with the original review, its implications and the definitions encapsulated within its methods.

The existence of a relevant systematic review also reduced some of the synthesis workload, allowing the team valuable time for exploring experimental initiatives reported in the grey literature or located in the ‘.nhs.uk’ internet domain. HSDR topics typically exhibit a shared concern with the established and evaluated, and the innovative and experimental. Although it is important to preserve the separate contributions of these two types of evidence, variously privileging rigour or relevance, the prior existence of a systematic review of effectiveness studies 45 released search effort for exploration of ephemeral contributions. Potentially, NHS service initiatives extend the evaluation horizon by showcasing locally developed solutions with the, as yet unevaluated, potential to deliver health benefit. Faced with concentrated timescales, it is tempting to prioritise a formal scoping search of one or more bibliographic databases when the contribution of web sources may also be important.

Case study 8: access to primary and community health-care services for adults with intellectual disabilities – a mapping and targeted systematic review

This evidence review topic was initially proposed at the annual review meeting of the three evidence synthesis centres with members of the NIHR HSDR programme. The meeting forms the first stage of the process of scoping, where the teams identify prior experience and expertise in relation to a long list of topics generated by the HSDR priority generation process. At this time, the ScHARR review team identified a key researcher with expertise and networks in intellectual disabilities (IDs) to work with the review team to deliver against the identified topic.

The exact timing of this review was negotiated around the availability of the researcher and a joint reviewer/information scientist was scheduled to project manage the topic. As the review eventually took the form of an extensive mapping review, it was not formally included on PROSPERO. However, the protocol is published as a supplementary appendix to the final report. The review protocol was finalised in May 2018, following 2 months of scoping and iteration with the HSDR team, and the final report was delivered in December 2018 and published in 2020. 46

The review topic was initially stated as follows:

• access and quality of health services for people with learning disability.

Within the context of the NIHR HSDR programme, this was a landmark topic, returning to the scene of a previous review by its predecessor, the Service Delivery and Organisation programme. 47 – 50 The ‘footprint’ of this preceding review influenced several subsequent reviewing decisions, emphasising how each new synthesis fits within the evidence landscape. Indeed, even the title of the original review illustrates how both the topic and methodology had advanced. The title identified the output generically as a ‘literature review’, compared with our ‘mapping and targeted systematic review’, and the topic was described as ‘learning disabilities’ compared with the recent ‘IDs’.

By the time we undertook this review, we had consolidated lessons learned from the previous 3-year work programme. 11 This endorsed a three-stage review process for most evidence synthesis topics, namely mapping, scoping and then systematic review. 51 Although some researchers use mapping and scoping interchangeably, these processes are becoming distinct as their respective methods evolve. Mapping represents an attempt to obtain one’s bearings and take stock of the nature and quantity of research within a defined wider area of potential study, often with a view to identifying synthesis gaps where a subsequent potential review might be located. Scoping represents a process for identifying the limits or bounds to a specific planned review, whereby the implications in terms of both logistics and conceptual scope are explored, the limits and quantities of likely literature gauged and potential inclusion and exclusion criteria piloted. Therefore, scoping can be likened to drawing a line to describe the boundary of a review (i.e. deciding what is inside and what is outside its scope), whereas mapping is more about exploring and describing what studies are within particular bounds.

Although not every review requires a formal output at all three stages, these processes of mapping, scoping and reviewing feature in almost all of the three evidence synthesis teams’ projects. Therefore, we authored a protocol that involved two components: (1) a mapping review to offer breadth and then, following focused scoping, (2) a targeted systematic review to complement this with depth.

The mapping review aimed to address the following questions:

• What are the gaps in evidence about access to primary and community health care for people with ID?
• What are the barriers to accessing primary and community health-care services for people with ID and their carers?
• What actions, interventions or models of service provision improve access to health services for people with ID and their carers?

One feature of this scoping process was consulting people with IDs, family carers and formal paid carers so that the review could be informed by the views and experiences of stakeholders (for more details, see Appendix 1 , Table 13 ). This consultation had multiple aims, namely to:

• illuminate the model of access to health care for people with IDs
• inform and refine our search strategies by identifying barriers to accessing health care and any solutions developed
• identify gaps in the literature.

We contacted the clinical director and senior commissioning manager of a Clinical Commissioning Group for services for people with IDs and asked them to identify relevant community groups for people with IDs and their carers. We sent information about the review to these groups and asked to visit them to discuss their experiences of accessing health care. Discussions were loosely guided by a topic guide, drawn from the previous review, covering how people identify a health need, what actions they take, the issues influencing their decision to take a particular course of action and the barriers to and facilitators of their access and use of the chosen service. The barriers and facilitators were used to identify relevant search terms and for future comparison with the barriers and facilitators identified in the qualitative literature.

Other scoping decisions proved important to the direction of the review. The original review distinguished between access and efficiency in focusing on ability to use a service (i.e. access), rather than whether or not the service was provided to a high standard (i.e. efficiency/effectiveness). In contrast, the update review sought studies reporting the effectiveness of any measures or interventions designed to improve access to the relevant services. Comparing the scopes of the original and more recent reviews illustrates how the HSDR programme, and its associated science, has developed and expanded how it conceptualises accessibility and related aspects of service delivery over the last decade or so.

The subsequent targeted systematic review focused on evidence from the UK, building on the findings of the mapping review. Key elements of the targeted systematic review were:

• development of the research questions based on the findings of the mapping review and information from PPI meetings
• a focused systematic database search following inspection of the mapping review findings;
• a full data extraction of relevant studies
• a quality assessment of included full peer-reviewed papers and no formal quality assessment of conference abstracts or grey literature.

Key challenge 1: defining ‘relevance’

This review would include evidence from the UK only. This decision was taken to ensure relevance and to assist in managing the large volume of literature on this topic. A key scoping decision is which interpretation of relevance and context makes most sense in relating the study populations to the target population within each review. In some reviews, relevance is defined simplistically, as either geographical or chronological proximity. In other situations, relevance relates to similarities in how interventions work (i.e. conceptual homogeneity). For example, primary care settings for this review represent a ‘first point of contact’ for ID patients or their carers. In either case, the review team has to determine how to incorporate relevance into their review, 7 either formatively, by methods used in the review, or summatively, in using one of many tools available for transferability or applicability. 52 , 53

Key challenge 2: ‘mapping as scoping’

Broad topics are often characterised by uncertainties in relation to the quantity and quality of research. With limited time, the review team must decide how best to optimise the review resource. ‘Empty’ reviews are not helpful to decision-making, and yet overpopulated reviews are likely to either overrun timewise or prove superficial in their analysis and interpretation. A mapping review offers the possibility to offer breadth before the depth of a targeted systematic review. For this mapping review, initial descriptive analysis took place based on title and abstracts only. Subsequently, a subset (of approximately 20%) of the overall studies was sifted from the original retrieval results, supplemented by grey literature searches to form the data set for the targeted review. This approach keeps the scope as elastic as possible until informed decisions on coverage have been made. At this point, a formal protocol is finalised to provide the necessary scientific rigour. A further benefit of this staged approach is that it diversifies review outputs for different target audiences. Those requiring broad coverage can use the mapping review and those requiring a filtered approach that includes quality assessment are presented with a conventional systematic review.

Key challenge 3: building on previous systematic reviews

Although it may seem attractive to build on previous systematic reviews, the complexity of review decisions may offer minimal advantage from starting a review de novo. As illustrated by this review, not only had the preferred terminology changed, but also different fundamental assumptions about how to interpret ‘access’ shaped the subsequent review. Search terms need to be checked for new index terms and for the emergence of new terminology around the user community. Searches must be specified from the date of last search (not from publication date) together with overlap to avoid key items falling down a gap between reviews. Will the original data sources be replicated for fidelity to the original review? In the case of this particular review, the information specialist revisited the original list of sources to modify, and even improve, the original research. They also checked new index terms and terminology to ensure that the strategies continued to meet the requirements of the current review. This tension between comparability and contemporaneity needs to be managed according to the intended purpose of the review. Above all, the review team needs to decide whether to build on the foundation of the previous review, dismantle the previous review and reuse its original studies only alongside recent additions, or simply construct their own review, overlapping with the previous review.

This review and case study has shown the value of recognising the distinct processes of mapping, scoping and systematic reviewing in managing the overall review, regardless of whether or not these lead to formal nominated review outputs. In general, unless the review customer has already conducted reliable searches and scoping of what evidence exists, some initial mapping and/or scoping needs to be conducted before any useful systematic review can be planned and conducted.

The case study also showed the potential contribution of different stakeholders in informing the scope and coverage of the review, including members of the public (in this case, adults with IDs). Stakeholder involvement included helping to identify gaps in the literature.

The case study also highlighted challenges that arise from changes in terminology and, importantly, adjustments in conceptualisation, as reflected in successive versions of related reviews. This may mean that a previous related systematic review cannot simply be updated, and the focus and intended value of the new review has to be carefully articulated. Even, as here, where the new systematic review is justified as updating and complementing previous reviews, the additional time and effort remains considerable compared with starting a new review from scratch.

• Cite this Page Anderson R, Booth A, Eastwood A, et al. Synthesis for health services and policy: case studies in the scoping of reviews. Southampton (UK): NIHR Journals Library; 2021 Aug. (Health Services and Delivery Research, No. 9.15.) Chapter 3, Results.
• PDF version of this title (1.1M)

In this Page

• Case study 1: rapid evidence synthesis of ‘Digital-First Primary Care’
• Case study 2: review of experiences of the ‘nearest relative’ provisions of the Mental Health Act 1983
• Case study 4: social care access for BAME and LGBT+ populations – a rapid realist review
• Case study 7: implementation of interventions to reduce preventable hospital admissions for cardiovascular or respiratory conditions – an evidence map and realist synthesis
• Case study 8: access to primary and community health-care services for adults with intellectual disabilities – a mapping and targeted systematic review

Other titles in this collection

• Health Services and Delivery Research

Recent Activity

• Results - Synthesis for health services and policy: case studies in the scoping ... Results - Synthesis for health services and policy: case studies in the scoping of reviews

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

Jump to navigation

Cochrane Training

Chapter 9: summarizing study characteristics and preparing for synthesis.

Joanne E McKenzie, Sue E Brennan, Rebecca E Ryan, Hilary J Thomson, Renea V Johnston

Key Points:

• Synthesis is a process of bringing together data from a set of included studies with the aim of drawing conclusions about a body of evidence. This will include synthesis of study characteristics and, potentially, statistical synthesis of study findings.
• A general framework for synthesis can be used to guide the process of planning the comparisons, preparing for synthesis, undertaking the synthesis, and interpreting and describing the results.
• Tabulation of study characteristics aids the examination and comparison of PICO elements across studies, facilitates synthesis of these characteristics and grouping of studies for statistical synthesis.
• Tabulation of extracted data from studies allows assessment of the number of studies contributing to a particular meta-analysis, and helps determine what other statistical synthesis methods might be used if meta-analysis is not possible.

Cite this chapter as: McKenzie JE, Brennan SE, Ryan RE, Thomson HJ, Johnston RV. Chapter 9: Summarizing study characteristics and preparing for synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook .

9.1 Introduction

Synthesis is a process of bringing together data from a set of included studies with the aim of drawing conclusions about a body of evidence. Most Cochrane Reviews on the effects of interventions will include some type of statistical synthesis. Most commonly this is the statistical combination of results from two or more separate studies (henceforth referred to as meta-analysis) of effect estimates.

An examination of the included studies always precedes statistical synthesis in Cochrane Reviews. For example, examination of the interventions studied is often needed to itemize their content so as to determine which studies can be grouped in a single synthesis. More broadly, synthesis of the PICO (Population, Intervention, Comparator and Outcome) elements of the included studies underpins interpretation of review findings and is an important output of the review in its own right. This synthesis should encompass the characteristics of the interventions and comparators in included studies, the populations and settings in which the interventions were evaluated, the outcomes assessed, and the strengths and weaknesses of the body of evidence.

Chapter 2 defined three types of PICO criteria that may be helpful in understanding decisions that need to be made at different stages in the review:

• The review PICO (planned at the protocol stage) is the PICO on which eligibility of studies is based (what will be included and what excluded from the review).
• The PICO for each synthesis (also planned at the protocol stage) defines the question that the specific synthesis aims to answer, determining how the synthesis will be structured, specifying planned comparisons (including intervention and comparator groups, any grouping of outcome and population subgroups).
• The PICO of the included studies (determined at the review stage) is what was actually investigated in the included studies.

In this chapter, we focus on the PICO for each synthesis and the PICO of the included studies , as the basis for determining which studies can be grouped for statistical synthesis and for synthesizing study characteristics. We describe the preliminary steps undertaken before performing the statistical synthesis. Methods for the statistical synthesis are described in Chapter 10 , Chapter 11 and Chapter 12 .

9.2 A general framework for synthesis

Box 9.2.a A general framework for synthesis that can be applied irrespective of the methods used to synthesize results

Box 9.2.a provides a general framework for synthesis that can be applied irrespective of the methods used to synthesize results. Planning for the synthesis should start at protocol-writing stage, and Chapter 2 and Chapter 3 describe the steps involved in planning the review questions and comparisons between intervention groups. These steps included specifying which characteristics of the interventions, populations, outcomes and study design would be grouped together for synthesis (the PICO for each synthesis: stage 1 in Box 9.2.a ).

This chapter primarily concerns stage 2 of the general framework in Box 9.2.a . After deciding which studies will be included in the review and extracting data, review authors can start implementing their plan, working through steps 2.1 to 2.5 of the framework. This process begins with a detailed examination of the characteristics of each study (step 2.1), and then comparison of characteristics across studies in order to determine which studies are similar enough to be grouped for synthesis (step 2.2). Examination of the type of data available for synthesis follows (step 2.3). These three steps inform decisions about whether any modification to the planned comparisons or outcomes is necessary, or new comparisons are needed (step 2.4). The last step of the framework covered in this chapter involves synthesis of the characteristics of studies contributing to each comparison (step 2.5). The chapter concludes with practical tips for checking data before synthesis (Section 9.4 ).

Steps 2.1, 2.2 and 2.5 involve analysis and synthesis of mainly qualitative information about study characteristics. The process used to undertake these steps is rarely described in reviews, yet can require many subjective decisions about the nature and similarity of the PICO elements of the included studies. The examples described in this section illustrate approaches for making this process more transparent.

9.3 Preliminary steps of a synthesis

9.3.1 summarize the characteristics of each study (step 2.1).

A starting point for synthesis is to summarize the PICO characteristics of each study (i.e. the PICO of the included studies, see Chapter 3 ) and categorize these PICO elements in the groups (or domains) pre-specified in the protocol (i.e. the PICO for each synthesis). The resulting descriptions are reported in the ‘Characteristics of included studies’ table, and are used in step 2.2 to determine which studies can be grouped for synthesis.

In some reviews, the labels and terminology used in each study are retained when describing the PICO elements of the included studies. This may be sufficient in areas with consistent and widely understood terminology that matches the PICO for each synthesis. However, in most areas, terminology is variable, making it difficult to compare the PICO of each included study to the PICO for each synthesis, or to compare PICO elements across studies. Standardizing the description of PICO elements across studies facilitates these comparisons. This standardization includes applying the labels and terminology used to articulate the PICO for each synthesis ( Chapter 3 ), and structuring the description of PICO elements. The description of interventions can be structured using the Template for Intervention Description and Replication (TIDIeR) checklist, for example (see Chapter 3 and Table 9.3.a ).

Table 9.3.a illustrates the use of pre-specified groups to categorize and label interventions in a review of psychosocial interventions for smoking cessation in pregnancy (Chamberlain et al 2017). The main intervention strategy in each study was categorized into one of six groups: counselling, health education, feedback, incentive-based interventions, social support, and exercise. This categorization determined which studies were eligible for each comparison (e.g. counselling versus usual care; single or multi-component strategy). The extract from the ‘Characteristics of included studies’ table shows the diverse descriptions of interventions in three of the 54 studies for which the main intervention was categorized as ‘counselling’. Other intervention characteristics, such as duration and frequency, were coded in pre-specified categories to standardize description of the intervention intensity and facilitate meta-regression (not shown here).

Table 9.3.a Example of categorizing interventions into pre-defined groups

* The definition also specified eligible modes of delivery, intervention duration and personnel.

While this example focuses on categorizing and describing interventions according to groups pre-specified in the PICO for each synthesis, the same approach applies to other PICO elements.

9.3.2 Determine which studies are similar enough to be grouped within each comparison (step 2.2)

Once the PICO of included studies have been coded using labels and descriptions specified in the PICO for each synthesis, it will be possible to compare PICO elements across studies and determine which studies are similar enough to be grouped within each comparison.

Tabulating study characteristics can help to explore and compare PICO elements across studies, and is particularly important for reviews that are broad in scope, have diversity across one or more PICO elements, or include large numbers of studies. Data about study characteristics can be ordered in many different ways (e.g. by comparison or by specific PICO elements), and tables may include information about one or more PICO elements. Deciding on the best approach will depend on the purpose of the table and the stage of the review. A close examination of study characteristics will require detailed tables; for example, to identify differences in characteristics that were pre-specified as potentially important modifiers of the intervention effects. As the review progresses, this detail may be replaced by standardized description of PICO characteristics (e.g. the coding of counselling interventions presented in Table 9.3.a ).

Table 9.3.b illustrates one approach to tabulating study characteristics to enable comparison and analysis across studies. This table presents a high-level summary of the characteristics that are most important for determining which comparisons can be made. The table was adapted from tables presented in a review of self-management education programmes for osteoarthritis (Kroon et al 2014). The authors presented a structured summary of intervention and comparator groups for each study, and then categorized intervention components thought to be important for enabling patients to manage their own condition. Table 9.3.b shows selected intervention components, the comparator, and outcomes measured in a subset of studies (some details are fictitious). Outcomes have been grouped by the outcome domains ‘Pain’ and ‘Function’ (column ‘Outcome measure’ Table 9.3.b ). These pre-specified outcome domains are the chosen level for the synthesis as specified in the PICO for each synthesis. Authors will need to assess whether the measurement methods or tools used within each study provide an appropriate assessment of the domains ( Chapter 3, Section 3.2.4 ). A next step is to group each measure into the pre-specified time points. In this example, outcomes are grouped into short-term (<6 weeks) and long-term follow-up (≥6 weeks to 12 months) (column ‘Time points (time frame)’ Table 9.3.b ).

Variations on the format shown in Table 9.3.b can be presented within a review to summarize the characteristics of studies contributing to each synthesis, which is important for interpreting findings (step 2.5).

Table 9.3.b Table of study characteristics illustrating similarity of PICO elements across studies

BEH = health-directed behaviour; CON = constructive attitudes and approaches; EMO = emotional well-being; ENG = positive and active engagement in life; MON = self-monitoring and insight; NAV = health service navigation; SKL = skill and technique acquisition. ANCOVA = Analysis of covariance; CI = confidence interval; IQR = interquartile range; MD = mean difference; SD = standard deviation; SE = standard error, NS = non-significant. Pain and function measures: Dutch AIMS-SF = Dutch short form of the Arthritis Impact Measurement Scales; HAQ = Health Assessment Questionnaire; VAS = visual analogue scale; WOMAC = Western Ontario and McMaster Universities Osteoarthritis Index. 1 Ordered by type of comparator; 2 Short-term (denoted ‘immediate’ in the review Kroon et al (2014)) follow-up is defined as <6 weeks, long-term follow-up (denoted ‘intermediate’ in the review) is ≥6 weeks to 12 months; 3 For simplicity, in this example the available data are assumed to be the same for all outcomes within an outcome domain within a study. In practice, this is unlikely and the available data would likely vary by outcome; 4 Indicates that an effect estimate and its standard error may be computed through imputation of missing statistics, methods to convert between statistics (e.g. medians to means) or contact with study authors. *Indicates the selected outcome when there was multiplicity in the outcome domain and time frame.

9.3.3 Determine what data are available for synthesis (step 2.3)

Once the studies that are similar enough to be grouped together within each comparison have been determined, a next step is to examine what data are available for synthesis. Tabulating the measurement tools and time frames as shown in Table 9.3.b allows assessment of the potential for multiplicity (i.e. when multiple outcomes within a study and outcome domain are available for inclusion ( Chapter 3, Section 3.2.4.3 )). In this example, multiplicity arises in two ways. First, from multiple measurement instruments used to measure the same outcome domain within the same time frame (e.g. ‘Short-term Pain’ is measured using the ‘Pain VAS’ and ‘Pain on walking VAS’ scales in study 3). Second, from multiple time points measured within the same time frame (e.g. ‘Short-term Pain’ is measured using ‘Pain VAS’ at both 2 weeks and 1 month in study 6). Pre-specified methods to deal with the multiplicity can then be implemented (see Table 9.3.c for examples of approaches for dealing with multiplicity). In this review, the authors pre-specified a set of decision rules for selecting specific outcomes within the outcome domains. For example, for the outcome domain ‘Pain’, the selected outcome was the highest on the following list: global pain, pain on walking, WOMAC pain subscore, composite pain scores other than WOMAC, pain on activities other than walking, rest pain or pain during the night. The authors further specified that if there were multiple time points at which the outcome was measured within a time frame, they would select the longest time point. The selected outcomes from applying these rules to studies 3 and 6 are indicated by an asterisk in Table 9.3.b .

Table 9.3.b also illustrates an approach to tabulating the extracted data. The available statistics are tabulated in the column labelled ‘Data’, from which an assessment can be made as to whether the study contributes the required data for a meta-analysis (column ‘Effect & SE’) ( Chapter 10 ). For example, of the seven studies comparing health-directed behaviour (BEH) with usual care, six measured ‘Short-term Pain’, four of which contribute required data for meta-analysis. Reordering the table by comparison, outcome and time frame, will more readily show the number of studies that will contribute to a particular meta-analysis, and help determine what other synthesis methods might be used if the data available for meta-analysis are limited.

Table 9.3.c Examples of approaches for selecting one outcome (effect estimate) for inclusion in a synthesis.* Adapted from López-López et al (2018)

9.3.4 Determine if modification to the planned comparisons or outcomes is necessary, or new comparisons are needed (step 2.4)

The previous steps may reveal the need to modify the planned comparisons. Important variations in the intervention may be identified leading to different or modified intervention groups. Few studies or sparse data, or both, may lead to different groupings of interventions, populations or outcomes. Planning contingencies for anticipated scenarios is likely to lead to less post-hoc decision making ( Chapter 2 and Chapter 3 ); however, it is difficult to plan for all scenarios. In the latter circumstance, the rationale for any post-hoc changes should be reported. This approach was adopted in a review examining the effects of portion, package or tableware size for changing selection and consumption of food, alcohol and tobacco (Hollands et al 2015). After preliminary examination of the outcome data, the review authors changed their planned intervention groups. They judged that intervention groups based on ‘size’ and those based on ‘shape’ of the products were not conceptually comparable, and therefore should form separate comparisons. The authors provided a rationale for the change and noted that it was a post-hoc decision.

9.3.5 Synthesize the characteristics of the studies contributing to each comparison (step 2.5)

A final step, and one that is essential for interpreting combined effects, is to synthesize the characteristics of studies contributing to each comparison. This description should integrate information about key PICO characteristics across studies, and identify any potentially important differences in characteristics that were pre-specified as possible effect modifiers. The synthesis of study characteristics is also needed for GRADE assessments, informing judgements about whether the evidence applies directly to the review question (indirectness) and analyses conducted to examine possible explanations for heterogeneity (inconsistency) (see Chapter 14 ).

Tabulating study characteristics is generally preferable to lengthy description in the text, since the structure imposed by a table can make it easier and faster for readers to scan and identify patterns in the information presented. Table 9.3.b illustrates one such approach. Tabulating characteristics of studies that contribute to each comparison can also help to improve the transparency of decisions made around grouping of studies, while also ensuring that studies that do not contribute to the combined effect are accounted for.

9.4 Checking data before synthesis

Before embarking on a synthesis, it is important to be confident that the findings from the individual studies have been collated correctly. Therefore, review authors must compare the magnitude and direction of effects reported by studies with how they are to be presented in the review. This is a reasonably straightforward way for authors to check a number of potential problems, including typographical errors in studies’ reports, accuracy of data collection and manipulation, and data entry into RevMan. For example, the direction of a standardized mean difference may accidentally be wrong in the review. A basic check is to ensure the same qualitative findings (e.g. direction of effect and statistical significance) between the data as presented in the review and the data as available from the original study.

Results in forest plots should agree with data in the original report (point estimate and confidence interval) if the same effect measure and statistical model is used. There are legitimate reasons for differences, however, including: using a different measure of intervention effect; making different choices between change-from-baseline measures, post-intervention measures alone or post-intervention measures adjusted for baseline values; grouping similar intervention groups; or making adjustments for unit-of-analysis errors in the reports of the primary studies.

9.5 Types of synthesis

The focus of this chapter has been describing the steps involved in implementing the planned comparisons between intervention groups (stage 2 of the general framework for synthesis ( Box 9.2.a )). The next step (stage 3) is often performing a statistical synthesis. Meta-analysis of effect estimates, and its extensions have many advantages. There are circumstances under which a meta-analysis is not possible, however, and other statistical synthesis methods might be considered, so as to make best use of the available data. Available summary and synthesis methods, along with the questions they address and examples of associated plots, are described in Table 9.5.a . Chapter 10 and Chapter 11 discuss meta-analysis (of effect estimate) methods, while Chapter 12 focuses on the other statistical synthesis methods, along with approaches to tabulating, visually displaying and providing a structured presentation of the findings. An important part of planning the analysis strategy is building in contingencies to use alternative methods when the desired method cannot be used.

Table 9.5.a Overview of available methods for summary and synthesis

9.6 Chapter information

Authors: Joanne E McKenzie, Sue E Brennan, Rebecca E Ryan, Hilary J Thomson, Renea V Johnston

Acknowledgements: Sections of this chapter build on Chapter 9 of version 5.1 of the Handbook , with editors Jonathan Deeks, Julian Higgins and Douglas Altman. We are grateful to Julian Higgins, James Thomas and Tianjing Li for commenting helpfully on earlier drafts.

Funding: JM is supported by an NHMRC Career Development Fellowship (1143429). SB and RR’s positions are supported by the NHMRC Cochrane Collaboration Funding Program. HT is funded by the UK Medical Research Council (MC_UU_12017-13 and MC_UU_12017-15) and Scottish Government Chief Scientist Office (SPHSU13 and SPHSU15). RJ’s position is supported by the NHMRC Cochrane Collaboration Funding Program and Cabrini Institute.

9.7 References

Chamberlain C, O’Mara-Eves A, Porter J, Coleman T, Perlen SM, Thomas J, McKenzie JE. Psychosocial interventions for supporting women to stop smoking in pregnancy. Cochrane Database of Systematic Reviews 2017; 2 : CD001055.

Hollands GJ, Shemilt I, Marteau TM, Jebb SA, Lewis HB, Wei Y, Higgins JPT, Ogilvie D. Portion, package or tableware size for changing selection and consumption of food, alcohol and tobacco. Cochrane Database of Systematic Reviews 2015; 9 : CD011045.

Kroon FPB, van der Burg LRA, Buchbinder R, Osborne RH, Johnston RV, Pitt V. Self-management education programmes for osteoarthritis. Cochrane Database of Systematic Reviews 2014; 1 : CD008963.

López-López JA, Page MJ, Lipsey MW, Higgins JPT. Dealing with effect size multiplicity in systematic reviews and meta-analyses. Research Synthesis Methods 2018; 9 : 336–351.

For permission to re-use material from the Handbook (either academic or commercial), please see here for full details.

How to Synthesize Written Information from Multiple Sources

Shona McCombes

Content Manager

B.A., English Literature, University of Glasgow

Shona McCombes is the content manager at Scribbr, Netherlands.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

On This Page:

When you write a literature review or essay, you have to go beyond just summarizing the articles you’ve read – you need to synthesize the literature to show how it all fits together (and how your own research fits in).

Synthesizing simply means combining. Instead of summarizing the main points of each source in turn, you put together the ideas and findings of multiple sources in order to make an overall point.

At the most basic level, this involves looking for similarities and differences between your sources. Your synthesis should show the reader where the sources overlap and where they diverge.

Unsynthesized Example

Franz (2008) studied undergraduate online students. He looked at 17 females and 18 males and found that none of them liked APA. According to Franz, the evidence suggested that all students are reluctant to learn citations style. Perez (2010) also studies undergraduate students. She looked at 42 females and 50 males and found that males were significantly more inclined to use citation software ( p < .05). Findings suggest that females might graduate sooner. Goldstein (2012) looked at British undergraduates. Among a sample of 50, all females, all confident in their abilities to cite and were eager to write their dissertations.

Synthesized Example

Studies of undergraduate students reveal conflicting conclusions regarding relationships between advanced scholarly study and citation efficacy. Although Franz (2008) found that no participants enjoyed learning citation style, Goldstein (2012) determined in a larger study that all participants watched felt comfortable citing sources, suggesting that variables among participant and control group populations must be examined more closely. Although Perez (2010) expanded on Franz’s original study with a larger, more diverse sample…

Step 1: Organize your sources

After collecting the relevant literature, you’ve got a lot of information to work through, and no clear idea of how it all fits together.

Before you can start writing, you need to organize your notes in a way that allows you to see the relationships between sources.

One way to begin synthesizing the literature is to put your notes into a table. Depending on your topic and the type of literature you’re dealing with, there are a couple of different ways you can organize this.

Summary table

A summary table collates the key points of each source under consistent headings. This is a good approach if your sources tend to have a similar structure – for instance, if they’re all empirical papers.

Each row in the table lists one source, and each column identifies a specific part of the source. You can decide which headings to include based on what’s most relevant to the literature you’re dealing with.

For example, you might include columns for things like aims, methods, variables, population, sample size, and conclusion.

For each study, you briefly summarize each of these aspects. You can also include columns for your own evaluation and analysis.

The summary table gives you a quick overview of the key points of each source. This allows you to group sources by relevant similarities, as well as noticing important differences or contradictions in their findings.

Synthesis matrix

A synthesis matrix is useful when your sources are more varied in their purpose and structure – for example, when you’re dealing with books and essays making various different arguments about a topic.

Each column in the table lists one source. Each row is labeled with a specific concept, topic or theme that recurs across all or most of the sources.

Then, for each source, you summarize the main points or arguments related to the theme.

The purposes of the table is to identify the common points that connect the sources, as well as identifying points where they diverge or disagree.

Step 2: Outline your structure

Now you should have a clear overview of the main connections and differences between the sources you’ve read. Next, you need to decide how you’ll group them together and the order in which you’ll discuss them.

For shorter papers, your outline can just identify the focus of each paragraph; for longer papers, you might want to divide it into sections with headings.

There are a few different approaches you can take to help you structure your synthesis.

If your sources cover a broad time period, and you found patterns in how researchers approached the topic over time, you can organize your discussion chronologically .

That doesn’t mean you just summarize each paper in chronological order; instead, you should group articles into time periods and identify what they have in common, as well as signalling important turning points or developments in the literature.

If the literature covers various different topics, you can organize it thematically .

That means that each paragraph or section focuses on a specific theme and explains how that theme is approached in the literature.

Source Used with Permission: The Chicago School

If you’re drawing on literature from various different fields or they use a wide variety of research methods, you can organize your sources methodologically .

That means grouping together studies based on the type of research they did and discussing the findings that emerged from each method.

If your topic involves a debate between different schools of thought, you can organize it theoretically .

That means comparing the different theories that have been developed and grouping together papers based on the position or perspective they take on the topic, as well as evaluating which arguments are most convincing.

Step 3: Write paragraphs with topic sentences

What sets a synthesis apart from a summary is that it combines various sources. The easiest way to think about this is that each paragraph should discuss a few different sources, and you should be able to condense the overall point of the paragraph into one sentence.

This is called a topic sentence , and it usually appears at the start of the paragraph. The topic sentence signals what the whole paragraph is about; every sentence in the paragraph should be clearly related to it.

A topic sentence can be a simple summary of the paragraph’s content:

“Early research on [x] focused heavily on [y].”

For an effective synthesis, you can use topic sentences to link back to the previous paragraph, highlighting a point of debate or critique:

“Several scholars have pointed out the flaws in this approach.” “While recent research has attempted to address the problem, many of these studies have methodological flaws that limit their validity.”

By using topic sentences, you can ensure that your paragraphs are coherent and clearly show the connections between the articles you are discussing.

As you write your paragraphs, avoid quoting directly from sources: use your own words to explain the commonalities and differences that you found in the literature.

Don’t try to cover every single point from every single source – the key to synthesizing is to extract the most important and relevant information and combine it to give your reader an overall picture of the state of knowledge on your topic.

Step 4: Revise, edit and proofread

Like any other piece of academic writing, synthesizing literature doesn’t happen all in one go – it involves redrafting, revising, editing and proofreading your work.

Checklist for Synthesis

•   Do I introduce the paragraph with a clear, focused topic sentence?
•   Do I discuss more than one source in the paragraph?
•   Do I mention only the most relevant findings, rather than describing every part of the studies?
•   Do I discuss the similarities or differences between the sources, rather than summarizing each source in turn?
•   Do I put the findings or arguments of the sources in my own words?
•   Is the paragraph organized around a single idea?
•   Is the paragraph directly relevant to my research question or topic?
•   Is there a logical transition from this paragraph to the next one?

Further Information

How to Synthesise: a Step-by-Step Approach

Help…I”ve Been Asked to Synthesize!

Learn how to Synthesise (combine information from sources)

How to write a Psychology Essay

Related Articles

Student Resources

How To Cite A YouTube Video In APA Style – With Examples

How to Write an Abstract APA Format

APA References Page Formatting and Example

APA Title Page (Cover Page) Format, Example, & Templates

How do I Cite a Source with Multiple Authors in APA Style?

How to Write a Psychology Essay

Navigating Multi-Agency NEPA Processes to Advance Multimodal Transportation Projects (2016)

Chapter: chapter 5 - case study synthesis.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

29 C H A P T E R 5 The research found many different ways to carry out NEPA for situations involving more than one mode or U.S. DOT agency. The case studies offer examples of agencies working out hybrid processes and trying new approaches that worked for their particular situation and parties involved. In other cases, participants took away some lessons learned that will benefit them the next time they face a similar situation. In its own way, each case study demonstrates that flexibility and openness to new approaches were necessary to achieve satis- factory outcomes. In the previous chapter, transferrable strategies were listed under each of the five challenges. Many of these were listed multiple times, because there were instances where a particu- lar strategy was used to address more than one challenge. This chapter begins with a consolidated list of transferrable strate- gies and tactics. Crosscutting themes and stumbling blocks to avoid for successful outcomes are then presented. Consolidated List of Transferrable Strategies and Tactics Twenty-three strategies for addressing the challenges to multimodal NEPA projects emerged from the case studies. Table 10 lists these strategies, identifies the challenges they addressed, and references the case studies that used each strategy. More detailed descriptions of each application can be found in the individual case study summaries, presented as Appendices A through L. Many of these strategies relate to the challenge of coordinat- ing between and among U.S. DOT and local agencies. Success- ful techniques varied widely and included committees, joint project offices, memoranda of agreement, and frequent in- person meetings. Coordination also occurred through the use of local task forces, groups, or technical documents to address and record solutions to technical issues. Crosscutting Themes and Keys to Success As expected, the most prevalent theme across the case studies was the need for early and continuous coordination across all agencies—federal, state, and local—with a potential stake in the project. Coordination emerged in every case study as either a factor for success or, when effective coordination was lacking, as a source of frustration and delay. This theme is appli- cable to all five challenges. Coordination is critical to the success of any project involving more than one partner. Variations on this theme include knowing the best point at which to engage specific agencies and the most effective strategy to employ in a given situation. Remaining flexible and seeking opportunities to find common ground also emerged as important themes, and are generally tied to the need for cooperation. Case Study Synthesis Case Study Highlight: T-REX and “One DOT” FTA and FHWA operated as “One DOT,” and established an Interagency Agreement early in NEPA, streamlining the NEPA process for T-REX. The two agencies identified areas where their NEPA requirements differed and identified an approach to reconciling these issues in an Inter- agency Agreement. Staff from both agencies understood their roles and responsibilities throughout NEPA, saving time and resources. The One DOT approach—a U.S. DOT initiative at the time—was intended to foster collabora- tion across modal administrations. It was applied through construction and garnered the FHWA Colorado Division and FTA Region VIII a special award in recognition of the efforts.

30 Transferrable Strategies and Tactics Challenges Case Studies that Demonstrate Strategy/Tactic 1. U ni qu e Ag en cy - Sp ec ifi c Pr og ra m R eq ui re m en ts 2. D iff er in g In te rp re ta tio ns o f N EP A R eq ui re m en ts 3. A nt ic ip at in g W hi ch A ge nc ie s Ha ve M ajo r Fe de ra l A ct io ns 4. E ffi ci en t Co or di na tio n am on g A ge nc ie s 1. Utilize various channels to maintain regular communication with all appropriate partners. All 2. Establish process where federal agencies coordinate directly. I-70 East National Gateway DART DFW Extension 3. Foster a sense of teamwork and collaboration and develop relationships among all parties, particularly with and between federal agency staff. Dulles Project T-REX I-70 East Columbia River Crossing DART DFW Extension 4. Establish interagency agreement(s) to detail specific agency requirements, procedures to be followed, and agency roles. T-REX Mountain View Columbia River Crossing Port of Miami Tunnel 5. Build coordination among agencies pre-NEPA and capitalize on preexisting relationships. I-70 East Columbia River Crossing East Link 6. Hire a mediator and/or facilitator to help work through challenging issues and facilitate agreement. XpressWest Eastern Corridor 7. Leverage work that took place before NEPA began. T-REX 8. For phased projects, engage all lead and cooperating federal agencies in every phase, even if their interests aren’t directly affected across all phases. Dulles Project Port of Miami Tunnel DART DFW Extension 9. Designate a single lead agency or designate which agency's requirements and/or processes will be followed early; conduct evaluations for other agency requirements separately. Dulles Project National Gateway DART DFW Extension 10. Aim to prepare a single NEPA document and address unique agency requirements in standalone sections. DART DFW Extension 11. Develop a detailed critical path method schedule that is updated and referenced on a regular basis. T-REX 12. Hire or identify a staff expert to provide necessary expertise in specific technical areas and/or expedite unfamiliar processes. Columbia River Crossing DART DFW Extension Table 10. Transferrable strategies and tactics applied in case studies.

31 13. Co-locate or assign staff from partner agencies, or defer to agency specialists, to provide specific technical expertise and/or expedite unfamiliar processes. National Gateway DART DFW Extension Dulles Project T-REX Columbia River Crossing I-70 East 14. Establish a committee, task force, or working group to provide technical guidance and support. T-REX I-70 East 15. Create a formal governance structure (with management board) to coordinate decision-making among private and public partners. CREATE 16. Establish a detailed charter or coordination plan that includes ground rules and procedures for conflict resolution and facilitation. East Link 17. Develop technical memoranda or white papers to explain and reconcile differences in agency requirements. T-REX I-70 East East Link 18. Apply documented policies, procedures, and successful methodologies for resolving issues that were implemented on other multimodal NEPA projects. I-70 East East Link Transferrable Strategies and Tactics Challenges Case Studies that Demonstrate Strategy/Tactic 1. U ni qu e Ag en cy - Sp ec ifi c Pr og ra m R eq ui re m en ts 2. D iff er in g In te rp re ta tio ns o f N EP A R eq ui re m en ts 3. A nt ic ip at in g W hi ch A ge nc ie s Ha ve M ajo r Fe de ra l A ct io ns 4. E ffi ci en t Co or di na tio n am on g A ge nc ie s 19. Develop a formal dispute resolution process that engages higher-level staff to resolve procedural differences. T-REX Columbia River Crossing 20. Closely align state and federal environmental processes. Port of Miami Tunnel 21. Incorporate a thorough scoping process. XpressWest 22. Visit the regional or field office of an agency to encourage greater participation. Mountain View Eastern Corridor 23. Create an action so that the most appropriate U.S. DOT agency can lead the NEPA process. DART DFW Extension Table 10. (Continued). Closely related is engaging all necessary staff as needed throughout the process. Several of the strategies were related to this theme, including engaging staff with expertise in spe- cific NEPA or agency-specific requirements, as was done on National Gateway and the DART DFW Extension; using executive-level staff to resolve differences, as was done on Columbia River Crossing and T-REX; and seeking external experts as needed to provide guidance or facilitate resolu- tions, as was done on XpressWest. Similarly, ensuring that all agencies (both local and fed- eral) have the same level of interest in and commitment to the project is critical to moving the NEPA process forward and minimizing delays on a combined process. In the East- ern Corridor and Mountain View case studies, one U.S. DOT

32 agency did not actively participate in the multimodal NEPA process due to a lack of local commitment to a multimodal outcome that would lead to a federal action on their part. On I-70 East, a merged process was eventually split because of differing levels of funding commitment to the highway and transit elements. Another overarching theme was strategic use of the most advantageous agency relationships and high-level interests. This was perhaps most evident on the National Gateway proj- ect, which used a high-level kick-off meeting in Washington, D.C. to capture the attention of the participating states and engage their governors. The project team also emphasized the economic and community benefits of the National Gateway to win support. Leveraging shared interests and executive-level strategic relationships can help engage agency partners that may not otherwise be interested in coordinating. Allocating adequate time and resources to the NEPA pro- cess was cited frequently as crucial to addressing the identified challenges to multimodal projects. Some of the strategies may require extra time to execute. Factoring in the time required to meet the approval requirements of multiple agencies— even after coordination strategies are implemented—is good practice. Becoming familiar with agency and private partner processes and reconciling differences early in the NEPA process was also a significant crosscutting theme. This was particularly important for addressing the first two challenges, but lack of understanding may also preclude identification of project elements that may trigger federal action—a problem under the third challenge of anticipating major federal action. Gaining an early understanding of agency constraints and expectations and recognizing they may differ also recurred across such case studies as Mountain View, Eastern Corridor, CREATE, and XpressWest. Finally, the case studies demonstrate that there is no single best way to approach the NEPA process for multimodal situ- ations. The T-REX and I-70 East projects illustrate this point particularly well. In the T-REX project, a single EIS was pre- pared for both the highway and the transit improvements, and both modes then moved to a single design-build procurement for implementation in a shared corridor. The subsequent I-70 East process was modeled after T-REX, but due to the differ- ing levels of funding commitment in that case, the highway and transit elements were ultimately split into separate NEPA processes (with associated delays and costs). Ultimately, the success of multimodal NEPA activities may depend more on the willingness and motivation of the agencies to work together, to find common ground, and to work around differing processes, and less upon the team’s organizational structure or approach. Achieving an effective interagency approach depends on how well the project spon- sor and other agencies are able to bridge the differences in requirements and agency practices. Stumbling Blocks to Avoid The case studies also highlighted stumbling blocks to avoid when undertaking a multi-agency NEPA process. In particular: • Differences in perspective and emphasis between U.S. DOT agencies—whether from differing program structures, legislative mandates, past legal challenges, or leadership priorities—can complicate the development of a shared process. Agreement on a single process and scope that accommodates each agency’s needs may require coopera- tion and compromise. • Different levels of commitment to a project can nullify or limit the effectiveness of prior agreements. • Insufficient time or resources allocated to applying these strategies can hinder successful implementation. • Limited commitment on the part of project sponsors to coordinate or learn other agency requirements, and failure to communicate the benefits of full agency engagement, can be barriers to implementing the recommended strategies. • Resistance on the part of state and local sponsors to adapt their current processes to reflect the nuances of U.S. DOT agency NEPA processes can lead to frustration and delays. • Reluctance of federal agencies to engage in projects before their major action is identified can thwart coordination and overall progress. The strategies and tactics summarized in Table 10, along with the Self-Assessment Tool in Appendix O, can be used to help anticipate and avoid the stumbling blocks listed in this section. Appendix M identifies key NEPA requirements for each of the U.S. DOT agencies featured in the case studies. This appendix can help future project sponsors identify the rel- evant requirements before starting a multi-agency, multi- modal NEPA process.

TRB’s National Cooperative Highway Research Program (NCHRP) Report 827: Navigating Multi-Agency NEPA Processes to Advance Multimodal Transportation Projects analyzes approaches taken by state departments of transportation (DOTs), their local partners, and other project sponsors to satisfy National Environmental Policy Act (NEPA) requirements for transportation projects involving more than one mode. Case studies illustrate successful practices and provide examples of institutional arrangements used to comply with NEPA requirements for two or more U.S. DOT agencies.

READ FREE ONLINE

Welcome to OpenBook!

You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

Do you want to take a quick tour of the OpenBook's features?

Show this book's table of contents , where you can jump to any chapter by name.

...or use these buttons to go back to the previous chapter or skip to the next one.

Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

To search the entire text of this book, type in your search term here and press Enter .

Share a link to this book page on your preferred social network or via email.

View our suggested citation for this chapter.

Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

Get Email Updates

Do you enjoy reading reports from the Academies online for free ? Sign up for email notifications and we'll let you know about new publications in your areas of interest when they're released.

Biometrics & security

Id verification.

Facial identification and verification for consumer and security applications.

Activity recognition and threat detection across camera views.

Consumer devices & applications

Spatial computing, gesture recognition, and gaze estimation for headsets.

Virtual try-on

Millions of identities and clothing options to train best-in-class models.

Driver monitoring

Simulate driver and occupant behavior captured with multi-modal cameras.

Pedestrian detection

Simulate edge cases and rare events to ensure the robust performance of autonomous vehicles.

Lost in Context: How Much Can You Fit into a Transformer

The announcement of Gemini 1.5 ...

Visit the Blog

Gen AI book

Data visualizer

API documentation

Synthetic data

Synthetic data book

Synthetic data guide

Synthetic data whitepaper

Industry survey

OpenSynthetics

Join Our Team

Together, we’re building the future of computer vision & machine learning

Explore Careers

In this (very) long post, we present an entire whitepaper on synthetic data, proving that synthetic data works even without complicated domain adaptation techniques in a wide variety of practical applications. We consider three specific problems, all related to human faces, show that synthetic data works for all three, and draw some other interesting and important conclusions.

Introduction

Synthetic data is an invaluable tool for many machine learning problems, especially in computer vision, which we will concentrate on below. In particular, many important computer vision problems, including segmentation, depth estimation, optical flow estimation, facial landmark detection, background matting, and many more, are prohibitively expensive to label manually.

Synthetic data provides a way to have unlimited perfectly labeled data at a fraction of the cost of manually labeled data. With the 3D models of objects and environments in question, you can create an endless stream of data with any kind of labeling under different (randomized) conditions such as composition and placement of objects, background, lighting, camera placement and parameters, and so on. For a more detailed overview of synthetic data, see ( Nikolenko, 2019 ).

Naturally, artificially produced synthetic data cannot be perfectly photorealistic. There always exists a domain gap between real and synthetic datasets, stemming both from this lack of photorealism and also in part from different approaches to labeling: for example, manually produced segmentation masks are generally correct but usually rough and far from pixel-perfect.

Therefore, most works on synthetic data center around the problem of domain adaptation : how can we close this gap? There exist approaches that improve the realism, called synthetic-to-real refinement , and approaches that impose constraints on the models—their feature space, training process, or both—in order to make them operate similarly on both real and synthetic data. This is the main direction of research in synthetic data right now, and much of recent research is devoted to suggesting new approaches to domain adaptation.

However, CGI-based synthetic data becomes better and better with time, and some works also suggest that domain randomization , i.e., simply making the synthetic data distribution sufficiently varied to ensure model robustness, may work out of the box. On the other hand, recent advances in related problems such as style transfer (synthetic-to-real refinement is basically style transfer between the domains of synthetic and real images) suggest that refinement-style domain adaptation may be done with very simple techniques; it might happen that while these techniques are insufficient for photorealistic style transfer for high-resolution photographs they are quite enough to make synthetic data useful for computer vision models.

Still, it turns out that synthetic data can provide significant improvements even without complicated domain adaptation approaches. In this whitepaper, we consider three specific use cases where we have found synthetic data to work well under either very simple or no domain adaptation at all. We are also actively pursuing research on domain adaptation, and ideas coming from modern style transfer approaches may prove to bring new interesting results here as well; but in this document, we concentrate on very straightforward applications of synthetic data and show that synthetic data can just work out of the box. In one of the case studies, we compare two main techniques to using synthetic data in this simple way—training on hybrid datasets and fine-tuning on real data after pretraining on synthetic—also with interesting results.

Here at Synthesis AI, we have developed the Face API for mass generation of high-quality synthetic 3D models of human heads, so all three cases have to do with human faces: face segmentation, background matting for human faces, and facial landmark detection. Note that all three use cases also feature some very complex labeling: while facial landmarks are merely very expensive to label by hand, manually labeled datasets for background matting are virtually impossible to obtain.

Before we proceed to the use cases, let us describe what they all have in common.

Data Generation and Domain Adaptation

In this section, we describe the data generation process and the synthetic-to-real domain adaptation approach that we used throughout all three use cases.

Face API by Synthesis AI

The Face API, developed by Synthesis AI, can generate millions of images comprising unique people, with expressions and accessories, in a wide array of environments, with unique camera settings. Below we show some representative examples of various Face API capabilities.

The Face API has tens of thousands of unique identities that span the genders, age groups, and ethnicity/skin tones, and new identities are added continuously. These 

It also allows for modifications to the face, including expressions and emotions, eye gaze, head turn, head & facial hair, and more:

Furthermore, the Face API allows to adorn the subjects with accessories, including clear glasses, sunglasses, hats, other headwear, headphones, and face masks.  

Finally, it allows for indoor & outdoor environments with accurate lighting, as well as additional directional/spot lighting to further vary the conditions and emulate reality.

The output includes:

• Pupil Coordinates
• Facial Landmarks (iBug 68-like)
• Camera Settings
• Segmentation Images & Values
• Depth from Camera
• Surface Normals
• Alpha / Transparency

Full documentation can be found at the Synthesis AI website . For the purposes of this whitepaper, let us just say that Face API is a more than sufficient source of synthetic human faces with any kind of labeling that a computer vision practitioner might desire.

Synthetic-to-Real Refinement by Instance Normalization

Below, we consider three computer vision problems where we have experimented with using synthetic data to train more or less standard deep learning models. Although we could have applied  complex domain adaptation techniques, we instead chose to use one simple idea inspired by style transfer models. Here we show some practical and less computationally intensive approaches that can work well too.  

Most recent works on style transfer, including MUNIT ( Huang et al., 2018 ), StyleGAN ( Karras et al., 2018 ), StyleGAN2 ( Karras et al., 2019 ), and others, make use of the idea of adaptive instance normalization (AdaIN) proposed by Huang and Belongie (2017) . 

The basic idea of AdaIN is to substitute the statistics of the style image in place of the batch normalization (BN) parameters for the corresponding BN layers during the processing of the content image:

This is a natural extension of an earlier idea of conditional instance normalization ( Dimoulin et al., 2016 ) where BN parameters were learned separately for each style. Both conditional and adaptive instance normalization can be useful for style transfer, but AdaIN is better suited for common style transfer tasks because it only needs a single style image to compute the statistics and does not require pretraining or, generally speaking, any information regarding future styles in advance.

In style transfer architectures such as MUNIT or StyleGAN, AdaIN layers are used as a key component for a complex involved architecture that usually also employs CycleGAN ( Zhu et al., 2017 ) and/or ProGAN ( Karras et al., 2017 ) ideas. As a result, these architectures are hard to train and, what is even more important, require a lot of computational resources to use. This makes state of the art style transfer architectures unsuitable for synthetic-to-real refinement since we need to apply them to every image in the training set.

However, style transfer results already in the original work on AdaIN ( Huang and Belongie, 2017 ) already look quite good, and it is possible to use AdaIN in a much simpler architecture than state of the art style transfer. Therefore, in our experiments we use a similar approach for synthetic-to-real refinement, replacing BN statistics for synthetic images with statistics extracted from real images. 

This approach has been shown to work several times in literature, in several variations ( Li et al., 2016 ; Chang et al., 2019 ) We follow either Chang et al. (2019) , which is a simpler and more direct version, or the approach introduced by Seo et al. (2020) , called domain-specific optimized normalization (DSON), where for each domain we maintain batch normalization statistics and mixture weights learned on the corresponding domain:

Thus, we have described our general approach to synthetic-to-real domain adaptation; we used it in some of our experiments but note that in many cases, we did not do any domain adaptation at all (these cases will be made clear below). With that, we are ready to proceed to specific computer problems.

Face Segmentation with Synthetic Data: Syn-to-Real Transfer As Good As Real-to-Real Transfer

Our first use case deals with the segmentation problem. Since we are talking about applications of our Face API, this will be the segmentation of human faces. What’s more, we do not simply cut out the mask of a human face from a photo but want to segment different parts of the face.

We have used two real datasets in this study:

• LaPa (Landmark guided face Parsing dataset), presented by Liu et al. (2020) , contains more than 22,000 images, with variations in pose and facial expression and also with some occlusions among the images; it contains facial landmark labels (which we do not use) and faces segmented into 11 classes, as in the following example:

• CelebAMask-HQ , presented by Lee et al. (2019) , contains 30,000 high-resolution celebrity face images with 19 segmentation classes, including various hairstyles and a number of accessories such as glasses or hats:

For the purposes of this study, we reduced both datasets to 9 classes (same as LaPa but without the eyebrows). As synthetic data, we used 200K diverse images produced by our Face API; no domain adaptation was applied in this case study (we have tried it and found no improvement or even some small deterioration in performance metrics).

As the basic segmentation model, we have chosen the DeepLabv3+ model ( Chen et al., 2018 ) with the DRN-56 backbone, an encoder-decoder model with spatial pyramid pooling and atrous convolutions. DeepLabv3+ produces good results, often serves as a baseline in works on semantic segmentation, and, importantly, is relatively lightweight and easy to train. In particular, due to this choice all images were resized down to 256p.

The results, summarized in the table below, confirmed our initial hypothesis and even outperformed our expectations in some respects. The table shows the mIoU (mean intersection-over-union) scores on the CelebAMask-HQ and LaPa test sets for DeepLabv3+ trained on real data only from CelebAMask-HQ, mixed with synthetic data in various proportions.

In the table below, we show the results for different proportions of real data (CelebAMask-HQ) in the training set, tested on two different test sets.

First of all, as expected, we found that training on a hybrid dataset with both real and synthetic data is undoubtedly beneficial. When both training and testing on CelebAMask-HQ (first two rows in the table), we obtain noticeable improvements across all proportions of real and synthetic data in the training set. The same holds for the two bottom rows in the table that show the results of DeepLabv3+ trained on CelebAMask-HQ and tested on LaPa.

But the most interesting and, in our opinion, important result is that in this context, domain transfer across two (quite similar) real datasets produces virtually the same results as domain transfer from synthetic to real data: results on LaPa with 100% only real data are almost identical to the results on LaPa with 0% real and only synthetic data. Let us look at the plots below and then discuss what conclusions we can draw:

Most importantly, note that the domain gap on the CelebA test set amounts to a 9.6% performance drop for the Syn-to-CelebA domain shift and 9.2% for the LaPa-to-CelebA domain shift. This very small difference suggests that while domain shift is a problem, it is not a problem specifically for the synthetic domain, which performs basically the same as a different domain of real data. The results on the LaPa test set tell a similar story: 14.4% performance drop for CelebA-to-LaPa and 16.8% for Syn-to-LaPa.

Second, note the “fine-tune” bars that exhibit (quite significant) improvements over other models trained on a different domain. This is another effect we have noted in our experiments: it appears that fine-tuning on real data after pretraining on a synthetic dataset often works better than just training on a mixed hybrid syn-plus-real dataset.

Below, we show a more detailed look into where the errors are:  

During cross-domain testing, synthetic data is competitive with real data and even outperforms it on difficult classes such as eyes and lips. Synthetic data seems to perform worse on nose and hair, but that can explained by differences in labelling of these two classes across real and synthetic.

Thus, in this very straightforward use case we have seen that even in a very direct application, with very efficient syn-to-real refinement, synthetic data generated by our Face API works basically at the same level as training on a different real dataset.

This is already very promising, but let us proceed to even more interesting use cases!

Background Matting: Synthetic Data for Very Complex Labeling

The primary use case for background matting, useful to keep in mind throughout this section, is cutting out a person from a “green screen” image/video or, even more interesting, from any background. This is, of course, a key computer vision problem in the current era of online videoconferencing.

Formally, background matting is a task very similar to face/person segmentation, but with two important differences. First, we are looking to predict not only the binary segmentation mask but also the alpha (opacity) value, so the result is a “soft” segmentation mask with values in the [0, 1] range. This is very important to improve blending into new backgrounds.

Second, the specific variation of background matting that we are experimenting with here takes two images as input: a pure background photo and a photo with the object (person). In other words, the matting problem here is to subtract the background from the foreground. Here is a sample image from the demo provided by Lin et al. (2020) , the work that we take as the basic model for this study:

The purpose of this work was to speed up high-quality background matting for high-resolution images so that it could work in real time; indeed, Lin et al. have also developed a working Zoom plugin that works well in real videoconferencing. 

We will not dwell on the model itself for too long. Basically, Lin et al. propose a pipeline that first produces a coarse output with atrous spatial pyramid pooling similar to DeepLabv3 ( Chen et al., 2017 ) and then recover high-resolution matting details with a refinement network (not to be confused with syn-to-real refinement!). Here is the pipeline as illustrated in the paper :

For our experiments, we have used the MobileNetV2 backbone ( Sandler et al., 2018 ). For training we used virtually all parameters as provided by Lin et al. (2020) except augmentation, which we have made more robust.

One obvious problem with background matting is that it is extremely difficult to obtain real training data. Lin et al. describe the PhotoMatte13K dataset of 13,665 2304×3456 images with manually corrected mattes that they acquired, but release only the test set (85 images). Therefore, for real training we used the AISegment.com Human Matting Dataset ( released on Kaggle ) for the foreground part, refining its mattes a little with open source matting software (see below in more detail about this). The AISegment.com dataset contains ~30,000 600×800 images—note the huge difference in resolution with PhotoMatte13K .

Note that this dataset does not contain the corresponding background images, so for background images we used our own high-quality HDRI panoramas. In general, our pipeline for producing the real training set was as follows:

• cut out the object from an AISegment.com image according to the ground truth matte;
• take a background image, apply several relighting/distortion augmentations, and paste the object onto the resulting background image.

This is a standard way to obtain training sets for this problem. The currently largest academic dataset for this problem, Deep Image Matting by Xu et al. (2017) , uses the same kind of procedure.

For the synthetic part, we used our Face API engine to generate a dataset of ~10K 1024×1024 images, using the same high-quality HDRI panoramas for the background. Naturally, the synthetic dataset has very accurate alpha channels, something that could hardly be achieved in manual labeling of real photographs. In the example below, note how hard it would be to label the hair for matting:

Before we proceed to the results, a couple of words about the quality metrics. We used slightly modified metrics from the original paper, also described in more detailed and motivated by Rhemann et al. (2009) :

• mse : mean squared error for the alpha channel and foreground computed along the object boundary;
• mae : mean absolute error for the alpha channel and foreground computed along the object boundary;
• grad : spatial-gradient metric that measures the difference between the gradients of the computed alpha matte and the ground truth computed along the object boundary;
• conn : connectivity metric that measures average degrees of connectivity for individual pixels in the computed alpha matte and the ground truth computed along the object boundary;
• IOU : standard intersection over union metric for the “person” class segmentation obtained from the alpha matte by thresholding.

We have trained four models:

• Real model trained only on the real dataset;
• Synthetic model trained only on the synthetic dataset;
• Mixed model trained on a hybrid syn+real dataset;
• DA model, also trained on the hybrid syn+real dataset but with batchnorm-based domain adaptation as shown above updating batchnorm statistics only on the real training set.

The plots below show the quality metrics on the test set PhotoMatte85 (85 test images) where we used our HDRI panoramas as background images:

And here are the same metrics on the PhotoMatte85 test set with 4K images downloaded from the Web as background images:

It is hard to give specific examples where the difference would be striking, but as you can see, adding even a small high-quality synthetic dataset (our synthetic set was ~3x smaller than the real dataset) brings tangible improvements in the quality. Moreover, for some metrics related to visual quality (conn and grad in particular) the model trained only on synthetic data shows better performance than the model trained on real data. The Mixed and DA models are better yet, and show improvements across all metrics, again demonstrating the power of mixed syn+real datasets.

Above, we have mentioned automatic refinement of the AISegment.com dataset with open-source matting software that we applied before training. To confirm that these refinements indeed make the dataset better, we have compared the performance on refined and original AISegment.com dataset. The results clearly show that our refinement techniques bring important improvements:

Overall, in this case study we have seen how synthetic data helps in cases when real labeled data is very hard to come by. The next study is also related to human faces but switches from variants of segmentation to a slightly different problem.

Facial Landmark Detection: Fine-Tuning Beats Domain Adaptation

For many facial analysis tasks, including face recognition, face frontalization, and face 3D modeling, one of the key steps is facial landmark detection , which aims to locate some predefined keypoints on facial components. In particular, in this case study we used 51 out of 68 IBUG facial landmarks. Note that there are several different standards of facial landmarks, as illustrated below ( Sagonas et al., 2016 ):

While this is a classic computer vision task with a long history, it unfortunately still suffers from many challenges in reality. In particular, many existing approaches struggle to cope with occlusions, extreme poses, difficult lighting conditions, and other problems. The occlusion problem is probably the most important obstacle to locating facial landmarks accurately.

As the basic model for recognizing facial landmarks, we use the stacked hourglass networks introduced by Newell et al. (2016) . The architecture consists of multiple hourglass modules, each representing a fully convolutional encoder-decoder architecture with skip connections:

Again, we do not go into full details regarding the architecture and training process because we have not changed the basic model, our emphasis is on its performance across different training sets.

The test set in this study consists of real images and comes from the 300 Faces In-the-Wild (300W) Challenge ( Sagonas et al., 2016 ). It consists of 300 indoor and 300 outdoor images of varying sizes that have ground truth manual labels. Here is a sample:

The real training set is a combination of several real datasets, semi-automatically unified to conform to the IBUG format. In total, we use ~3000 real images of varying sizes in the real training set.

For the synthetic training set, since real train and test sets mostly contain frontal or near-frontal good quality images, we generated a relatively restricted dataset with the Face API, without images in extreme conditions but with some added racial diversity, mild variety in camera angles, and accessories. The main features of our synthetic training set are:

• 10,000 synthetic images with 1024×1024 resolution, with randomized facial attributes and uniformly represented ethnicities;
• 10% of the images contain (clear) glasses;
• 60% of the faces have a random expression (emotion) with intensity [0.7, 1.0], and 20% of the faces have a random expression with intensity [0.1, 0.3];
• the maximum angle of face to camera is 45 degrees; camera position and face angles are selected accordingly.

Here are some sample images from our synthetic training set:

Next we present our key results that were achieved with the synthetic training data in several different attempts at closing the domain gap. We present a comparison between 5 different setups:

• trained on real data only;
• trained on synthetic data only;
• trained on a mixture of synthetic and real datasets;
• trained on a mixture of synthetic and real datasets with domain adaptation based on batchnorm statistics;
• pretrained on the synthetic dataset and fine-tuned on real data.

We measure two standard metrics on the 300W test set: normalized mean error (NME), the normalized average Euclidean distance between true and predicted landmarks (smaller is better), and probability of correct keypoint (PCK), the percentage of detections that fall into a predefined range of normalized deviations (larger is better).

The results clearly show that while it is quite hard to outperform the real-only benchmark (the real training set is large and labeled well, and the models are well-tuned to this kind of data), facial landmark detection can still benefit significantly from a proper introduction of synthetic data.

Even more interestingly, we see that the improvement comes not from simply training on a hybrid dataset but from pretraining on a synthetic dataset and fine-tuning on real data.

To further investigate this effect, we have tested the fine-tuning approach across a variety of real dataset subsets. The plots below show that as the size of the real dataset used for fine-tuning decreases, the results also deteriorate (this is natural and expected):

This fine-tuning approach is a training schedule that we have not often seen in literature, but here it proves to be a crucial component for success. Note that in the previous case study (background matting), we also tested this approach but it did not yield noticeable improvements.

In this whitepaper, we have considered simple ways to bring synthetic data into your computer vision projects. We have conducted three case studies for three different computer vision tasks related to human faces, using the power of Synthesis AI’s Face API to produce perfectly labeled and highly varied synthetic datasets. Let us draw some general conclusions from our results.

First of all, as the title suggests, it just works ! In all case studies, we have been able to achieve significant improvements or results on par with real data by using synthetically generated datasets, without complex domain adaptation models. Our results suggest that synthetic data is a simple but very efficient way to improve computer vision models, especially in tasks with complex labeling.

Second, we have seen that synthetic-to-real domain gap can be the same as real-to-real domain gap . This is an interesting result because it suggests that while domain transfer still, obviously, remains a problem, it is not specific to synthetic data, which proves to be on par with real data if you train and test in different conditions. We have supported this with our face segmentation study.

Third, even a small amount of synthetic data can help a lot . This is a somewhat counterintuitive conclusion: traditionally, synthetic datasets have been all about quantity and diversity over quality. However, we have found that in problems where labels are very hard to come by and are often imprecise, such as background matting in one of our case studies, even a relatively small synthetic dataset can go a long way towards getting the labels correct for the model.

Fourth, fine-tuning on real data after pretraining on a synthetic dataset seems to work better than training on a hybrid dataset . We do not claim that this will always be the case, but a common theme in our case studies is that these approaches may indeed yield different results, and it might pay to investigate both (especially since they are very straightforward to implement and compare).

We believe that synthetic data may become one of the main driving forces for computer vision in the near future, as real datasets reach saturation and/or become hopelessly expensive. In this whitepaper, we have seen that it does not have to be hard to incorporate synthetic data into your models. Try it, it might just work!

Embrace Synthetic Data.

Sign up for unspammy email with datasets, research papers, product updates, and tips for ML practitioners.

Applications

© 2024 Synthesis AI.

Explore datasets and labels with our Data Visualizer

Efficacy and safety of radiofrequency in the treatment of hidradenitis suppurativa; a systematic review

• Review Article
• Published: 25 May 2024
• Volume 39 , article number  139 , ( 2024 )

Cite this article

• Mohammad Ali Nilforoushzadeh   ORCID: orcid.org/0000-0001-6143-8039 1 , 2 ,
• Nazila Heidari   ORCID: orcid.org/0000-0001-7259-4926 1 , 2 , 3 ,
• Amirhossein Heidari   ORCID: orcid.org/0000-0003-4327-8814 1 , 2 , 4 ,
• Yekta Ghane   ORCID: orcid.org/0000-0002-7309-4066 1 , 2 , 5 ,
• Seyedayin Hosseini   ORCID: orcid.org/0000-0003-3221-2270 1 , 6 ,
• Zahra Lotfi   ORCID: orcid.org/0009-0003-2468-9877 1 ,
• Fariba Jaffary   ORCID: orcid.org/0000-0002-9378-9738 1 , 2 ,
• Minou Najar Nobari   ORCID: orcid.org/0009-0000-3010-3543 7 ,
• Zahra Sadat Aghamiri   ORCID: orcid.org/0009-0008-6544-7512 1 , 3 &
• Niloufar Najar Nobari   ORCID: orcid.org/0000-0002-4245-1980 1 , 2  

57 Accesses

2 Altmetric

Explore all metrics

Background and aims

Hidradenitis suppurativa (HS) is an inflammatory skin disease affecting apocrine gland-bearing sites of the body. Radiofrequency (RF) is a minimally invasive method that acts by minimizing thermal damage to the dermis, resulting in collagen synthesis and scar improvement. We systematically reviewed the efficacy and safety of RF in treating HS.

A systematic search was performed up to November 18th, 2023, in PubMed/Medline, Ovid Embase, and Web of Science. Clinical studies with English full texts were included. The National Institute of Health (NIH) Quality Assessment Tool for clinical trials and Methodological quality and synthesis of case series and case reports by Murad et al. were utilized for critical appraisal.

Out of 55 identified studies, 11 met our inclusion criteria with 167 subjects who underwent RF therapy alone or combined with an intense pulsed laser (IPL), known as LAight®. LAight® significantly improved clinical outcomes in mild-to-moderate HS patients based on the Dermatology Life Quality Index (DLQI), International Hidradenitis Suppurativa Score System (IHS4), Pain-Numerical Rating Scale (NRS), and Hidradenitis Suppurativa Clinical Response (HiSCR). Moreover, RF therapy alone significantly alleviated the clinical manifestations in patients with mild-to-moderate HS. Additionally, fractional microneedling RF significantly decreased HS-associated inflammatory markers. RF was found to be safe with limited adverse events. However, in moderate-to-severe HS, RF has failed to yield satisfactory results.

RF is a safe energy-based method with promising outcomes, especially for long-term application in mild-to-moderate HS. In moderate-to-severe cases, RF should be combined with a systemic medication for further beneficial impacts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Intense pulsed light treatment for hidradenitis suppurativa: a within-person randomized controlled trial

Laser and light-based treatment modalities for the management of hidradenitis suppurativa, evidence-based approach to the treatment of hidradenitis suppurativa/acne inversa, based on the european guidelines for hidradenitis suppurativa, data availability.

All relevant data are within the manuscript and its supplementary files.

Abbreviations

Dermatology Life Quality Index

Extracellular matrix

Hospital Anxiety and Depression Scale

Hidradenitis suppurativa Clinical Response

• Hidradenitis suppurativa

Intercellular adhesion molecule-1

Interferon gamma

International Hidradenitis Suppurativa Score System

Interleukin

• Intense pulsed light

Matrix metalloproteinase

Numerical Rating Scale

Physician Global Assessment

• Radiofrequency

Tumor necrose factor- alpha

Anduquia-Garay F, Rodríguez-Gutiérrez MM, Poveda-Castillo IT, Valdes-Moreno PL, Agudelo-Rios DA, Benavides-Moreno JS et al (2021) Hidradenitis suppurativa: Basic considerations for its approach: A narrative review. Ann Med Surg 68:102679. https://doi.org/10.1016/j.amsu.2021.102679

Article   Google Scholar  

Świerczewska Z, Lewandowski M, Surowiecka A, Barańska-Rybak W (2022) Immunomodulatory drugs in the treatment of hidradenitis suppurativa-possibilities and limitations. Int J Mol Sci 23(17). https://doi.org/10.3390/ijms23179716 .

Napolitano M, Megna M, Timoshchuk EA, Patruno C, Balato N, Fabbrocini G et al (2017) Hidradenitis suppurativa: from pathogenesis to diagnosis and treatment. Clin Cosmet Investig Dermatol 10:105–115. https://doi.org/10.2147/CCID.S111019

Article   CAS   PubMed   PubMed Central   Google Scholar  

Sodagar S, Ghane Y, Heidari A, Heidari N, Khodadust E, Ahmadi SAY et al (2023) Association between metabolic syndrome and prevalent skin diseases: A systematic review and meta-analysis of case-control studies. Health Sci Rep 6(9):e1576. https://doi.org/10.1002/hsr2.1576

Article   PubMed   PubMed Central   Google Scholar  

Ocker L, Abu Rached N, Seifert C, Scheel C, Bechara FG (2022) Current medical and surgical treatment of hidradenitis suppurativa& –a comprehensive review. J Clin Med 11(23):7240

Zouboulis CC, Bechara FG, Dickinson-Blok JL, Gulliver W, Horváth B, Hughes R et al (2019) Hidradenitis suppurativa/acne inversa: a practical framework for treatment optimization - systematic review and recommendations from the HS ALLIANCE working group. J Eur Acad Dermatol Venereol 33(1):19–31. https://doi.org/10.1111/jdv.15233

Article   CAS   PubMed   Google Scholar  

Scala E, Cacciapuoti S, Garzorz-Stark N, Megna M, Marasca C, Seiringer P et al (2021) Hidradenitis Suppurativa: Where We Are and Where We Are Going. Cells. 2021;10(8). https://doi.org/10.3390/cells10082094

Markota Čagalj A, Marinović B, Bukvić Mokos Z (2022) New and emerging targeted therapies for hidradenitis suppurativa. Int J Mol Sci 23(7). https://doi.org/10.3390/ijms23073753 .

Sabat R, Jemec GBE, Matusiak Ł, Kimball AB, Prens E, Wolk K (2020) Hidradenitis suppurativa. Nat Rev Dis Prim 6(1):18. https://doi.org/10.1038/s41572-020-0149-1

Article   PubMed   Google Scholar  

Schlapbach C, Hänni T, Yawalkar N, Hunger RE (2011) Expression of the IL-23/Th17 pathway in lesions of hidradenitis suppurativa. J Am Acad Dermatol 65(4):790–798. https://doi.org/10.1016/j.jaad.2010.07.010

van der Zee HH, de Ruiter L, van den Broecke DG, Dik WA, Laman JD, Prens EP (2011) Elevated levels of tumour necrosis factor (TNF)-α, interleukin (IL)-1β and IL-10 in hidradenitis suppurativa skin: a rationale for targeting TNF-α and IL-1β. Br J Dermatol 164(6):1292–1298. https://doi.org/10.1111/j.1365-2133.2011.10254.x

Bodkin AJ, Criswell SL (2021) Hidradenitis suppurativa lesions are associated with decreased collagen, decreased elastin, and increased neovascularization. J Histotechnol 44(2):80–89. https://doi.org/10.1080/01478885.2020.1861924

Alikhan A, Sayed C, Alavi A, Alhusayen R, Brassard A, Burkhart C et al (2019) North American clinical management guidelines for hidradenitis suppurativa: A publication from the United States and Canadian Hidradenitis Suppurativa Foundations: Part II: Topical, intralesional, and systemic medical management. J Am Acad Dermatol 81(1):91–101. https://doi.org/10.1016/j.jaad.2019.02.068

Ruiz-Esparza J, Gomez JB (2003) Nonablative radiofrequency for active acne vulgaris: the use of deep dermal heat in the treatment of moderate to severe active acne vulgaris (thermotherapy): a report of 22 patients. Dermatol Surg. 29(4):333–9. https://doi.org/10.1046/j.1524-4725.2003.29081.x . (discussion 9)

Iwasaki J, Marra DE, Fincher EF, Moy RL (2008) Treatment of hidradenitis suppurativa with a nonablative radiofrequency device. Dermatol Surg 34(1):114–117. https://doi.org/10.1111/j.1524-4725.2007.34025.x

Yang JH, Cho SI, Kim DH, Yoon JY, Moon J, Kim JW et al (2022) Pilot study of fractional microneedling radiofrequency for hidradenitis suppurativa assessed by clinical response and histology. Clin Exp Dermatol 47(2):335–342. https://doi.org/10.1111/ced.14905

Schultheis M, Staubach P, Nikolakis G, Grabbe S, Ruckes C, von Stebut E et al (2022) LAight® therapy significantly enhances treatment efficacy of 16 weeks of topical clindamycin solution in hurley I and II hidradenitis suppurativa: results from period A of RELIEVE, a Multicenter Randomized. Controlled Trial Dermatology 238(3):476–486. https://doi.org/10.1159/000518540

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD et al (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372:n71. https://doi.org/10.1136/bmj.n71

National Heart L, Blood Institute (2014) Quality assessment of clinical trials. https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools

National Heart L, and Blood Institute (2014) Quality assessment tool for before-after (Pre-Post) studies with no control group. https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools

Murad MH, Sultan S, Haffar S, Bazerbachi F (2018) Methodological quality and synthesis of case series and case reports. BMJ Evid Based Med 23(2):60–63. https://doi.org/10.1136/bmjebm-2017-110853

Schultheis M, Staubach P, Grabbe S, Ruckes C, von Stebut E, Kirschner U et al (2022) LAight® therapy is an effective treatment option to maintain long-term remission of hurley I and II hidradenitis suppurativa: results from period B of RELIEVE, a multicenter randomized, Controlled trial. Dermatology 238(6):1092–1103. https://doi.org/10.1159/000524739

Lyons A, Narla S, Kohli I, Zubair R, Jacobsen G, Ceresnie M et al (2022) Safety and efficacy of intense pulsed light with radiofrequency in united states hidradenitis suppurativa patients. J Drugs Dermatol 21(4):430–432. https://doi.org/10.36849/JDD.6562

Wilden S, Friis M, Tuettenberg A, Staubach-Renz P, Wegner J, Grabbe S et al (2021) Combined treatment of hidradenitis suppurativa with intense pulsed light (IPL) and radiofrequency (RF). J Dermatolog Treat 32(5):530–537. https://doi.org/10.1080/09546634.2019.1677842

Schultheis M, Scheuern M, Radkov I (2021) Combining deroofing with laight(®) therapy for long-term disease control of hurley stage II hidradenitis suppurativa: case report. Clin Pract 11(1):26–31. https://doi.org/10.3390/clinpract11010005

Kirschner U, Lang BM, Steinbrink K, Zimmer S (2020) Effectiveness of adalimumab in combination with intense pulsed light and radiofrequency therapy (LAight®) for severe hidradenitis suppurativa: a case report. J Dermatol Dermatol Surg-Jdds 24(2):133–134. https://doi.org/10.4103/jdds.jdds_36_19

Bisschoff IJ, Kasche D, Kirschner U, Krajewski PK, Matusiak L, Szepietowski JC (2020) LAight® therapy improves hidradenitis suppurativa in patients declining surgical intervention: two case reports. Acta Dermato-Venereologica 100. https://doi.org/10.2340/00015555-3558

Behrangi E, Atefi N, Mireshghollah P, Goodarzi A, Dehghani A, Dehnavi AZ et al (2023) The efficacy and safety of endo-radiofrequency for the treatment of hidradenitis suppurativa. Skin Res Technol 29(9). https://doi.org/10.1111/srt.13450

Mehta N, Khan E, Gupta S (2022) Radiofrequency epilation using acupuncture needles to decrease the disease burden in hidradenitis suppurativa. Dermatol Surg 48(7):788–789. https://doi.org/10.1097/DSS.0000000000003477

de Oliveira A, Bloise G, Moltrasio C, Coelho A, Agrelli A, Moura R et al (2022) Transcriptome meta-analysis confirms the hidradenitis suppurativa pathogenic triad: upregulated inflammation, altered epithelial organization, and dysregulated metabolic signaling. Biomolecules. 12(10). https://doi.org/10.3390/biom12101371

Verrier T, Solhonne B, Sallenave JM, Garcia-Verdugo I (2012) The WAP protein Trappin-2/Elafin: a handyman in the regulation of inflammatory and immune responses. Int J Biochem Cell Biol 44(8):1377–1380. https://doi.org/10.1016/j.biocel.2012.05.007

Pol A, Pfundt R, Zeeuwen P, Molhuizen H, Schalkwijk J (2003) Transcriptional regulation of the elafin gene in human keratinocytes. J Invest Dermatol 120(2):301–307. https://doi.org/10.1046/j.1523-1747.2003.12043.x

Vossen A, van der Zee HH, Prens EP (2018) hidradenitis suppurativa: a systematic review integrating inflammatory pathways into a cohesive pathogenic model. Front Immunol 9:2965. https://doi.org/10.3389/fimmu.2018.02965

Diaz MJ, Aflatooni S, Abdi P, Li R, Anthony MR, Neelam S et al (2023) Hidradenitis suppurativa: molecular etiology, pathophysiology, and management-a systematic review. Curr Issues Mol Biol 45(5):4400–4415. https://doi.org/10.3390/cimb45050280

Hotz C, Boniotto M, Guguin A, Surenaud M, Jean-Louis F, Tisserand P et al (2016) Intrinsic defect in keratinocyte function leads to inflammation in hidradenitis suppurativa. J Invest Dermatol 136(9):1768–1780. https://doi.org/10.1016/j.jid.2016.04.036

Gniadecki R, Jemec GB (2004) Lipid raft-enriched stem cell-like keratinocytes in the epidermis, hair follicles and sinus tracts in hidradenitis suppurativa. Exp Dermatol 13(6):361–363. https://doi.org/10.1111/j.0906-6705.2004.00166.x

Rousselle P, Braye F, Dayan G (2019) Re-epithelialization of adult skin wounds: Cellular mechanisms and therapeutic strategies. Adv Drug Deliv Rev 146:344–365. https://doi.org/10.1016/j.addr.2018.06.019

Heidari A, Ghane Y, Heidari N, Sadeghi S, Goodarzi A (2024) A systematic review of Janus kinase inhibitors and spleen tyrosine kinase inhibitors for Hidradenitis suppurativa treatment. Int Immunopharmacol 127:111435. https://doi.org/10.1016/j.intimp.2023.111435

Fischer AH, Haskin A, Okoye GA (2017) Patterns of antimicrobial resistance in lesions of hidradenitis suppurativa. J Am Acad Dermatol 76(2):309–13.e2. https://doi.org/10.1016/j.jaad.2016.08.001

Ertus C, Scailteux LM, Lescoat A, Berthe P, Auffret V, Dupuy A et al (2023) Major adverse cardiovascular events in patients with atopic dermatitis treated with oral Janus kinase inhibitors: a systematic review and meta-analysis. Br J Dermatol 189(4):368–380. https://doi.org/10.1093/bjd/ljad229

Li J, Zhang Z, Wu X, Zhou J, Meng D, Zhu P (2021) Risk of adverse events after anti-TNF treatment for inflammatory rheumatological disease. A meta-analysis. Front Pharmacol 12:746396. https://doi.org/10.3389/fphar.2021.746396

Ru Y, Ding X, Luo Y, Li H, Sun X, Zhou M et al (2021) Adverse events associated with anti-il-23 agents: clinical evidence and possible mechanisms. Front Immunol 12. https://doi.org/10.3389/fimmu.2021.670398

Hamzavi IH, Griffith JL, Riyaz F, Hessam S, Bechara FG (2015) Laser and light-based treatment options for hidradenitis suppurativa. J Am Acad Dermatol 73(5 Suppl 1):S78-81. https://doi.org/10.1016/j.jaad.2015.07.050

Dalrymple JC, Monaghan JM (1987) Treatment of hidradenitis suppurativa with the carbon dioxide laser. Br J Surg 74(5):420. https://doi.org/10.1002/bjs.1800740535

Fragoso NM, Masson R, Gillenwater TJ, Shi VY, Hsiao JL (2023) Emerging treatments and the clinical trial landscape for hidradenitis suppurativa—Part II: Procedural and wound care therapies. Dermatol Ther 13(8):1699–1720. https://doi.org/10.1007/s13555-023-00957-5

Kawana S, Tachihara R, Kato T, Omi T (2010) Effect of smooth pulsed light at 400 to 700 and 870 to 1,200 nm for acne vulgaris in Asian skin. Dermatol Surg 36(1):52–57. https://doi.org/10.1111/j.1524-4725.2009.01380.x

Dunbar SW, Goldberg DJ (2015) Radiofrequency in cosmetic dermatology: an update. J Drugs Dermatol 14(11):1229–1238

CAS   PubMed   Google Scholar  

Hantash BM, Renton B, Berkowitz RL, Stridde BC, Newman J (2009) Pilot clinical study of a novel minimally invasive bipolar microneedle radiofrequency device. Lasers Surg Med 41(2):87–95. https://doi.org/10.1002/lsm.20687

Download references

Acknowledgements

Not declared.

No external funding was received for this research.

Author information

Authors and affiliations.

Skin Repair Research Center, Shahid Beheshti University of Medical Sciences, Ahmadi Alley, Nelson Mandela St. 1518963113, MirdamadTehran, Iran

Mohammad Ali Nilforoushzadeh, Nazila Heidari, Amirhossein Heidari, Yekta Ghane, Seyedayin Hosseini, Zahra Lotfi, Fariba Jaffary, Zahra Sadat Aghamiri & Niloufar Najar Nobari

Skin and Stem Cell Research Center, Tehran University of Medical Sciences, Tehran, Iran

Mohammad Ali Nilforoushzadeh, Nazila Heidari, Amirhossein Heidari, Yekta Ghane, Fariba Jaffary & Niloufar Najar Nobari

School of Medicine, Iran University of Medical Sciences, Tehran, Iran

Nazila Heidari & Zahra Sadat Aghamiri

Faculty of Medicine, Tehran Medical Sciences, Islamic Azad University, Tehran, Iran

Amirhossein Heidari

School of Medicine, Tehran University of Medical Sciences, Tehran, Iran

Yekta Ghane

School of Medicine, Shahid Beheshti University of Medical Sciences, Tehran, Iran

Seyedayin Hosseini

Department of Orofacial Pain and Dysfunction, UCLA School of Dentistry, Los Angeles, CA, USA

Minou Najar Nobari

You can also search for this author in PubMed   Google Scholar

Contributions

Equal contributions to the current study are MaN and NH in the design, conceptualization, and revision of the final version of the manuscript. AH and YG in the database search, screening publications, data extraction, literature review, and manuscript drafting. SaH and ZL in data extraction and revising the final version of the manuscript. FJ, MnN, and ZsA are critically revising the final version of the manuscript. NnN in design, supervision, and revising the manuscript critically for the importance of intellectual content. All authors have read and approved the final version to be published and agreed to be accountable for all aspects of the work. All authors agreed on the order in which their names are listed in the manuscript.

Corresponding author

Correspondence to Niloufar Najar Nobari .

Ethics declarations

Ethical approval.

An ethics statement is not applicable because this study is based exclusively on published literature.

Competing interests

The authors declare there is no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Mohammad Ali Nilforoushzadeh and Nazila Heidari contributed equally to this work and shared the first authorship.

• Hidradenitis suppurativa (HS) is a burdensome, multifactorial, complex disease with unclear pathogenesis and limited effective treatments.

• Radiofrequency (RF) is an energy-based treatment method that decreases inflammatory mediators and stimulates collagen synthesis following minimal thermal injury.

• RF alone or in combination with an intense pulsed laser has been suggested as a potential, safe treatment option for patients with mild-to-moderate HS.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 39 KB)

Supplementary file2 (docx 27 kb), supplementary file3 (docx 20 kb), supplementary file4 (docx 58 kb), rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Nilforoushzadeh, M., Heidari, N., Heidari, A. et al. Efficacy and safety of radiofrequency in the treatment of hidradenitis suppurativa; a systematic review. Lasers Med Sci 39 , 139 (2024). https://doi.org/10.1007/s10103-024-04077-0

Download citation

Received : 26 January 2024

Accepted : 06 May 2024

Published : 25 May 2024

DOI : https://doi.org/10.1007/s10103-024-04077-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic review
• Find a journal
• Publish with us
• Track your research