Decisions	Information	Equivocality	Analyzability	Decision making
Producing all product families and acquiring information	Bills of materials and processes	HE	LA	Intuitive
Limiting products to needs of Latin American site	Forecasted demand, bills of materials and processes, experience	LE	HA	Intuitive and normative
Prioritizing modular production process	Experience, discussions, gut feeling	HE	LA	Intuitive
Agreeing on definition of a powertrain across product families	Experience, discussions, bills of materials and processes	HE	LA	Intuitive
Establishing rules and procedures for mapping powertrain components	Experience, discussions, bills of materials and processes	HE	LA	Intuitive
Mapping powertrain components	Bills of materials and processes, experience	LE	HA	Normative
Determining need for modular assembly of powertrains	Product demand, bills of materials and processes, experience	HE	LA	Intuitive
Identifying powertrain modules	Bills of material and processes, experience, prior activities	LE	HA	Intuitive and normative
Analyzing fit between current product design and powertrain modules	Bills of processes and materials, experience, spread sheet calculations	LE	HA	Normative
Identifying need for common assembly sequence	Experience, discussions, gut feeling, bills of materials and processes	HE	LA	Intuitive
Establishing rules and procedures for modular assembly	Experience, discussions, gut feeling, bills of materials and processes	HE	LA	Intuitive
Proposing a common assembly sequence for multi-product production system	Experience, discussions, gut feeling, bills of materials and processes	HE	LA	Intuitive
Analyzing differences between existing and common assembly sequence	Experience, discussions, gut feeling, bills of materials and processes	LE	LA	Intuitive and normative
Specifying common assembly sequence	Bills of processes and materials, experience	LE	LA	Intuitive and normative
Adapting production process to site specific needs	Experience, discussions, gut feeling, bills of materials and processes	LE	HA	Intuitive and normative
Identifying problems and improving production process	Bills of processes and materials, experience, spread sheet calculations, prototyping, line balancing, production databases	LE	HA	Intuitive and normative
Setting objective for reducing factory floor space	Experience, discussions, mental simulations, managerial reports	LE	HA	Intuitive and normative
Identifying needs of production site in Latin America	Bills of materials and processes, forecasted demand, line balancing, site visits, experience, discussions	LE	HA	Intuitive
Proposing layout for multi-product production system	Bills of materials and processes, forecasted demand, line balancing, site visits, experience, discussions and testing on site	E	LA	Intuitive and normative
Evaluating and testing layout for multi-product production system	Dimensions, production process, material flow, spread sheet, calculations and forecasted demand	LE	HA	Intuitive and normative
Setting objectives for standardizing tools for production process	Experience, discussions, prior activities	LE	LA	Intuitive
Mapping current equipment and tools	Bills of processes, work instructions, experience and site visits	LE	HA	Intuitive and normative
Specifying tools and equipment for multi-product production system	Bills of processes, work instructions, experience, prior activities, testing on site	LE	LA	Intuitive and normative
Prioritizing the reduction of traveling distance of internal logistics	Experience, discussions, prior activities	LE	LA	Intuitive
Identifying logistic needs for multi-product production system	Experience, discussions and mental simulations	LE	LA	Intuitive
Evaluating current logistics capabilities	Forecasted demand, assembly sequence, parts, routes, warehousing and on-site analysis	LE	HA	Normative
Proposing logistics solutions for multi-product production system	Forecasted demand, assembly sequence, parts, routes, warehousing, on-site analysis and testing on site	E	HA	Intuitive and normative
Agreeing on need for improving competence of operative staff	Experience, discussions and expert input	LE	HA	Intuitive
Determining critical issues for improving staff competence	Experience, discussions, expert input, prior activities, forecasted demand, line balancing, time studies and material flow	LE	HA	Intuitive
Specifying policies for staffing, organization strategies, and training	Experience, discussions, expert input, prior activities, forecasted demand, line balancing, time studies, material flow and testing on site	LE	HA	Intuitive and normative
Adopting performance indicators on site	Experience, discussions, expert input and managerial reports	LE	HA	Intuitive
Comparing current production system to a multi-product one	Prior activities, forecasted demand, material flow, spread sheet calculations, expert and management input	LE	HA	Intuitive and normative
Determining advantages and trade-offs of multi-product production system	Prior activities, forecasted demand, material flow, spread sheet calculations, expert and management input	E	HA	Intuition and normative

Notes: Equivocality (HE, high equivocality; E, equivocality; LE, low equivocality), Analyzability (HA, high analyzability; LA, low analyzability)

Ahlskog , M. , Bruch , J. and Jackson , M. ( 2017 ), “ Knowledge integration in manufacturing technology development ”, Journal of Manufacturing Technology Management , Vol. 28 No. 8 , pp. 1035 - 1054 .

Andersen , A.-L. , Brunoe , T.D. , Nielsen , K. and Rösiö , C. ( 2017 ), “ Towards a generic design method for reconfigurable manufacturing systems: analysis and synthesis of current design methods and evaluation of supportive tools ”, Journal of Manufacturing Systems , Vol. 42 No. 1 , pp. 179 - 195 .

Barratt , M. , Choi , T.Y. and Li , M. ( 2011 ), “ Qualitative case studies in operations management: trends, research outcomes, and future research implications ”, Journal of Operations Management , Vol. 29 No. 4 , pp. 329 - 342 .

Battaïa , O. , Dolgui , A. , Heragu , S.S. , Meerkov , S.M. and Tiwari , M.K. ( 2018 ), “ Design for manufacturing and assembly/disassembly: joint design of products and production systems ”, International Journal of Production Research , Vol. 56 No. 24 , pp. 7181 - 7189 .

Bellgran , M. and Säfsten , K. ( 2010 ), Production Development: Design and Operation of Production Systems , Springer Science & Business Media , New York, NY .

Bendoly , E. , Donohue , K. and Schultz , K.L. ( 2006 ), “ Behavior in operations management: assessing recent findings and revisiting old assumptions ”, Journal of Operations Management , Vol. 24 No. 6 , pp. 737 - 752 .

Bennett , R.H. ( 1998 ), “ The importance of tacit knowledge in strategic deliberations and decisions ”, Management Decision , Vol. 36 No. 9 , pp. 589 - 597 .

Bruch , J. and Bellgran , M. ( 2012 ), “ Creating a competitive edge when designing production systems – facilitating the sharing of design information ”, International Journal of Services Sciences , Vol. 4 Nos 3-4 , pp. 257 - 276 .

Byström , K. ( 2002 ), “ Information and information sources in tasks of varying complexity ”, Journal of the American Society for Information Science and Technology , Vol. 53 No. 7 , pp. 581 - 591 .

Cabagnols , A. and Le Bas , C. ( 2002 ), “ Differences in the determinants of product and process innovations: the french case ”, in Kleinknecht , A. and Mohnen , P. (Eds), Innovation and Firm Performance: Econometric Explorations of Survey Data , Palgrave Macmillan UK , London , pp. 112 - 149 .

Calabretta , G. , Gemser , G. and Wijnberg , N.M. ( 2017 ), “ The interplay between intuition and rationality in strategic decision making: a paradox perspective ”, Organization Studies , Vol. 38 Nos 3-4 , pp. 365 - 401 .

Cochran , D.S. , Foley , J.T. and Bi , Z. ( 2017 ), “ Use of the manufacturing system design decomposition for comparative analysis and effective design of production systems ”, International Journal of Production Research , Vol. 55 No. 3 , pp. 870 - 890 .

Cyert , R.M. and March , J.G. ( 1992 ), A Behavioral Theory of The Firm , Blackwell , Cambridge, MA .

Daft , R.L. and Lengel , R.H. ( 1986 ), “ Organizational information requirements, media richness and structural design ”, Management Science , Vol. 32 No. 5 , pp. 554 - 571 .

Daft , R.L. and Macintosh , N.B. ( 1981 ), “ A tentative exploration into the amount and equivocality of information processing in organizational work units ”, Administrative Science Quarterly , Vol. 26 No. 2 , pp. 207 - 224 .

Dane , E. and Pratt , M.G. ( 2007 ), “ Exploring intuition and its role in managerial decision making ”, Academy of Management Review , Vol. 32 No. 1 , pp. 33 - 54 .

Dane , E. , Rockmann , K.W. and Pratt , M.G. ( 2012 ), “ When should i trust my gut? Linking domain expertise to intuitive decision-making effectiveness ”, Organizational Behavior and Human Decision Processes , Vol. 119 No. 2 , pp. 187 - 194 .

Dean , J.W. and Sharfman , M.P. ( 1996 ), “ Does decision process matter? A study of strategic decision-making effectiveness ”, Academy of Management Journal , Vol. 39 No. 2 , pp. 368 - 392 .

Dudas , C. , Ng , A.H.C. , Pehrsson , L. and Boström , H. ( 2014 ), “ Integration of data mining and multi-objective optimisation for decision support in production systems development ”, International Journal of Computer Integrated Manufacturing , Vol. 27 No. 9 , pp. 824 - 839 .

Eisenhardt , K.M. ( 1989 ), “ Building theories from case study research ”, Academy of Management Review , Vol. 14 No. 4 , pp. 532 - 550 .

Eisenhardt , K.M. and Graebner , M.E. ( 2007 ), “ Theory building from cases: opportunities and challenges ”, Academy of Management Journal , Vol. 50 No. 1 , pp. 25 - 32 .

Elbanna , S. , Child , J. and Dayan , M. ( 2013 ), “ A model of antecedents and consequences of intuition in strategic decision-making: evidence from egypt ”, Long Range Planning , Vol. 46 No. 1 , pp. 149 - 176 .

Eling , K. , Griffin , A. and Langerak , F. ( 2014 ), “ Using intuition in fuzzy front-end decision-making: a conceptual framework ”, Journal of Product Innovation Management , Vol. 31 No. 5 , pp. 956 - 972 .

Ellström , P.-E. ( 2008 ), “ Knowledge creation through interactive research: a learning approach ”, Proceedings of the ECER Conference , Jönköping .

Eriksson , P.E. , Patel , P.C. , Sjödin , D.R. , Frishammar , J. and Parida , V. ( 2016 ), “ Managing interorganizational innovation projects: Mitigating the negative effects of equivocality through knowledge search strategies ”, Long Range Planning , Vol. 49 No. 6 , pp. 691 - 705 .

Frishammar , J. ( 2003 ), “ Information use in strategic decision making ”, Management Decision , Vol. 41 No. 4 , pp. 318 - 326 .

Frishammar , J. , Florén , H. and Wincent , J. ( 2011 ), “ Beyond managing uncertainty: insights from studying equivocality in the fuzzy front end of product and process innovation projects ”, IEEE Transactions on Engineering Management , Vol. 58 No. 3 , pp. 551 - 563 .

Frishammar , J. , Kurkkio , M. , Abrahamsson , L. and Lichtenthaler , U. ( 2012 ), “ Antecedents and consequences of firm’s process innovation capability: a literature review and a conceptual framework ”, IEEE Transactions on Engineering Management , Vol. 59 No. 4 , pp. 519 - 529 .

Galbraith , J.R. ( 1973 ), Designing Complex Organizations , Addison-Wesley Longman Publishing , Reading, MA .

Gaubinger , K. , Michael , R. , Swan , S. and Werani , T. ( 2014 ), Innovation and Product Management a Holistic and Practical Approach to Uncertainty Reduction , Springer , Berlin and Heidelberg .

Gershwin , S.B. ( 2018 ), “ The future of manufacturing systems engineering ”, International Journal of Production Research , Vol. 56 Nos 1–2 , pp. 224 - 237 .

Gigerenzer , G. and Gaissmaier , W. ( 2011 ), “ Heuristic decision making ”, Annual Review of Psychology , Vol. 62 No. 1 , pp. 451 - 482 .

Gino , F. and Pisano , G. ( 2008 ), “ Toward a theory of behavioral operations ”, Manufacturing & Service Operations Management , Vol. 10 No. 4 , pp. 676 - 691 .

Gore , J. and Sadler-Smith , E. ( 2011 ), “ Unpacking intuition: a process and outcome framework ”, Review of General Psychology , Vol. 15 No. 4 , pp. 304 - 316 .

Hämäläinen , R.P. , Luoma , J. and Saarinen , E. ( 2013 ), “ On the importance of behavioral operational research: the case of understanding and communicating about dynamic systems ”, European Journal of Operational Research , Vol. 228 No. 3 , pp. 623 - 634 .

Handfield , R.B. and Melnyk , S.A. ( 1998 ), “ The scientific theory-building process: a primer using the case of tqm ”, Journal of Operations Management , Vol. 16 No. 4 , pp. 321 - 339 .

Haußmann , C. , Dwivedi , Y.K. , Venkitachalam , K. and Williams , M.D. ( 2012 ), “ A summary and review of galbraith’s organizational information processing theory ”, in Dwivedi , Y.K. , Wade , M.R. and Schneberger , S.L. (Eds), Information Systems Theory: Explaining and Predicting Our Digital Society , Vol. 2 , Springer New York , New York, NY , pp. 71 - 93 .

Hodgkinson , G.P. , Sadler-Smith , E. , Burke , L.A. , Claxton , G. and Sparrow , P.R. ( 2009 ), “ Intuition in organizations: implications for strategic management ”, Long Range Planning , Vol. 42 No. 3 , pp. 277 - 297 .

Jacobides , M.G. ( 2007 ), “ The inherent limits of organizational structure and the unfulfilled role of hierarchy: lessons from a near-war ”, Organization Science , Vol. 18 No. 3 , pp. 455 - 477 .

Jalonen , H. ( 2011 ), “ The uncertainty of innovation: a systematic review of the literature ”, Journal of Management Research , Vol. 4 No. 1 , pp. 1 - 47 .

Johnson , P. , Buehring , A. , Cassell , C. and Symon , G. ( 2007 ), “ Defining qualitative management research: an empirical investigation ”, Qualitative Research in Organizations and Management: An International Journal , Vol. 2 No. 1 , pp. 23 - 42 .

Jonassen , D.H. ( 2012 ), “ Designing for decision making ”, Educational Technology Research and Development , Vol. 60 No. 2 , pp. 341 - 359 .

Julmi , C. ( 2019 ), “ When rational decision-making becomes irrational: a critical assessment and re-conceptualization of intuition effectiveness ”, Business Research , Vol. 12 No. 1 , pp. 291 - 314 .

Kahraman , C. , Onar , S.C. and Oztaysi , B. ( 2015 ), “ Fuzzy multicriteria decision-making: a literature review ”, International Journal of Computational Intelligence Systems , Vol. 8 No. 4 , pp. 637 - 666 .

Käki , A. , Kemppainen , K. and Liesiö , J. ( 2019 ), “ What to do when decision-makers deviate from model recommendations? Empirical evidence from hydropower industry ”, European Journal of Operational Research , Vol. 278 No. 3 , pp. 869 - 882 .

Karlsson , C. ( 2010 ), Researching Operations Management , Routledge , New York, NY .

Ketokivi , M. and Choi , T. ( 2014 ), “ Renaissance of case research as a scientific method ”, Journal of Operations Management , Vol. 32 No. 5 , pp. 232 - 240 .

Khatri , N. and Ng , H.A. ( 2000 ), “ The role of intuition in strategic decision making ”, Human Relations , Vol. 53 No. 1 , pp. 57 - 86 .

Kihlander , I. and Ritzén , S. ( 2012 ), “ Compatibility before completeness – identifying intrinsic conflicts in concept decision making for technical systems ”, Technovation , Vol. 32 No. 2 , pp. 79 - 89 .

Klein , R. , Koch , S. , Steinhardt , C. and Strauss , A.K. ( 2019 ), “ A review of revenue management: recent generalizations and advances in industry applications ”, European Journal of Operational Research , available at: https://doi.org/10.1016/j.ejor.2019.06.034

Koufteros , X. , Vonderembse , M. and Jayaram , J. ( 2005 ), “ Internal and external integration for product development: the contingency effects of uncertainty, equivocality, and platform strategy ”, Decision Sciences , Vol. 36 No. 1 , pp. 97 - 133 .

Krzeminska , A. and Eckert , C. ( 2015 ), “ Complementarity of internal and external r&d: Is there a difference between product versus process innovations? ”, R and D Management , Vol. 46 No. S3 , pp. 931 - 944 .

Kubler , S. , Robert , J. , Derigent , W. , Voisin , A. and Le Traon , Y. ( 2016 ), “ A state-of the-art survey & testbed of fuzzy AHP (FAHP) applications ”, Expert Systems with Applications , Vol. 65 , pp. 398 - 422 .

Kurkkio , M. , Frishammar , J. and Lichtenthaler , U. ( 2011 ), “ Where process development begins: a multiple case study of front end activities in process firms ”, Technovation , Vol. 31 No. 9 , pp. 490 - 504 .

Liao , Y. , Deschamps , F. , Loures , E.D.F.R. and Ramos , L.F.P. ( 2017 ), “ Past, present and future of industry 4.0 – a systematic literature review and research agenda proposal ”, International Journal of Production Research , Vol. 55 No. 12 , pp. 3609 - 3629 .

Liberatore , M.J. and Luo , W. ( 2010 ), “ The analytics movement: implications for operations research ”, INFORMS Journal on Applied Analytics , Vol. 40 No. 4 , pp. 313 - 324 .

Liu , R. and Hart , S. ( 2011 ), “ Does experience matter? – A study of knowledge processes and uncertainty reduction in solution innovation ”, Industrial Marketing Management , Vol. 40 No. 5 , pp. 691 - 698 .

Loch , C.H. and Wu , Y. ( 2007 ), “ Behavioral operations management ”, Foundations and Trends® in Technology, Information and Operations Management , Vol. 1 No. 3 , pp. 121 - 232 .

Luoma , J. ( 2016 ), “ Model-based organizational decision making: a behavioral lens ”, European Journal of Operational Research , Vol. 249 No. 3 , pp. 816 - 826 .

Mccutcheon , D.M. and Meredith , J.R. ( 1993 ), “ Conducting case study research in operations management ”, Journal of Operations Management , Vol. 11 No. 3 , pp. 239 - 256 .

March , J.G. and Simon , H.A. ( 1993 ), Organizations , Blackwell Business , Oxford .

Mardani , A. , Jusoh , A. and Zavadskas , E.K. ( 2015 ), “ Fuzzy multiple criteria decision-making techniques and applications – two decades review from 1994 to 2014 ”, Expert Systems with Applications , Vol. 42 No. 8 , pp. 4126 - 4148 .

Martinez-Ros , E. ( 1999 ), “ Explaining the decisions to carry out product and process innovations: the spanish case ”, The Journal of High Technology Management Research , Vol. 10 No. 2 , pp. 223 - 242 .

Marzi , G. , Dabić , M. , Daim , T. and Garces , E. ( 2017 ), “ Product and process innovation in manufacturing firms: a 30-year bibliometric analysis ”, Scientometrics , Vol. 113 No. 2 , pp. 673 - 704 .

Matzler , K. , Uzelac , B. and Bauer , F. ( 2014 ), “ Intuition’s value for organizational innovativeness and why managers still refrain from using it ”, Management Decision , Vol. 52 No. 3 , pp. 526 - 539 .

Meredith , J. ( 1998 ), “ Building operations management theory through case and field research ”, Journal of Operations Management , Vol. 16 No. 4 , pp. 441 - 454 .

Metters , R. , Queenan , C. , Ferguson , M. , Harrison , L. , Higbie , J. , Ward , S. , Barfield , B. , Farley , T. , Kuyumcu , H.A. and Duggasani , A. ( 2008 ), “ The ‘killer application’ of revenue management: Harrah’s cherokee casino & hotel ”, INFORMS Journal on Applied Analytics , Vol. 38 No. 3 , pp. 161 - 175 .

Mikalef , P. and Krogstie , J. ( 2018 ), Big Data Analytics As An Enabler of Process Innovation Capabilities: A Configurational Approach , Springer International Publishing , Cham , pp. 426 - 441 .

Miles , M.B. , Huberman , A.M. and Saldaña , J. ( 2013 ), Qualitative Data Analysis: A Methods Sourcebook , Sage Publications , Thousand Oaks, CA .

Milewski , S.K. , Fernandes , K.J. and Mount , M.P. ( 2015 ), “ Exploring technological process innovation from a lifecycle perspective ”, International Journal of Operations & Production Management , Vol. 35 No. 9 , pp. 1312 - 1331 .

Mintzberg , H. , Raisinghani , D. and Théorêt , A. ( 1976 ), “ The structure of ‘unstructured’ decision processes ”, Administrative Science Quarterly , Vol. 21 No. 2 , pp. 246 - 275 .

Papadakis , V.M. , Lioukas , S. and Chambers , D. ( 1998 ), “ Strategic decision-making processes: the role of management and context ”, Strategic Management Journal , Vol. 19 No. 2 , pp. 115 - 147 .

Parida , V. , Patel , P.C. , Frishammar , J. and Wincent , J. ( 2017 ), “ Managing the front-end phase of process innovation under conditions of high uncertainty ”, Quality & Quantity , Vol. 51 No. 5 , pp. 1983 - 2000 .

Piening , E.P. and Salge , T.O. ( 2015 ), “ Understanding the antecedents, contingencies, and performance implications of process innovation: a dynamic capabilities perspective ”, Journal of Product Innovation Management , Vol. 32 No. 1 , pp. 80 - 97 .

Pisano , G.P. ( 1997 ), The Development Factory: Unlocking the Potential of Process Innovation , Harvard Business Press , Boston, MA .

Reichstein , T. and Salter , A. ( 2006 ), “ Investigating the sources of process innovation among uk manufacturing firms ”, Industrial and Corporate Change , Vol. 15 No. 4 , pp. 653 - 682 .

Rönnberg , D.S. ( 2019 ), “ Knowledge processing and ecosystem co-creation for process innovation: Managing joint knowledge processing in process innovation projects ”, International Entrepreneurship and Management Journal , Vol. 15 No. 1 , pp. 135 - 162 .

Rönnberg , D.S. , Frishammar , J. and Eriksson , P.E. ( 2016 ), “ Managing uncertainty and equivocality in joint process development projects ”, Journal of Engineering and Technology Management , Vol. 39 No. 1 , pp. 13 - 25 .

Rönnberg , D.S. , Parida , V. , Leksell , M. and Petrovic , A. ( 2018 ), “ Smart factory implementation and process innovation ”, Research-Technology Management , Vol. 61 No. 5 , pp. 22 - 31 .

Rösiö , C. and Bruch , J. ( 2018 ), “ Exploring the design process of reconfigurable industrial production systems: activities, challenges, and tactics ”, Journal of Manufacturing Technology Management , Vol. 29 No. 1 , pp. 85 - 103 .

Saaty , T.L. ( 2008 ), “ Decision making with the analytic hierarchy process ”, Interntional Journal of Services Sciences , Vol. 1 No. 1 , pp. 83 - 98 .

Schneider , P. ( 2018 ), “ Managerial challenges of industry 4.0: an empirically backed research agenda for a nascent field ”, Review of Managerial Science , Vol. 12 No. 3 , pp. 803 - 848 .

Shapiro , S. and Spence , M.T. ( 1997 ), “ Managerial intuition: a conceptual and operational framework ”, Business Horizons , Vol. 40 No. 1 , pp. 63 - 68 .

Simon , H.A. ( 1997 ), Models of Bounded Rationality: Empirically Grounded Economic Reason , MIT Press , Cambridge, MA .

Stevens , E. ( 2014 ), “ Fuzzy front-end learning strategies: exploration of a high-tech company ”, Technovation , Vol. 34 No. 8 , pp. 431 - 440 .

Svensson , L. , Ellström , P.-E. and Brulin , G. ( 2007 ), “ Introduction–on interactive research ”, International Journal of Action Research , Vol. 3 No. 3 , pp. 233 - 249 .

Swamidass , P.M. ( 1991 ), “ Empirical science: new frontier in operations management research ”, Academy of Management Review , Vol. 16 No. 4 , pp. 793 - 814 .

Tamura , H. ( 2005 ), “ Behavioral models for complex decision analysis ”, European Journal of Operational Research , Vol. 166 No. 3 , pp. 655 - 665 .

Terjesen , S. and Patel , P.C. ( 2017 ), “ In search of process innovations: the role of search depth, search breadth, and the industry environment ”, Journal of Management , Vol. 43 No. 5 , pp. 1421 - 1446 .

Thakur , V. and Mangla , S.K. ( 2019 ), “ Change management for sustainability: evaluating the role of human, operational and technological factors in leading indian firms in home appliances sector ”, Journal of Cleaner Production , Vol. 213 , pp. 847 - 862 .

Whetten , D.A. ( 1989 ), “ What constitutes a theoretical contribution? ”, Academy of Management Review , Vol. 14 No. 4 , pp. 490 - 495 .

White , L. ( 2016 ), “ Behavioural operational research: towards a framework for understanding behaviour in or interventions ”, European Journal of Operational Research , Vol. 249 No. 3 , pp. 827 - 841 .

Yin , R.K. ( 2013 ), Case Study Research: Design and Methods , Sage Publications , Thousand Oaks, CA .

Yu , C. , Zhang , Z. , Lin , C. and Wu , J.Y. ( 2017 ), “ Knowledge creation process and sustainable competitive advantage: the role of technological innovation capabilities ”, Sustainability , Vol. 9 No. 12 , pp. 1 - 16 .

Zack , M.H. ( 2001 ), “ If managing knowledge is the solution, then what’s the problem? ”, in Yogesh , M. (Ed.), Knowledge Management And Business Model Innovation , IGI Global , Hershey, PA , pp. 16 - 36 .

Zack , M.H. ( 2007 ), “ The role of decision support systems in an indeterminate world ”, Decision Support Systems , Vol. 43 No. 4 , pp. 1664 - 1674 .

Zelt , S. , Schmiedel , T. and Vom Brocke , J. ( 2018 ), “ Understanding the nature of processes: an information-processing perspective ”, Business Process Management Journal , Vol. 24 No. 1 , pp. 67 - 88 .

Acknowledgements

The authors gratefully acknowledge the contributions of all the participants from the anonymous company used as a case study in this research. Financial support from the Knowledge Foundation (KKS), and the industrial graduate school “Innofacture” is also gratefully acknowledged.

Corresponding author

About the authors.

Erik Flores-Garcia is Doctoral Candidate at the Innofacture Industrial Graduate School, Mälardalen University, Sweden. His research interests include simulation, production decisions and process innovation.

Jessica Bruch is Professor in production systems at Mälardalen University, Sweden. Her research interest concerns various aspects of production development and addresses both technological and organizational aspects on the project, company and inter-organizational level.

Magnus Wiktorsson is Professor in production logistics at the Royal Institute of Technology (KTH), Sweden. His research interests include two ongoing major changes in production logistics: the digitization of all processes and the need for transformation into environmentally sustainable production.

Mats Jackson is Professor in innovative production at Jönköping University, Sweden. His research interests include flexibility of production systems, industrialization and innovation in production systems.

Related articles

All feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

IRESEARCH DECISION MAKING CASE STUDY

• Affiliation: University of West London

• University of the West of Scotland

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• Daan van Knippenberg

• ZB RAD EKON FAK RIJE
• Eva Bolfíková

• Jana Frenová

• J Risk Finance
• Ourania Theodosiadou

• Steven N. Tani
• Eric R. Johnson
• William B. Widhelm
• Derek W. Bunn

• Raimo P. Hämäläinen
• J OPER RES SOC
• Z. Govindarajulu
• Craig W. Kirkwood
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Emerging Topics

Actionable alm: the benefits of integrated asset and liability management and decision-making by leveraging enhanced business intelligence tools.

In today's rapidly changing economic and market environment, insurance companies face numerous challenges. These challenges encompass a wide range of factors, from the rapidly rising interest rates to geopolitical uncertainty. To navigate these challenges successfully, insurance companies should leverage available data and modern technology to make informed decisions and stay ahead of the curve. One useful tool is business intelligence (BI) software that empowers insurance companies to streamline their data-driven decision-making processes, improve risk management, and optimize operations to meet the evolving demands of customers and achieve strategic objectives.

This article introduces a forward-looking, action-driven asset liability management (ALM) reporting framework and then explores the benefits of leveraging BI tools in ALM decision-making. Lastly, a case study is presented to illustrate the practical applications of BI tools. Through this case study, readers will gain a deeper understanding of the tangible benefits and transformative potential of BI in real-world scenarios.

Introduction to ALM Reporting Framework

A comprehensive ALM reporting package should include a collection of management reports and charts that summarize the company’s asset and liability positions, as well as attribution of asset and liability value changes by key drivers, and tail risk measurements.

The ALM reporting framework typically involves four steps:

• Define ALM metrics: The metrics should be comprehensive and provide insights on the key risks that impact asset and liability values. A non-exhaustive list of metrics includes cashflow profiles, market and book values, gross and net yields, defaults and spreads, and interest rate and credit durations and convexities.
• Define key risk drivers: Insurers should define a set of key risk drivers that are common to both assets and liabilities. By using a common set of risk factors, the company can measure value changes from both sides of the balance sheet (i.e., risk-factor based attribution) and assess net exposures against its own risk appetite and tolerance.
• Establish risk exposure limits and management “playbook”: Once risk drivers are defined, insurers should set tolerance levels for each risk driver based on stress testing or other techniques. The risk exposures should be monitored closely against the limits. Insurers should establish a set of management actions to be taken in the event that risk exposures approach or exceed the defined limits.
• Take management action: When risk exposures approach the defined limits, management can refer to the playbook and choose the appropriate management action for risk mitigation. If the company receives significant benefits or payoffs by taking on a specific risk (e.g., net mark-to-market gains under rising interest rate environment due to significantly longer liability duration than asset duration), then management may choose to temporarily maintain the exposure level (i.e., no management actions) or execute management actions to lock-in the positive economic value (e.g., asset rotation or interest rate hedging). In the case of no management actions being taken, management needs to be aware of the potential downside risks (e.g., interest rate moves in the opposite direction, driving net economic asset value to drop).

Exhibit 1 is an illustrative example of common risk factor-based attribution that can be applicable to attribute value changes for both assets and liabilities.

Exhibit 1 Risk-factor Based Attribution

Exhibit 1 shows a set of common economic and non-economic risk factors that impact insurers’ asset and liability values. These risk factors are only for illustration purpose, and each company should build their attribution based on its own risk profile.

The ALM reporting framework provides the following key benefits that allow insurers to stay informed and competitive:

• More robust risk control and business stability:
• Clarity on whether risk taking activities are appropriately compensated.
• Agility in responding to changing market conditions.
• Effective risk management.
• Opportunities to improve risk-adjusted return through better usage of financial resources:
• Monitoring of investment performance against liability value changes and tailored risk limit and tolerance.
• Transparency around where and how value is created against the company’s risk appetite and limits.
• Actionable metrics to inform management action on improving ALM position.

Benefits of Business Intelligence Tools

ALM reporting can present various challenges to insurers in terms of data quality and timeliness, metric complexity, and cross-team collaboration. According to Oliver Wyman’s 2023 ALM survey, over 60% of participants selected one of these challenges as a top challenge in ALM reporting, illustrated in Exhibit 2 below. Only 57% of participants indicated they are able to spend more time on analysis than model maintenance.

Exhibit 2 ALM Reporting Challenges

In recent years, insurers have embraced BI tools to enhance the ALM reporting process and strategic decision-making. These tools quickly and efficiently generate comprehensive reports and visualizations that highlight underlying drivers of performance and the impact of various risk factors.

Key benefits of BI tools include:

• Access to real-time data from various sources: BI tools have the capability to collect and integrate data from multiple sources, enabling management to access pre-defined metrics as well as perform their own ad hoc analysis. Data within the BI tool is continuously updated in real-time, empowering management with dynamic and actionable insights based on the most current and relevant information. When combined with a pre-defined playbook, management is provided with real-time access to data and insights, enabling them to respond swiftly to changes in the economic environment.
• Business oriented interactive “what-if” analytics: BI tools provide user-friendly interfaces and self-service capabilities, allowing business users to independently access and analyze data with less reliance on IT support or interacting directly with the underlying data. By reducing the reliance on IT support, BI tools streamline the decision-making process and promote agility within organizations. Business users can create customized reports and visualizations tailored to their needs, empowering them to make faster and more informed decisions.
• Enhanced productivity: By providing streamlined processes to access data and generate reports in real-time, BI tools eliminate the need for manual data gathering and manipulation. This allows employees to focus on value-added tasks and gain insights into the available data. Additionally, BI tools offer features such as data visualization and interactive dashboards, which enable users to quickly interpret complex data sets. This enhanced productivity also empowers management to make data-driven decisions and take proactive actions to drive business growth.
• Version control and information sharing: BI tools provide collaboration features that enable users to share reports and dashboards with other stakeholders within the organization. This promotes information sharing and reduces the risk of working with outdated or conflicting information. To increase traceability, BI tools include version control capabilities that track changes made to reports over time and allows teams to build upon the same version of the BI tool.

ALM dashboards serve as valuable decision-making tools for insurance companies in today’s rapidly changing environment. Coupling the ALM dashboard with BI tools addresses operational challenges associated with growing data size and multiple data sources, enabling organizations to effectively analyze and manage key risks embedded in their business.

Case Study—ALM Dashboard

The life insurance company XYZ manages ALM on an economic basis. Their ALM report has been fairly basic and simple over the past 20 years. The ALM report was produced in Excel which required the ALM team to gather data from various areas within the organization and perform manual data processing and manipulation. The end-to-end ALM reporting processes took about three months to complete. As rates have rapidly increased and the market has been more volatile in recent years, the company realized the necessity for more robust and effective ALM reporting to help navigate evolving market conditions.

Exhibit 3 below illustrates XYZ’s current ALM report. It includes standard ALM metrics such as asset mix, market values, gross and net yields, interest rate duration and convexity, and spread.

Exhibit 3 ALM Position Report

XYZ’s current reporting also includes asset and liability cashflow profiles. This provides insight into the timing and magnitude of projected asset and liability cashflows, enabling XYZ to proactively manage liquidity needs. (see Exhibit 4)

Exhibit 4 Asset and Liability Cashflow Profile

In recent years, the interest rate curve has been inverted and there is uncertainty around the future shape of the curve. XYZ incorporated key rate duration (KRD) to their ALM report to assess the sensitivity of assets and liabilities to individual interest rate tenors. This is important because there has been significant deviation of rate changes by tenors and future interest rate curve changes may not be parallel either. This allows XYZ to gain a more detailed understanding of duration mis-matches that may exist. (see Exhibit 5)

Exhibit 5 Key Rate Duration (KRD)

XYZ also implemented a risk-factor based attribution analysis that breaks down the asset and liability value changes into the common set of risk factors. As demonstrated in Exhibit 6 below, the main drivers of the change in net market value in this reporting period are interest rates and new business.

Exhibit 6 Risk-factor Based Attribution

As part of the broader transformation initiative at XYZ, the ALM team worked with IT to implement the ALM dashboard in a data visualization tool and connect it to a cloud-based data warehouse. The data warehouse contains preprocessed and standardized liabilities, assets, assumptions, and other company data. The ALM team and senior management are granted access to a web-based portal from which they can access the pre-defined ALM reports, build custom reports, and drill down to more granular analytics and underlying data. Additionally, the system incorporates a trigger that sends an email notification to management when risk exposures approach or exceed limits. (see Exhibit 7)

Exhibit 7 ALM Projection Ecosystem

After all the enhancements made, the end-to-end ALM reporting time is reduced to four weeks. Management also has deeper understanding of their assets, liabilities, and risk-factors, which enables them to make more informed and timely decisions in a changing economic environment.

Statements of fact and opinions expressed herein are those of the individual authors and are not necessarily those of the Society of Actuaries, the newsletter editors, or the respective authors’ employers .

Joy Chen, FSA, MAAA, CERA, is a principal at Oliver Wyman. She can be reached at [email protected]

Sarah Cook, FSA, MAAA, is a consultant at Oliver Wyman. She can be reached at [email protected]

Mandy Jiao, ASA, is a manager at Oliver Wyman. She can be reached at [email protected]

Seong-Weon Park, FSA, MAAA, is a senior principal at Oliver Wyman. He can be reached at [email protected]

Evolution of Management Research in Greece: A Comparative Analysis of 2014–2018 and 2019–2023 Trends

• Published: 21 August 2024

Cite this article

• Thomas Krabokoukis   ORCID: orcid.org/0000-0001-5834-682X 1 &
• Dimitrios Kantianis 2  

Explore all metrics

This research investigates the evolution of research topics in the field of management in Greece, analyzing the period 2014–2023 and comparing the periods 2014–2018 and 2019–2023. Employing co-occurrence analysis on a substantial corpus of keywords, distinctive thematic clusters were identified for each period, reflecting shifts and patterns in research interests. In the 2014–2018 period, five clusters emerged, focusing on management practices, environmental sustainability, methodology and quality, strategic decision-making, and innovation. In contrast, the 2019–2023 period revealed four clusters emphasizing strategic decision-making in Small and Medium-sized Enterprises (SMEs), sustainable practices and risk management, crisis response, innovation, and quality management. Common themes such as Sustainability and Knowledge Management persisted across both periods, signifying enduring research interests. This study contributes insights into the dynamic landscape of management research in Greece over these distinct timeframes.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

COVID-19 in Relation to Business and Management: A Bibliometric Analysis

ESG literature mapping: insights from bibliometric analysis

Mapping the knowledge domains of COVID-19 and firm performance: main research trends and future direction

Explore related subjects.

• Artificial Intelligence

Data availability

The data supporting this study's findings were obtained from the Scopus database. The processed data were analyzed using VOSviewer.

Angelidou, M. (2015). Smart cities: A conjuncture of four forces. Cities, 47 , 95–106. https://doi.org/10.1016/j.cities.2015.05.004

Article   Google Scholar  

Belias, D., Rossidis, I., Velissariou, E., Amoiradis, C., Tsiotas, D., & Sdrolias, L. (2018). Successful and efficient knowledge management in the greek hospitality industry: Change the perspective!  Academic Journal of Interdisciplinary Studies, 7 (1), 185–191. https://doi.org/10.2478/ajis-2018-0019

Boufounou, P., & Argyrou, M. D. (2022). Changing the organizational culture to transform the economy: The case of Greece. Frontiers in Research Metrics and Analytics, 7 , 1050544. https://doi.org/10.3389/frma.2022.1050544

Crespy, A., & Ladi, S. (2019). In the name of ‘the people’? Popular Sovereignty and the 2015 Greek referendum. Journal of European Integration, 41 (7), 871–885. https://doi.org/10.1080/07036337.2019.1665661

Dey, P. K., Malesios, C., De, D., Budhwar, P., Chowdhury, S., & Cheffi, W. (2020). Circular economy to enhance sustainability of small and medium-sized enterprises. Business Strategy and the Environment, 29 (6), 2145–2169. https://doi.org/10.1002/bse.2492

Diakaki, C., Papageorgiou, M., Papamichail, I., & Nikolos, I. (2015). Overview and analysis of Vehicle Automation and Communication Systems from a motorway traffic management perspective. Transportation Research Part a: Policy and Practice, 75 , 147–165. https://doi.org/10.1016/j.tra.2015.03.015

Diamantidis, A. D., & Chatzoglou, P. (2019). Factors affecting employee performance: An empirical approach. International Journal of Productivity and Performance Management, 68 (1), 171–193. https://doi.org/10.1108/IJPPM-01-2018-0012

Diamantoulakis, P. D., Kapinas, V. M., & Karagiannidis, G. K. (2015). Big Data Analytics for Dynamic Energy Management in Smart Grids. Big Data Research, 2 (3), 94–101. https://doi.org/10.1016/j.bdr.2015.03.003

Giannoulakis, S., & Sakellaris, P. (2023). Financial crises, firm-level shocks and large downturns: Evidence from Greece. International Journal of Finance & Economics, 28 (2), 1549–1562. https://doi.org/10.1002/ijfe.2493

Ioannis, K., & Petr, R. (2022). The Contribution of Corporate Social Responsibility and Organizational Behavior in the Development of Human Resources of Companies: Case Study in Greece. Academic Journal of Interdisciplinary Studies , 11 (4), 56. https://doi.org/10.36941/ajis-2022-0098

Kapitsinis, N. (2023). The 2007/08 capitalist crisis evolution in Greece: A geographical political economy perspective. Human Geography, 16 (1), 3–16. https://doi.org/10.1177/19427786221096873

Krabokoukis, T. (2023). Exploring the state of research on tourism Sustainability: A bibliometric analysis in the post-covid era. Highlights of Sustainability, 2 (2), 50–61. https://doi.org/10.54175/hsustain2020005 .

Krabokoukis, T., & Polyzos, S. (2023). A Bibliometric Analysis of Integrating Tourism Development into Urban Planning. Sustainability, 15 (20), 14886. https://doi.org/10.3390/su152014886

Krabokoukis, T., Polyzos, S., & Kantianis, D. (2024). Mapping the landscape of transport infrastructure and regional development: A comprehensive bibliometric analysis. Theoretical and Empirical Researches in Urban Management , 19 (1), 5–29.

Madzík, P., Falát, L., Copuš, L., & Valeri, M. (2023). Digital transformation in tourism: Bibliometric literature review based on machine learning approach. European Journal of Innovation Management, 26 (7), 177–205. https://doi.org/10.1108/EJIM-09-2022-0531

Mariolis, T., Rodousakis, N., & Soklis, G. (2022). Inter-sectoral analysis of the Greek economy and the COVID-19 multiplier effects. European Politics and Society, 23 (4), 505–516. https://doi.org/10.1080/23745118.2021.1895555

Mastos, T. D., Nizamis, A., Vafeiadis, T., Alexopoulos, N., Ntinas, C., Gkortzis, D., Papadopoulos, A., Ioannidis, D., & Tzovaras, D. (2020). Industry 40 sustainable supply chains: An application of an IoT enabled scrap metal management solution. Journal of Cleaner Production, 269 , 122377. https://doi.org/10.1016/j.jclepro.2020.122377

Mikalef, P., & Pateli, A. (2017). Information technology-enabled dynamic capabilities and their indirect effect on competitive performance: Findings from PLS-SEM and fsQCA. Journal of Business Research, 70 , 1–16. https://doi.org/10.1016/j.jbusres.2016.09.004

Nikolaidis, Y., & Adamidou, A. (2016). Quality assurance mapping in Greek service companies. The TQM Journal, 28 (3), 431–454. https://doi.org/10.1108/TQM-02-2014-0021

Panagiotakopoulos, A. (2013). The impact of employee learning on staff motivation in Greek small firms: The employees’ perspective. Development and Learning in Organizations: An International Journal, 27 (2), 13–15. https://doi.org/10.1108/14777281311302030

Petrakos, G., Rontos, K., Vavoura, C., & Vavouras, I. (2023). The Impact of Recent Economic Crises on Income Inequality and the Risk of Poverty in Greece. Economies, 11 (6), 166. https://doi.org/10.3390/economies11060166

Sánchez-Camacho, C., Carranza, R., Martín-Consuegra, D., & Díaz, E. (2022). Evolution, trends and future research lines in corporate social responsibility and tourism: A bibliometric analysis and science mapping. Sustainable Development, 30 (3), 462–476. https://doi.org/10.1002/sd.2260

Sigala, M. (2013). Examining the adoption of destination management systems: An inter-organizational information systems approach. Management Decision, 51 (5), 1011–1036. https://doi.org/10.1108/MD-11-2012-0800

Sigala, M., & Chalkiti, K. (2015). Knowledge management, social media and employee creativity. International Journal of Hospitality Management, 45 , 44–58. https://doi.org/10.1016/j.ijhm.2014.11.003

Sools, A., Triliva, S., Fragkiadaki, E., Tzanakis, M., & Gkinopoulos, T. (2018). The Greek Referendum Vote of 2015 as a Paradoxical Communicative Practice: A Narrative. Future-Making Approach. Political Psychology, 39 (5), 1141–1156. https://doi.org/10.1111/pops.12474

Spinthiropoulos, K., Tsiatsiou, E., Garefalakis, A., Chaitidis, G., & Stavropoulou, E. (2021). Greek economic crisis management program and its impact on public employees’ earnings: A case series and literature review. Polish Journal of Management Studies , 24 (2), 431–448. https://doi.org/10.17512/pjms.2021.24.2.27

Stylos, N., Vassiliadis, C. A., Bellou, V., & Andronikidis, A. (2016). Destination images, holistic images and personal normative beliefs: Predictors of intention to revisit a destination. Tourism Management, 53 , 40–60. https://doi.org/10.1016/j.tourman.2015.09.006

Tsiotas, D., Dialesiotis, S., & Christopoulou, O. (2023). Examining the relationship between regional economic resilience and epidemiologic spread of COVID-19: Evidence from Greece. Environment, Development and Sustainability. https://doi.org/10.1007/s10668-023-04240-7

Tsirbas, Y., & Zirganou-Kazolea, L. (2022). Greek political scientists under the crisis and the case of the Greek bailout referendum: An intellectual barricade protecting the status quo? European Political Science, 21 (1), 58–74. https://doi.org/10.1057/s41304-021-00339-5

Vlachos, P. A., Panagopoulos, N. G., & Rapp, A. A. (2014). Employee judgments of and behaviors toward corporate social responsibility: A multi-study investigation of direct, cascading, and moderating effects. Journal of Organizational Behavior, 35 (7), 990–1017. https://doi.org/10.1002/job.1946

Download references

Author information

Authors and affiliations.

Department of Planning and Regional Development, University of Thessaly, 382 21, Volos, Greece

Thomas Krabokoukis

Department of Economic and Regional Development, Panteion University, 176 71, Athens, Greece

Dimitrios Kantianis

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Thomas Krabokoukis .

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Krabokoukis, T., Kantianis, D. Evolution of Management Research in Greece: A Comparative Analysis of 2014–2018 and 2019–2023 Trends. J Knowl Econ (2024). https://doi.org/10.1007/s13132-024-02293-1

Download citation

Received : 22 May 2024

Accepted : 10 August 2024

Published : 21 August 2024

DOI : https://doi.org/10.1007/s13132-024-02293-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Management research
• Thematic analysis
• Co-occurrence analysis
• Crisis management
• Sustainability
• Find a journal
• Publish with us
• Track your research

• Study Protocol
• Open access
• Published: 19 August 2024

Evaluating the impact of the global evidence, local adaptation (GELA) project for enhancing evidence-informed guideline recommendations for newborn and young child health in three African countries: a mixed-methods protocol

• Tamara Kredo   ORCID: orcid.org/0000-0001-7115-9535 1 , 2 , 3 , 4 ,
• Emmanuel Effa 5 ,
• Nyanyiwe Mbeye 6 ,
• Denny Mabetha 1 ,
• Bey-Marrié Schmidt 1 , 7 ,
• Anke Rohwer 2 ,
• Michael McCaul 2 ,
• Idriss Ibrahim Kallon 2 ,
• Susan Munabi-Babigumira 8 ,
• Claire Glenton 8 ,
• Taryn Young 2 ,
• Simon Lewin 1 , 9 ,
• Per Olav Vandvik 10 , 11 &
• Sara Cooper 2 , 4 , 12  

Health Research Policy and Systems volume  22 , Article number:  114 ( 2024 ) Cite this article

90 Accesses

Metrics details

Poverty-related diseases (PRD) remain amongst the leading causes of death in children under-5 years in sub-Saharan Africa (SSA). Clinical practice guidelines (CPGs) based on the best available evidence are key to strengthening health systems and helping to enhance equitable health access for children under five. However, the CPG development process is complex and resource-intensive, with substantial scope for improving the process in SSA, which is the goal of the Global Evidence, Local Adaptation (GELA) project. The impact of research on PRD will be maximized through enhancing researchers and decision makers’ capacity to use global research to develop locally relevant CPGs in the field of newborn and child health. The project will be implemented in three SSA countries, Malawi, South Africa and Nigeria, over a 3-year period. This research protocol is for the monitoring and evaluation work package of the project. The aim of this work package is to monitor the various GELA project activities and evaluate the influence these may have on evidence-informed decision-making and guideline adaptation capacities and processes. The specific project activities we will monitor include (1) our ongoing engagement with local stakeholders, (2) their capacity needs and development, (3) their understanding and use of evidence from reviews of qualitative research and, (4) their overall views and experiences of the project.

We will use a longitudinal, mixed-methods study design, informed by an overarching project Theory of Change. A series of interconnected qualitative and quantitative data collections methods will be used, including knowledge translation tracking sheets and case studies, capacity assessment online surveys, user testing and in-depth interviews, and non-participant observations of project activities. Participants will comprise of project staff, members of the CPG panels and steering committees in Malawi, South Africa and Nigeria, as well as other local stakeholders in these three African countries.

Ongoing monitoring and evaluation will help ensure the relationship between researchers and stakeholders is supported from the project start. This can facilitate achievement of common goals and enable researchers in South Africa, Malawi and Nigeria to make adjustments to project activities to maximize stakeholder engagement and research utilization. Ethical approval has been provided by South African Medical Research Council Human Research Ethics Committee (EC015-7/2022); The College of Medicine Research and Ethics Committee, Malawi (P.07/22/3687); National Health Research Ethics Committee of Nigeria (01/01/2007).

Peer Review reports

Sub-Saharan Africa (SSA) has the highest under-five mortality rate in the world [ 1 ]. Although the global under-five mortality rate declined from 76 to 38 per 1000 live births between 2000 and 2019, more than half of the deaths in children and youth in 2019 were among children under 5 years, approximately 5.2 million deaths [ 1 ]. Poverty-related diseases including pneumonia, diarrhoea and malaria remain amongst the leading causes of death in children under-5 years [ 2 ].Thus, despite progress in the health of young children globally, most countries in SSA fall below the average gains and do not meet maternal and child health targets set by the United Nations Sustainable Development Goal 3 to ‘ensure healthy lives and promote wellbeing’ (1). As of December 2021, under-five mortality rates were reported as 113.8, 38.6 and 32.2 per 1000 live births for Nigeria, Malawi and South Africa, respectively [ 3 ]. Factors accounting for regional disparities in child mortality rates include poverty, socioeconomic inequities, poor health systems and poor nutrition, with coronavirus disease 2019 (COVID-19) adding substantially to the burden [ 4 ].

Addressing healthcare issues such as these requires an evidence-informed approach, where intervention design and implementation are based on the best available evidence, to ensure that scarce resources are used effectively and efficiently, avoid harm, maximize good and improve healthcare delivery and outcomes [ 5 , 6 , 7 ]. Evidence-informed practices have been growing in SSA [ 6 ], and evidence ecosystems are becoming stronger. The evidence ecosystem reflects the formal and informal linkages and interactions between different actors (and their capacities and resources) involved in the production, translation and use of evidence [ 6 , 8 , 9 ]. Guidance that can be developed through this ecosystem includes evidence-based health technology assessments (HTA) and clinical practice guidelines (CPGs). CPGs include recommendations that are actionable statements that are informed by systematic reviews of evidence, and an assessment of the benefits and harms of alternative care options and are intended to optimize patient care [ 10 ]. They can help bridge the gap between research evidence and practice and are recognized as important quality-improvement tools that aim to standardize care, inform funding decisions and improve access to care, among others.

CPG method advancements, challenges and research gaps

Over the past decade, internationally and in SSA, there has been a rapid growth of CPGs developed for a range of conditions [ 11 ]. In particular, rapid evidence syntheses and guideline development methods has advanced in response to urgent evidence needs, especially during COVID [ 12 , 13 ]. For example, WHO has developed guidelines for all key infectious conditions that cause most deaths. This development has been accompanied by a growing volume of research evidence around CPGs, including the processes for their rapid development, adaptation, contextualization, implementation and evaluation, and further spurred on by COVID. For example, global knowledge leaders, such as the WHO and the GRADE Working Group, have set standards for CPG development, outlining the steps of what is known as ‘de novo’ (from scratch) CPG development [ 14 ]. Another global group, the Guidelines International Network (G-I-N), is a network dedicated to leading, strengthening and supporting collaboration in CPG development, adaptation and implementation. They have published minimum standards and the G-I-N McMaster guideline checklist, which contains a comprehensive list of topics and items outlining the practical steps to consider for developing CPGs [ 15 ].

As CPG standards have evolved, however, so has the complexity of development and adaptation. In the context of poorer settings, such as sub-Saharan Africa (SSA), CPG development is prohibitively human and finance resource intensive. It requires scarce skills, even in the growing evidence-based healthcare (EBHC) community, and financial investments by government where resources are often directed to healthcare services, rather than policymaking processes. Against this backdrop, several studies have found that CPGs in the region often perform poorly on reporting on their rigour of development and editorial independence [ 16 , 17 , 18 ]. Other, more resource-efficient methods for guideline development in SSA are, therefore, essential and urgently needed. Moreover, investment in the overall management of the process is needed, including convening the guideline group and moving stepwise through a rigorous process.

Approaches for and challenges of guideline adaptation

There is also increased international recognition of the value of taking guidelines developed in one country and applying them to other countries. This can avoid duplication of effort and research waste in de novo guideline development, when useful guidelines may exist elsewhere [ 12 , 19 ]. Against this backdrop, several adaptation methods are emerging for contextualization of recommendations to country needs (e.g. ADAPTE, adolopment and SNAP-it, amongst others) [ 19 , 20 , 21 ]. For example, WHO is developing strategies for adapting and implementing their CPGs at country level. One example is the WHO Antenatal Care Recommendations Adaptation Toolkit lead by the Department of Sexual and Reproductive Health and Research [ 22 ]. Their approach is pragmatic and transparent. Another approach is so-called ‘adolopment’, a GRADE method, in which the original guideline evidence is used, either adopted or adapted, considering contextual evidence such as costs and feasibility and local values [ 20 ]. Adolopment involves convening a guideline panel, reviewing available evidence and local contextual evidence and weighing up the panel’s judgements to make recommendations that are fit for purpose [ 20 ].

Despite these advances in CPG adaptation methods, many countries and professional associations in sub-Saharan Africa still use expert opinion-based approaches or proceed to prepare their own systematic reviews and guidelines, ultimately perpetuating resource wastage and duplication of efforts [ 23 ]. Moreover, when countries do adapt and contextualize other countries’ guidelines, there is frequently a lack of transparency and reporting on changes, without clarity on why or by whom. This in turn casts doubts on the recommendation’s credibility. For example, guidelines for child health in sub-Saharan Africa are usually derived from the WHO and UNICEF. However, adaptation of such guidelines and recommendations to national contexts is not well described [ 24 ]. Transparency in guideline adaptation is critical for creating trustworthy, context-sensitive recommendations. What guideline adaptation methods work best and how these can be transparently implemented in the context of lower resource settings, remain key research questions. Therefore, despite the emergence of several guideline adaptation approaches, we need to explore and understand how best to adapt recommendations from one context to another [ 25 ].

Qualitative evidence to inform guideline panels decisions

Another major advancement within guideline research has been growing recognition of the potential contribution of qualitative research evidence [ 26 , 27 ]. Traditionally, guidelines have been informed by systematic reviews of the effectiveness of specific interventions [ 14 ]. Such reviews provide robust evidence about which interventions ‘work’. However, there is appreciation that evidence regarding the potential effectiveness of an intervention is not sufficient for making recommendations or decisions. Policymakers also need to consider other issues, including how different stakeholders’ value different outcomes, the intervention’s acceptability to those affected by it and the feasibility of implementing the intervention [ 28 , 29 , 30 ]. Evidence from qualitative research is particularly well suited to exploring factors that influence an intervention’s acceptability and feasibility [ 31 , 32 ]. The use of qualitative research to inform recommendations by guidelines has become easier in recent years as systematic reviews of qualitative studies have become more common, and the methods for these reviews are now well developed [ 33 ]. The first WHO guideline to systematically incorporate reviews of qualitative studies was published in 2012 in the field of task-shifting for maternal and child health [ 31 ]. The inclusion of this qualitative evidence helped shape the panel’s recommendations [ 32 ], and this approach is now included in the WHO Handbook for Guideline Development and has been applied in many other WHO CPGs [ 34 , 35 ].

However, a key challenge in using findings from systematic reviews of qualitative evidence is communicating often complex findings to users such as guideline panel members to facilitate effective knowledge translation. While there is now considerable research on communicating findings from reviews of intervention effectiveness [ 36 ], there is limited experience on the usefulness of different options for packaging and presenting findings from systematic reviews of qualitative evidence to CPG panels. To make best use of this evidence, we need presentation formats that are accessible to users who may be unfamiliar with qualitative methods, are concise and simple while retaining sufficient detail to inform decisions and clearly present ‘confidence in the evidence from systematic reviews of qualitative evidence’ (GRADE-CERQual) assessments of how much confidence users should place in each finding [ 37 ]. In addition, we need to understand how qualitative evidence included in global guidelines, such as those produced by WHO, is interpreted and used in country-level guideline adaptation processes.

Communicating clinical practice guidelines to end-users

A final key guideline method advancement has been around the development of multi-layered and digitally structured communication formats for end users [ 38 , 39 ]. Guidelines are not an end in themselves. Recommendations may lack impact if not adequately communicated and disseminated to those who need to implement them, namely healthcare providers, managers and the public. Indeed, in a South African study of primary care guideline national policymakers, subnational health managers and healthcare providers agreed that dissemination is a particular gap [ 40 ]. While guidelines typically are produced as static documents (e.g. PDF formats), information technology is needed to enhance dissemination. The MAGIC authoring and publication Platform (MAGICapp/) was developed for this purpose ( https://magicevidence.org/magicapp/ ). MAGICapp is a web-based tool that enables evidence synthesizers and guideline organizations to create, publish and dynamically update trustworthy and digitally structured evidence summaries, guidelines and decision aids in user-friendly formats on all devices. Such digital multi-layered formats allow different users to rapidly find recommendations, while having the supporting evidence for them one click away [ 41 ]. MAGICapp, used by WHO, NICE and professional societies across the world, holds potential to enhance the impact of evidence-informed guideline recommendations in practice, in an enhanced evidence ecosystem [ 9 ]. However, the usability of the MAGICapp in sub-Saharan Africa, based on local user preferences for different communication formats, are key research questions.

Against this backdrop, the Global Evidence, Local Adaptation (GELA) project will maximize the impact of research on poverty-related diseases through enhancing researchers and decision makers’ capacity to use global research to develop locally relevant guidelines for newborn and child health in Malawi, Nigeria and South Africa. These guidelines will build on and add value to the large-scale programme of child health guideline development from agencies such as the WHO, to support adaptation and implementation led by national ministries in collaboration with WHO Afro regional office.

Brief overview of the GELA project aim, objectives and approach

The overarching aim of GELA is to bridge the gap between current processes and global advances in evidence-informed decision-making and guideline development, adaptation and dissemination by building skills and sharing resources in ways that can be sustained beyond the project period. The project has seven linked and related work packages (WPs) to support delivery of the planned project deliverables. Table 1 provides a brief summary of the activities of each WP. This protocol outlines our approach for the monitoring and overall evaluation of the project activities and impact (WP 6).

The project will be implemented in three SSA countries: Malawi, South Africa and Nigeria over a 3-year period. The project adopts a multi-faceted multidisciplinary research and capacity strengthening programme using primary and secondary research, guideline adaptation methodology and digital platforms to support authoring delivery and dynamic adaptation. These processes will offer bespoke capacity strengthening opportunities for policy makers, researchers and civil society. Throughout the project, we plan for innovations in the tools we use, accompanied by comprehensive evaluation of all aspects of the research, research uptake into policy and capacity strengthening.

This current proposal is for WP6: monitoring and evaluation

Ongoing monitoring and evaluation of project processes and activities will help facilitate ongoing engagement between researchers and stakeholders throughout the research project. This will in turn help ensure that the project is centred on a common goal, with clear understandings of the different research activities and potential impact. This can also promote research uptake and enable researchers to make adjustments to project activities, maximizing stakeholder engagement and research utilization.

M&E aims & objectives

The overarching aim of the monitoring and evaluation work package is to monitor and evaluate the various GELA project activities and processes, including whether, how and why activities took place or if goals were met.

The specific monitoring and evaluation objectives are to:

Monitor ongoing engagement with local stakeholders across work packages and explore what worked and didn’t and why;

Assess the capacity development needs of guideline panels and steering group committees and explore their views and experiences of the project’s capacity development activities;

Explore guideline panelists’ experiences with reading and using evidence from reviews of qualitative research, including their preferences regarding how qualitative review findings are summarized and presented;

Evaluate guideline panelists’, steering group committees’ and project team members’ overall views and experiences of the project, including the what works or not, to influence evidence-informed decision-making and guideline adaptation processes

Overall approach

We will use a longitudinal, mixed-methods study design, informed by an overarching project Theory of Change (Table  2 ). The theoretical underpinning for the GELA project across all work packages is related to the three-layered behaviour change wheel comprising opportunity, capability and motivation [ 42 ]. The design, delivery and implementation of multi-stakeholder integrated activities based on identified priority areas and needs is expected to lead to guideline related improved capacity, practice and policy within each country’s health system. Certain objectives also have specific underpinning theoretical frameworks, in addition to the overarching project Theory of Change, which are explained under the respective objectives below. A series of interconnected qualitative and quantitative data collections methods will be used to address each objective.

In what follows, we describe each objective and the methods we will use to achieve it, separately. However, in many cases the qualitative data collection cuts across objectives, with the same interviews and observations being used to explore multiple issues simultaneously (e.g. knowledge translation, capacity, overall views and experiences of the project, etc.). The relationship between the different objectives and associated methods are depicted in Tables 3 and 4 . Table 3 outlines the stakeholder groups included in the monitoring and evaluation work package, including their composition and for which objectives they are targeted. Table 4 provides the timeline for the different data collection methods and how they relate to each across the objectives.

1. Objective 1: monitor ongoing engagement with local stakeholders across work packages and explore what worked and did not work and why

Overall approach for this objective.

This objective will be guided by an integrated knowledge translation (IKT) approach. IKT focuses on the important role of stakeholder engagement in enhancing evidence-informed decision-making [ 43 ]. As part of work package 4 (‘dissemination and communication’), knowledge translation (KT) champions have been identified in each of the three countries and will work together to develop and implement country-level KT strategies. This will include defining KT objectives, identifying and mapping relevant stakeholders, prioritizing those we will actively engage and developing a strategy for engaging each priority stakeholder. We will monitor these engagements through the development and implementation of a tracking sheet, qualitative case studies and semi-structured interviews.

Participants

Participants will comprise of knowledge translation (KT) champions and relevant country-level stakeholders. KT champions are GELA project staff who have dedicated time to work on the communication, dissemination and engagement aspects at a country-level. At least one KT champion has been identified for each of Malawi, Nigeria and South Africa.

Relevant country-level stakeholders will be identified as part of the KT strategy development (WP4) and will comprise any health decision-makers, e.g. health practitioners, community groups, health system managers, policy-makers, researchers and media.

Tracking sheet and qualitative case studies

A tracking sheet will be used to capture information for each stakeholder related to the purpose, message, medium or forum, messenger, timing and resources for engagement. KT champions in each country will be responsible for tracking these details on a continuous basis, and the tracking sheet will be monitored bi-monthly at a meeting with KT champions from the three country teams. This will help us monitor whether and how engagement activities are taking place, as well as the strategies for implementation. The tracking sheets will consist of different in-country stakeholders (e.g. government officers, health professional associations, researchers, media, etc.), and there may be several goals for engaging each individual stakeholder. The engagement strategy will be reviewed and updated as priority stakeholders change over the research stages and project period. As such, the sample size will be determined iteratively.

We will analyse information with descriptive statistics. For example, we will group and count by categories: number and type of stakeholders, type of engagement activities, type of KT products produced, type of forum or medium used for dissemination, frequency and duration of engagement, follow-ups, intensive engagement period and resources required for engagement.

We will also develop case stories (or impact stories) describing engagement activities and processes between project staff and relevant stakeholders. The case studies will help us monitor successful engagement, disseminate best practice scenarios and draw out lessons for future engagements. We will identify case stories through the tracking sheet and at bi-monthly meetings with the KT co-ordinator, where KT champions will be asked to share success stories or learning moments. KT champions will not know which ‘case’ will be selected for the case study in advance. The information will be collected by the KT co-ordinator, who is not involved in any of the country strategy implementation. The information collected from the KT champions (and messenger, if the messenger is not the KT champion) will be via a standard case story template, including aim of engagement, what the engagement was, experiences from both sides (quotes to be included in stories), success of engagement, lessons learnt and any future engagement plans. The number of cases will be determined iteratively. The intention is to develop one case story from each country annually, showcasing different cases, e.g. type of KT goal, type of stakeholder, type of KT medium/forum, etc.

Semi-structured interviews

At project close (month 30), we will conduct semi-structured interviews to explore if, why and how project KT goals were met and what planned stakeholder engagements worked (and did not work) and why. The interviews will be conducted with KT champions, other messengers (e.g. communication officers), country leads and selected stakeholders. At least two people from each county (KT champion and messenger and/or stakeholder) will be interviewed, and so there will be six to eight interviews in total. Participants will be selected purposively for information-rich cases that can help yield insights and in-depth understanding of the nature and success (or not) of our stakeholder engagements [ 44 ].

These interviews will form part of the interviews conducted with project team members more broadly as part of objective 4, the methods of which are therefore described in more detail below.

2. Objective 2: assess the capacity development needs of guideline panels and steering group committees and explore their views and experiences of the project’s capacity development activities.

Overarching theoretical lens.

We will draw on the Kirkpatrick model [ 45 ] as the underpinning theoretical framework for this objective. This model evaluates training effectiveness across four levels: (1) reaction, (2) learning, (3) behaviour and (4) results. The ‘reaction level’ assesses the degree of satisfaction of participants with the training event. The ‘learning level’ examines learning among participants both before and after the training event to determine any change in knowledge [ 46 , 47 ]. The ‘behaviour level’ assesses whether the training event has provided any favourable change in behaviour among participants. The final ‘results level’ assesses the use of knowledge gained through the training event within the workplace [ 46 , 47 ].

To assess the potential difference that project capacity development activities make, the outcomes of interest will be those related to training in evidence-based healthcare (EBHC). An overview of systematic reviews by Young and colleagues identified that EBHC training often aims to ‘improve critical appraisal skills and integration of results into decisions, and improved knowledge, skills, attitudes and behaviour among practising health professionals’ [ 48 , 49 ].

We will employ mixed methods to achieve this objective, including three rounds of online surveys (at baseline, mid-line and at the project close) as well as semi-structured interviews (at project close) and non-participant observations of meetings (various). The first online survey at baseline will assess the capacity needs of the guideline panels and steering group committees in South Africa, Malawi and Nigeria, and the two subsequent online surveys will assess the potential difference project capacity development activities make on these groups across all the four levels of the Kirkpatrick model, i.e. reaction, learning, behaviour and results. The capacity needs and progress of these groups will also be explored qualitatively through semi-structured interviews and observations of meetings.

Details of the project capacity development activities that will be implemented as part of work package 5 (‘capacity strengthening and sharing’) of the GELA project are outlined in Table  1 (above). All members of the guideline panels and steering group committees in South Africa, Malawi and Nigeria will be invited and encouraged to attend all project capacity development activities. ‘On the job’ capacity building will also take place during the various meetings convened with these groups, as they are supported to identify priority topics, to appraise and discuss the evidence used to inform the recommendations and to formulate the final recommendations.

Participants will comprise members of the guideline panels and steering group committees in South Africa, Malawi and Nigeria. Table 3 (above) provides details of the composition of the guideline panels and steering group committees.

Online surveys

Procedures and data collection tools.

At baseline (at approximately 6 months before engagement in any project training activities), at mid-line (month 18) and at the project close (month 30), all members of the guideline panels and steering group committees in South Africa, Malawi and Nigeria will be invited, via email, to participate in a survey. In each of the three countries the guideline development group and steering group committees will include approximately 20 and 10 members, respectively; we will therefore aim to have 90 participants in total complete the survey. The email invitation to all three survey rounds will inform participants about the nature of the study and direct them to an online survey. The landing page of the survey will provide information about the purpose of the research project and what is being requested from the participants, with a consent statement at the end which the participant will be required to agree to before being able to continue with the survey. Data will only be collected from participants who consent to freely participate in the study. The survey will be carried out using a secure online survey platform (such as Microsoft Forms) where all cookies and IP address collectors will be disabled to protect the confidentiality of the participants and to avoid tracking of the participant activities online. Unique identifiers (last six numbers of their ID) will be used to track participants responses over time and link data from baseline to project close.

The baseline survey will be a short (10–25 min) form that will ask participants about their capacity needs and knowledge/skills in evidence-based healthcare (EBHC) and decision-making. The survey will capture demographic variables of participants at baseline, mid-term and at the end of the project. It will assess the training needs of participants at baseline, participants’ satisfaction at the end of each training activity, the knowledge and skills at baseline, mid-term and at the end of the project. Participants’ behaviour will also be assessed using open-ended questions and vignettes. The surveys will focus on all four levels (i.e. reaction, learning, behaviour and results) of the Kirkpatrick model.

Data management and analysis

All data collected on the secure online survey platform will be coded, cleaned and entered into STATA. Data collected for the baseline survey will be analysed using descriptive statistics to determine the frequency of the various training needs and qualitative data gathered using the open-ended questions will be analysed thematically using manual coding (or if available and dataset is large), and NVivo or a similar tool will be used to identify the recurring themes which emerge in the data collected about the key training needs of participants.

Data collected for the surveys conducted at midpoint and at project close will be analysed using descriptive statistics to determine if there has been a change in the learning, knowledge gained and behaviours over time, as well as the extent of the potential application of evidence-based practice, while the data collected using the open-ended questions will be analysed using thematic analysis outlining how project capacity development activities informed particular outcomes and results in the participant’s workplace. To determine change in skills (and trends over time such as confidence improvement or decay), the descriptive statistics will be supplemented by appropriate inferential statistics for repeated measures (paired data) such as McNemar or paired t -tests, reporting change in percentages as mean differences (such as self-reported confidence) with 95% confidence intervals or/and frequencies. Descriptive trends over time will also be presented graphically using line graphs or other visual aids as appropriate. However, these will be interpreted with caution as the primary analysis is descriptive. Statistical significance will be set at a p value of 0.05.

At project close (month 30), we will conduct semi-structured interviews with a sample of members from the guideline panels and steering group committees in South Africa, Malawi and Nigeria. Sampling will be purposive, with the aim of understanding the broad range of needs, experiences and perspectives and ensuring that the sample reflects a range of socio-demographic characteristics and stakeholder categories. We will begin with a sample size of 10–15 participants in each country; however, sampling will continue if we have not reached saturation of the data through the initial sample size [ 44 ].

Participants will be contacted, either by telephone or via email, and invited to participate in an interview. Interviews will be conducted face-to-face or electronically (e.g. using Microsoft Teams) at a date and time chosen by participants. Face-to-face interviews will take place at a location convenient to participants, which is conducive to a confidential exchange. The interviews will last between 45 and 60 min and will be conducted by researchers trained in qualitative research methodologies and interviewing techniques. The interviews will be guided by a semi-structured topic guide and will include questions informed by the four levels (i.e. reaction, learning, behaviour and results) of the Kirkpatrick model. Specifically, the questions will explore participants’ views and experiences regarding their capacity development needs and expectations of the project; whether and why these expectations were met (or not), the project capacity development activities, what they learned (or not) from these activities and what impact participants believe they have had (or may have) on their practices.

Verbal and written information about the study will be provided to all participants taking part in interviews. Written informed consent will be obtained from all participants before proceeding with the interview. With the permission of participants, all interviews will be digitally recorded.

Non-participant observations

We will conduct non-participant observations of guideline panel and steering group committee meetings. Observational methods can provide useful data on what people do, how they interact with each other and how they engage with particular artefacts in situ (rather than their accounts of these) [ 50 ]. The steering group committees in each country will meet approximately twice over the project duration (with the option for additional meetings): an initial meeting for project orientation (month 2/3) and again to identify priority topics and guideline gaps (month 6). Guideline panels in each country will meet approximately three times over the project duration (with the option for additional meetings): an initial meeting for project orientation and outcome prioritization (month 6/7), another potential meeting if necessary to finalize outcome prioritization and a final meeting to draft recommendations for the guideline (months 17–20). Meetings for both groups will be held virtually or in person, informed by preferences of the committee.

With the exception of the initial steering group committee (month 2/3), at least one researcher will be present to observe guideline panel and steering group committee meetings. The observer will aim to identify any capacity-related needs, expectations, gaps, strengths, achievements and challenges and the contexts in which these occur. He or she will also pay particular attention to group dynamics and the interactions between members and different stakeholder groups, and the potential impact of these on capacity-related issues. Observations will be informed by Lofland’s [ 51 ] criteria for organizing analytical observations (acts, activities, meanings, participation, relationships and settings). The observer will take detailed observational notes. With consent of the attendees, all meetings will also be digitally recorded. The recordings will be used to identify further issues not identified and to deepen or clarify issues noted, through the real-time observations of verbal engagements.

Data management and analysis: semi-structured interviews and observations

Interview and meeting recordings will be transcribed verbatim, and all personal identifying information will be removed from transcripts. The anonymized transcripts, together with observational notes, will be downloaded into Nvivo, a software programme that aids with the management and analysis of qualitative data. Analysis of the qualitative data will proceed in several rounds. First, as with all qualitative data analysis, an ongoing process of iterative analysis of the data will be conducted throughout the data collection period. Second, we will use a thematic analysis approach, using the phases described by Braun and Clarke [ 52 ], to identify key themes pertaining to participants’ capacity development needs and expectations and whether, how and why project capacity development activities met (or not) these needs and expectations. Finally, findings from the surveys (as described above) will also be integrated with the findings from the thematic analysis using a ‘narrative synthesis’ approach, a technique recommended by the Cochrane Collaboration as a way of synthesizing diverse forms of qualitative and quantitative evidence in mixed methods studies [ 53 , 54 ]. This approach will allow for both robust triangulation, and a more comprehensive interpretation of the difference project capacity development activities may have made on the guideline panels and steering group committees.

3. Objective 3: explore guideline panelists’ experiences with reading and using evidence from reviews of qualitative research, including their preferences regarding how qualitative review findings are summarized and presented.

Objective 3 of the monitoring and evaluation stakeholder matrix work package explores how guideline panels view and experience evidence from the review(s) of qualitative research, including how it is summarized and presented. Here, we will employ a user testing approach, drawing on the methods and guidance of the SURE user test package 2022 developed by Cochrane Norway ( https://www.cochrane.no/our-user-test-package ) and which has been used to test various evidence-related products [ 55 , 56 , 57 , 58 ]. User testing involves observing people as they engage with a particular product and listening to them ‘think-aloud’. The goal is to gain an understanding of users’ views and experiences, the problems they face and to obtain suggestions for how a product may be improved [ 55 , 56 , 57 , 58 ].

We will begin by identifying or preparing relevant reviews of qualitative research. We will then develop review summary formats and explore guideline panel members’ views and experiences of these formats. We will revise the formats in multiple iterative cycles.

Identifying or preparing relevant reviews of qualitative research

As part of WP2 of the project (‘evidence synthesis’), we will identify relevant review(s) of qualitative research, including reviews exploring how people affected by the interventions of interest value different outcomes, the acceptability and feasibility of the intervention and potential equity, gender and human rights implications of the intervention. These reviews need to be assessed as sufficiently recent and of a sufficient quality. They also need to have applied GRADE-CERQual assessments to the review findings. Where necessary, we will update existing reviews or prepare reviews ourselves.

Developing the review summaries

In WP3 of the project (‘decision-making’) the evidence from these reviews will be provided to guideline panels as part of the evidence-to-decision (‘EtD’) frameworks that will inform the recommendations they develop (see Table  1 for further details about project work packages 2 and 3). Our next step will therefore be to prepare summaries of the reviews in a format that can easily be included in the EtD frameworks.

Each summary needs to present review findings that are relevant to specific parts of the EtD framework (typically the ‘values’, ‘acceptability’, ‘feasibility’ and ‘equity’ components). It also needs to include information about our confidence in these findings. Finally, the summary needs to indicate where this evidence comes from and to allow guideline panels to move from the summary to more detailed information about the evidence.

Most of this information is found in the review’s Summary of Qualitative Findings tables. However, these tables are usually too large for EtD frameworks and are not tailored to each framework component. We will, therefore, start by creating new summaries, using a format that we have previously used in EtD frameworks [ 59 , 60 , 61 ] but that we have not user tested. As opposed to the Summary of Qualitative Findings tables, where each finding and our confidence in the finding, is presented individually in separate rows, this format involves pulling the findings and confidence assessments together in short, narrative paragraphs.

User testing the summary format

For our first set of user tests, we will observe guideline panels participating in the CPG panel simulation workshops. For our second round of user tests, we will observe how the guideline panels experience and interact with this qualitative evidence during the real guideline processes. Third, we will then test a potentially refined format with a selection of guideline panel members using a semi-structured interview guide. Finally, at the end of the project, we will conduct semi-structured interviews with a selection of guideline panel members to explore their broader views and experiences of interpreting and using evidence from reviews of qualitative studies in their deliberation processes. Figure  1 provides a visual depiction of this iterative process.

Iterative approach for user testing evidence from reviews of qualitative research

We will draw on the adapted version of Peter Morville’s original honeycomb model of user experience [ 62 ] as the underpinning theoretical framework for this objective [ 63 ] (Fig.  1 ). This adapted version extends and revises the meaning of the facets of user experience depicted in the original model. It includes eight facets: accessibility, findability, usefulness, usability, understandability, credibility, desirability and affiliation. Accessibility involves whether there are physical barriers to gaining access; findability is about whether the person can locate the product or the content that they are looking for; usefulness is about whether the product has practical value for the person; usability comprises how easy and satisfying the product is to use; understandability is about whether the person comprehends correctly both what kind of product it is and the content of the product (and includes both user's subjective perception of her own understanding and an objective measure of actual/correct understanding); credibility comprises whether the product/content is experienced as trustworthy; desirability is about whether the product is something the person wants and has a positive emotional response to it; affiliation involves whether the person identifies with the product, on a personal or a social level, or whether it is alienating and experienced as being not designed for ‘someone like me’. The adapted model also adds to the original model a dimension of user experience over time, capturing the chronological and contingent nature of the different facets.

Participants will comprise members of the guideline panels in South Africa, Malawi and Nigeria. Table 3 (above) provides details of the composition of the guideline panels.

Non-participant observations: guideline panel simulation workshops and guideline panel meetings

We will conduct non-participant observations of the CPG panel simulation workshops and the subsequent guideline panel meetings for developing the recommendations. The CPG panel simulation workshops will run a simulation of a real guideline process and give guideline panels an opportunity to understand how the guideline process works before they participate in real panel meetings. The guideline panels in all three countries will be invited and encouraged to attend these workshops, which will form part of the project capacity development activities of WP5 (Table  1 ).

With the participants’ consent, both the simulation workshops and meetings will be digitally recorded and at least two observers will observe and take notes. The observations will focus on how guideline panel members refer to and interact with the summaries of qualitative evidence. Drawing on a user testing approach ( https://www.cochrane.no/our-user-test-package ), we will also look specifically for both problems and facilitators in the way the qualitative evidence is formatted, including ‘show-stoppers’ (the problem is so serious that it hindered participants from correct understanding or from moving forward), ‘big problems/frustrations’ (participants were confused or found something difficult but managed to figure it out or find a way around the problem eventually), ‘minor issues/cosmetic things’ (small irritations, frustrations and small problems that do not have serious consequences, as well as likes/dislikes), ‘positive/negative feedback’, ‘specific suggestions’, ‘preferences’ and any other ‘notable observations’, e.g. feelings of ‘uncertainty’.

Structured user testing interviews

Based on the insights gained from the non-participant observations (above), we may make changes or refinements to our original summary format (Fig.  1 ). Once the guideline panel meetings have concluded (approximately by month 20), we will then conduct structured user testing interviews to test the potentially refined summary format. These interviews will be conducted with a sample of members from the guideline panels in South Africa, Malawi and Nigeria. Sampling will be purposive, with the aim of understanding the broad range of experiences and perspectives and ensuring the sample reflects a range of socio-demographic characteristics and stakeholder categories. As recommended ( https://www.cochrane.no/our-user-test-package ), we will begin with a sample size of six to eight participants in each country; however, sampling will continue until saturation is achieved [ 44 ].

Participants will be contacted, either telephonically or via email, and invited to participate in an interview. Interviews will be conducted face-to-face or electronically (e.g. using Skype or Teams) at a date and time chosen by participants. Face-to-face interviews will take place at a location convenient to participants, which is conducive to a confidential exchange. In line with the SURE user test package 2022 guidance, the interviews will last approximately 60 min ( https://www.cochrane.no/our-user-test-package ). They will be facilitated by a test leader, who will accompanied by at least one observer who will take notes. Both the test leader and observer(s) will be trained in user testing interviewing methodology and techniques. Verbal and written information about the study will be provided to all participants taking part in interviews. Written informed consent will be obtained from all participants before proceeding with the interview. With the permission of participants, all interviews will be video recorded.

For these interviews we will show panel members the latest version of the format, explore immediate first impressions, and then opinions about different elements of the summary. We may also show panel members different formats where we think this may be helpful. We will use a structured interview guide which draws heavily on other interview guides that been developed to user test evidence-related products [ 55 , 56 , 57 , 58 ]. It will include questions related the participant’s background; their immediate first impressions of the summary format(s); in-depth walk-through of the summary format(s), with prompts to think aloud what they are looking at, thinking, doing and feeling; and suggestions for improving the way the summary is formatted and for improving the user testing itself. We may ask follow-up questions to specific issues we observed in the simulation workshops and guideline panel meetings and/or create scenarios that resemble issues we observed in the workshops/meetings. This will be decided upon based on the findings that emerge from these workshops/meetings. The guide will be finalized once the relevant qualitative evidence (from WP2) has been produced and we have gained insights from the workshops and meetings.

As with the non-participant observations of meetings and workshops, throughout the interview, the observers will make notes about the participant’s experience as heard, observed and understood. Drawing on a user testing approach, they will look specifically for both problems and facilitators, specific suggestions, preferences and any other notable observations (as described above under ‘non-participant observations’).

At project close (month 30), we will also conduct semi-structured interviews with a sample of members from the guideline panels in South Africa, Malawi and Nigeria. These will be the same interviews with guideline panel members as described in objective 2. In addition to exploring participants’ capacity development needs, expectations and achievements, the semi-structured topic guide will also explore their views and experiences of (and specific capacity in) interpreting and using evidence from reviews of qualitative studies in guideline processes. More specifically, questions will investigate participants’ familiarity/experience with qualitative evidence; their perceptions of different types of evidence, what constitutes qualitative evidence and the role of qualitative evidence in guideline processes; and their experiences of using the qualitative evidence in their deliberations as part of the project, including what influenced its use and whether they found it useful. Details pertaining to sampling, data collection procedures and collection tools are described in objective 2.

All interview and meeting recordings will be transcribed verbatim, and all personal identifying information will be removed from transcripts. The anonymized transcripts, together with observational notes (from the workshops, meetings and interviews), will be downloaded into a software programme that aids with the management and analysis of qualitative data. Analysis of the data will be guided by the user testing analysis methods described in the SURE user test package 2022 ( https://www.cochrane.no/our-user-test-package ). The analysis will proceed in several, iterative rounds to develop and revise the summary format and to inform the focus of subsequent data collection. After each user test, we will review our notes, first separately and then together. In line with the SURE user test package 2022 guidance, we will look primarily for barriers and facilitators related to correct interpretation of the summary’s contents, ease of use and favourable reception, drawing on the facets of the revised honeycomb model of user experience (Fig.  2 ). We will trace findings back to specific elements or characteristics of the summaries that appeared to facilitate or hinder problems. Before the next set of user tests, we will discuss possible changes that could address any identified barriers and make changes to the summary format.

Adapted version of Peter Morville’s honeycomb model of user experience

4. Objective 4: evaluate guideline panelists’, steering group committees’ and project team members’ overall views and experiences of the project, including what works or not, to influence evidence-informed decision-making and guideline adaptation processes.

This objective explores overall views and experiences of the project, with a focus on guideline panelists, steering group committees and project team members. Specifically, it seeks to gain an understanding of these three stakeholder groups’ more general views and experiences of the project activities they were involved with and whether, why and how these activities may influence (or not) evidence-informed decision-making and guideline adaptation processes. This will be achieved through semi-structured interviews.

Participants will comprise members of the guideline panels and steering group committees in South Africa, Malawi and Nigeria, as well as members of the project team (as described in Table  3 above).

At project close (month 30), we will conduct semi-structured interviews with a sample of members from the guideline panels and steering group committees in South Africa, Malawi and Nigeria. These will be the same interviews and participants as described in objective 2. In addition to exploring issues around capacity development and qualitative evidence, the interviews will also investigate participants’ views and experiences of the various project activities they were involved with, and whether, why and how these activities may influence (or not) evidence-informed decision-making and guideline adaptation processes. Details pertaining to sampling, data collection procedures and collection tools are described in objective 2.

At project close (month 30), we will also conduct semi-structured interviews with members of the project team (see Table  3 for details of project team composition). We will begin by interviewing all project management team members, WP leads and KT champions. Additional participants will be determined iteratively (depending on what emerges from initial interviews) and purposively, with the aim of understanding the broad range of experiences and perspectives and ensuring the sample reflects the various groups which make up the project team. Interviews will be conducted face-to-face or electronically (e.g. using Skype or Teams) at a date and time chosen by the interviewee. The interviews will last between 45 and 60 min and will be guided by a semi-structured topic guide. The questions will explore participants’ views and experiences of the respective work packages in which they were involved, including what the primary goals of the work package were; if, why and how these goals were met; and what worked and what did not work and why.

The same qualitative data analysis procedures and methods will be used as described in objective 2. For this objective, the thematic analysis will identify key themes pertaining to views and experiences of project activities, including what worked (or not) and why, whether, why and how the project may (or not) influence evidence-informed decision-making and guideline development, adaptation and dissemination processes in South Africa, Malawi and Nigeria and potential barriers and facilitators to the sustainability of this influence.

Evidence-based guideline development is a multi-stakeholder, multi-perspective, complex set of tasks. There is limited, if any, research that has followed these steps from the perspectives of policymakers or researchers from start to end. The GELA project protocol sets out to monitor and evaluate various key steps in the process, using in-depth qualitative methods alongside appropriate surveys not only to inform the project as it progresses but also to understand the overall impact of all steps on development of transparent and contextually-rich guideline recommendations. Following WHO’s guideline steps, the tasks range from scoping stakeholder-informed priority topics to conducting relevant data gathering and evidence synthesis, followed by guideline panel meetings to reach consensus decisions and finally to produce recommendations that can be useful to end-users and improve health and care outcomes. The GELA project is undertaking a 3-year project to conduct these tasks in the context of newborn and child health priorities. We are doing this in collaboration with national ministries of health, academics, non-governmental partners and civil society groups in Malawi, Nigeria and South Africa. Overall, we aim build capacity across all collaborators for evidence-informed guideline development, while producing fit for context guideline recommendations, in accessible formats that benefit children, caregivers and health care providers.

As such, this is a practical research project, in that the products should directly impact care decisions at the national level but with the added benefit of being able to learn about what works or does not work for collaborative guideline development in country. We will also be applying emergent guideline adaptation methods to explore reducing duplication of expensive guideline development efforts in our lower resource settings. Our project addresses newborn and child health, keeping this most vulnerable population in our focus, hoping that producing sound evidence-based recommendations has the potential to impact care.

Through some of our formative work, we have completed a landscape analysis identifying and describing all available newborn and child health guidelines in each of the partner countries. In all countries there were similar findings, (1) there is no easy access to guidelines for end-users, thus locating a guideline requires effort and screening through multiple sources; (2) considering national priority conditions in this age group, there were often gaps in available current guidelines for managing children; and (3) when we appraised the guidelines using the global standard, AGREE II tool, we found that the reporting of guideline methods were poor, leaving it uncertain whether the recommendations were credible or whether any influences or interests had determined the direction of a recommendation. Finally, we expected to find many adapted guidelines, based on WHO or UNICEF or similar guidance available globally; however, very few of the identified guidelines stated clearly whether they had been adapted from other sources and, if so, which recommendations were adopted and which adapted.

Given progress globally in methods for guideline development, the continued poor reporting on guideline methods at the country level speak to a breakdown in skills-sharing globally, for example, WHO produces guidelines that are recognized as rigorous and follow good practice and reporting, but the same standards are not supported in country. Overall, GELA aims to address these key gaps in national guideline approaches for adaptation, but we need to recognize that this will be a long term process and that we need to learn from each other about what works and what may not serve us. Therefore, this protocol outlines our approach for monitoring several aspects of the project in our efforts to move closer to trustworthy and credible guidelines that all can use and trust for countries like ours.

Availability of data and materials

Not applicable.

Abbreviations

Poverty-related diseases

Sub-Saharan Africa

• Clinical practice guidelines

Evidence-informed decision-making

Evidence-based healthcare

Global Evidence Local Adaptation

Knowledge translation

Levels and trends in child mortality: report 2020. Estimates developed by UNICEF. ISBN: 978-92-806-5147-8. https://www.who.int/publications/m/item/levels-and-trends-in-child-mortality-report-2020 . United Nations Children’s Fund. Accessed Aug 2024.

Liu L, Oza S, Hogan D, et al. Global, regional, and national causes of under-5 mortality in 2000–15: an updated systematic analysis with implications for the sustainable development goals. Lancet. 2016;388(10063):3027–35.

Article   PubMed   PubMed Central   Google Scholar  

Under-five mortality. 2021 https://data.unicef.org/topic/child-survival/under-five-mortality/ . United Nations International Children’s Fund. Accessed Aug 2024.

Stewart R, El-Harakeh A, Cherian SA. Evidence synthesis communities in low-income and middle-income countries and the COVID-19 response. Lancet. 2020;396(10262):1539–41.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Mijumbi RM, Oxman AD, Panisset U, Sewankambo NK. Feasibility of a rapid response mechanism to meet policymakers’ urgent needs for research evidence about health systems in a low income country: a case study. Implementation Sci. 2014;9:114.

Article   Google Scholar  

Stewart R, Dayal H, Langer L, van Rooyen C. The evidence ecosystem in South Africa: growing resilience and institutionalisation of evidence use. Palgrave Commun. 2019;5(1):90.

Uneke CJ, Ezeoha AE, Ndukwe CD, Oyibo PG, Onwe F. Promotion of evidence-informed health policymaking in Nigeria: bridging the gap between researchers and policymakers. Glob Public Health. 2012;7(7):750–65.

Article   PubMed   Google Scholar  

Young T, Garner P, Clarke M, Volmink J. Series: clinical epidemiology in South Africa Paper 1: evidence-based health care and policy in Africa.: past, present, and future. J Clin Epidemiol. 2017;83:24–30.

Vandvik PO, Brandt L. Future of evidence ecosystem series: evidence ecosystems and learning health systems: why bother? J Clin Epidemiol. 2020;123:166–70.

Graham R, Mancher M, Wolman D, Greenfield S, Steinberg E. Clinical practice guidelines we can trust. Washington, DC: The National Academy Press; 2011.

Book   Google Scholar  

Kredo T, Bernhardsson S, Machingaidze S, et al. Guide to clinical practice guidelines: the current state of play. Int J Qual Health Care. 2016;28(1):122–8.

McCaul M, Tovey D, Young T, et al. Resources supporting trustworthy, rapid and equitable evidence synthesis and guideline development: results from the COVID-19 evidence network to support decision-making (COVID-END). J Clin Epidemiol. 2022;151:88–95.

Tricco AC, Garritty CM, Boulos L, et al. Rapid review methods more challenging during COVID-19: commentary with a focus on 8 knowledge synthesis steps. J Clin Epidemiol. 2020;126:177–83.

World Health Organization. WHO handbook for guideline development. Geneva: World Health Organization; 2011.

Google Scholar  

Schünemann HJ, Wiercioch W, Etxeandia I, et al. Guidelines 2.0: systematic development of a comprehensive checklist for a successful guideline enterprise. CMAJ. 2014;186(3):E123–42.

Kredo T, Gerritsen A, van Heerden J, Conway S, Siegfried N. Clinical practice guidelines within the Southern African development community: a descriptive study of the quality of guideline development and concordance with best evidence for five priority diseases. Health Res Policy Syst. 2012;10:1.

Malherbe P, Smit P, Sharma K, McCaul M. Guidance we can trust? The status and quality of prehospital clinical guidance in sub-Saharan Africa: a scoping review. African J Emerge Med. 2021;11(1):79–86.

McCaul M, Clarke M, Bruijns SR, et al. Global emergency care clinical practice guidelines: a landscape analysis. African J Emerge Med. 2018;8(4):158–63.

Dizon JM, Grimmer K, Louw Q, Kredo T, Young T, Machingaidze S. South African guidelines excellence (SAGE): adopt, adapt, or contextualise? South African Med J. 2016;106(12):1177–8.

Article   CAS   Google Scholar  

Schünemann HJ, Wiercioch W, Brozek J, et al. GRADE evidence to Decision (EtD) frameworks for adoption, adaptation, and de novo development of trustworthy recommendations: GRADE-ADOLOPMENT. J Clin Epidemiol. 2017;81:101–10.

ADAPTE Collaboration. 2009 The ADAPTE process: Resource toolkit for guideline adaptation, version 2. http://www.g-i-n.net/ . Accessed Aug 2024.

Barreix M, Lawrie TA, Kidula N, et al. Development of the WHO antenatal care recommendations adaptation toolkit: a standardised approach for countries. Health Res Policy Syst. 2020;18(1):70.

Ioannidis JP. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q. 2016;94(3):485–514.

Opiyo N, Shepperd S, Musila N, English M, Fretheim A. The, “child health evidence week” and grade grid may aid transparency in the deliberative process of guideline development. J Clin Epidemiol. 2012;65(9):962–9.

Wang Z, Norris SL, Bero L. The advantages and limitations of guideline adaptation frameworks. Implemen Sci. 2018;13(1):72.

Langlois EV, Tunçalp Ö, Norris SL, Askew I, Ghaffar A. Qualitative evidence to improve guidelines and health decision-making. Bull World Health Organ. 2018. https://doi.org/10.2471/BLT.17.206540 .

Carmona C, Baxter S, Carroll C. Systematic review of the methodological literature for integrating qualitative evidence syntheses into health guideline development. Res Synth Methods. 2021;12(4):491–505.

Guindo LA, Wagner M, Baltussen R, et al. From efficacy to equity: Literature review of decision criteria for resource allocation and healthcare decisionmaking. Cost Eff Res Alloc. 2012;10(1):9.

Verkerk K, Van Veenendaal H, Severens JL, Hendriks EJ, Burgers JS. Considered judgement in evidence-based guideline development. Int J Qual Health Care. 2006;18(5):365–9.

Alonso-Coello P, Schunemann HJ, Moberg J, et al. GRADE Evidence to Decision (EtD) frameworks: a systematic and transparent approach to making well informed healthcare choices. 1: Introduction. BMJ. 2016. https://doi.org/10.1136/bmj.i2016 .

Glenton C, Lewin S, Gülmezoglu AM. Expanding the evidence base for global recommendations on health systems: strengths and challenges of the OptimizeMNH guidance process. Implement Sci. 2016;11:98.

Lewin S, Glenton C. Are we entering a new era for qualitative research? using qualitative evidence to support guidance and guideline development by the World Health Organization. Int J Equity Health. 2018;17(1):126.

Noyes J, Booth A, Cargo M, Flemming K, Harden A, Harris J, Garside R, Hannes K, Pantoja T, Thomas J. Chapter 21: Qualitative evidence. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.4 (updated August 2023). Cochrane, 2023. Available from www.training.cochrane.org/handbook . Accessed Aug 2024.

Glenton C, Lewin S, Norris S. Using evidence from qualitative research to develop WHO guidelines (Chapter 15) World Health Organization Handbook for Guideline Development. 2nd ed. WHO: Geneva; 2016.

World Health Organisation. WHO recommendations: intrapartum care for a positive childbirth experience. Geneva: WHO; 2018.

Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol. 2010;63(6):620–6.

Lewin S, Bohren M, Rashidian A, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings-paper 2: how to make an overall CERQual assessment of confidence and create a summary of qualitative findings table. Implement Sci. 2018;13(Suppl 1):10.

Treweek S, Oxman AD, Alderson P, et al. Developing and evaluating communication strategies to support informed decisions and practice based on evidence (DECIDE): protocol and preliminary results. Implement Sci. 2013;8:6.

Brandt L, Vandvik PO, Alonso-Coello P, et al. Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial. BMJ Open. 2017;7(2): e011569.

Kredo T, Cooper S, Abrams AL, et al. ’Building on shaky ground’-challenges to and solutions for primary care guideline implementation in four provinces in South Africa: a qualitative study. BMJ Open. 2020;10(5): e031468.

Vandvik PO, Brandt L, Alonso-Coello P, et al. Creating clinical practice guidelines we can trust, use, and share: a new era is imminent. Chest. 2013;144(2):381–9.

Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42.

Jessani NS, Rohwer A, Schmidt B-M, Delobelle P. Integrated knowledge translation to advance noncommunicable disease policy and practice in South Africa: application of the exploration, preparation, implementation, and sustainment (EPIS) framework. Health Res Policy Syst. 2021;19(1):82.

Francis JJ, Johnston M, Robertson C, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229–45.

Roos M, Kadmon M, Kirschfink M, et al. Developing medical educators–a mixed method evaluation of a teaching education program. Med Educ Online. 2014;19:23868.

Bijani M, Rostami K, Momennasab M, Yektatalab S. Evaluating the effectiveness of a continuing education program for prevention of occupational exposure to needle stick injuries in nursing staff based on Kirkpatrick’s model. J Natl Med Assoc. 2018;110(5):459–63.

PubMed   Google Scholar  

Carlfjord S, Roback K, Nilsen P. Five years’ experience of an annual course on implementation science: an evaluation among course participants. Implement Sci. 2017;12(1):101.

Young T, Rohwer A, Volmink J, Clarke M. What are the effects of teaching evidence-based health care (EBHC)? Overview of systematic reviews. PLoS ONE. 2014;9(1): e86706.

Young T, Dizon J, Kredo T, et al. Enhancing capacity for clinical practice guidelines in South Africa. Pan Afr Med J. 2020;36:18.

Green G, Thorogood N. Qualitative methods for health research. London: Sage; 2004.

Lofland J. Analyzing social settings. Belmont, CA: Wadsworth Publishing Company; 1971.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77–101.

Popay J, Roberts H, Sowden A. Guidance on the conduct of narrative synthesis in systematic reviews a product from the ESRC methods programme. I Health Research: Lancaster; 2006.

Pope C, Mays N, Popay J. How can we synthesize qualitative and quantitative evidence for healthcare policy-makers and managers? Healthc Manage Forum. 2006;19(1):27–31.

Glenton C, Santesso N, Rosenbaum S, et al. Presenting the results of Cochrane systematic reviews to a consumer audience: a qualitative study. Med Dec Making. 2010;30(5):566–77.

Rosenbaum SE, Glenton C, Cracknell J. User experiences of evidence-based online resources for health professionals: user testing of The Cochrane library. BMC Med Inform Decis Mak. 2008;8:34.

Rosenbaum SE, Glenton C, Nylund HK, Oxman AD. User testing and stakeholder feedback contributed to the development of understandable and useful Summary of Findings tables for Cochrane reviews. J Clin Epidemiol. 2010;63(6):607–19.

Rosenbaum SE, Glenton C, Wiysonge CS, et al. Evidence summaries tailored to health policy-makers in low- and middle-income countries. Bull World Health Organ. 2011;89(1):54–61.

Downe S, Finlayson KW, Lawrie TA, et al. Qualitative evidence synthesis (QES) for guidelines: Paper 1 - using qualitative evidence synthesis to inform guideline scope and develop qualitative findings statements. Health Res Policy Syst. 2019;17(1):76.

Lewin S, Glenton C, Lawrie TA, et al. Qualitative evidence synthesis (QES) for guidelines: Paper 2 - using qualitative evidence synthesis findings to inform evidence-to-decision frameworks and recommendations. Health Res Policy Syst. 2019;17(1):75.

Glenton C, Lewin S, Lawrie TA et al. Qualitative Evidence Synthesis (QES) for Guidelines: Paper 3 – Using qualitative evidence syntheses to develop implementation considerations and inform implementation processes. Health Res Policy Sys. 2019;17:74. https://doi.org/10.1186/s12961-019-0450-1

Morville P. 2004 User Experience Design [honeycomb model]. http://www.semanticstudios.com/publications/semantics/000029.php .

Rosenbaum S. 2010 Improving the user experience of evidence: A design approach to evidence-informed health care. PhD thesis. Oslo, Norway: The Oslo School of Architecture and Design.

Download references

Acknowledgements

We gratefully acknowledge the representatives from the National Ministries of Health in Nigeria, Malawi and South Africa for their support and partnership. We would also like to thank the appointed Steering Committees who have been providing input for the research project and guiding the prioritization of topics. We would also like to thank Joy Oliver and Michelle Galloway for their contribution an support of the project.

The GELA project is funded by EDCTP2 programme supported by the European Union (grant number RIA2020S-3303-GELA). The funding will cover all the activities for this Monitoring and Evaluation work package, including costs for personnel and publication of papers.

Author information

Authors and affiliations.

Health Systems Research Unit, South African Medical Research Council, Cape Town, South Africa

Tamara Kredo, Denny Mabetha, Bey-Marrié Schmidt & Simon Lewin

Division of Epidemiology and Biostatistics, Department of Global Health, Stellenbosch University, Cape Town, South Africa

Tamara Kredo, Anke Rohwer, Michael McCaul, Idriss Ibrahim Kallon, Taryn Young & Sara Cooper

Division of Clinical Pharmacology, Department of Medicine, Stellenbosch University, Cape Town, South Africa

Tamara Kredo

School of Public Health and Family Medicine, University of Cape Town, Cape Town, South Africa

Tamara Kredo & Sara Cooper

University of Calabar Teaching Hospital, Calabar, Nigeria

Emmanuel Effa

Kamuzu University of Health Science, Lilongwe, Malawi

Nyanyiwe Mbeye

School of Public Health, University of the Western Cape, Cape Town, South Africa

Bey-Marrié Schmidt

Western Norway University of Applied Sciences, Bergen, Norway

Susan Munabi-Babigumira & Claire Glenton

Norwegian University of Science and Technology, Trondheim, Norway

Simon Lewin

MAGIC Evidence Ecosystem Foundation, Oslo, Norway

Per Olav Vandvik

Department of Health Economics and Health Management, Institute for Health and Society, University of Oslo, Oslo, Norway

Cochrane South Africa, South African Medical Research Council, Cape Town, South Africa

Sara Cooper

You can also search for this author in PubMed   Google Scholar

Contributions

T.K., S.C., T.Y., S.L., C.G. and P.O.V. conceptualized the protocol idea and S.C. drafted the protocol with input from TK, D.M., A.R., B.M., M.M., I.I., C.G., T.Y., S.L. and P.O.V.; all authors approved the final version for submission for publication.

Corresponding author

Correspondence to Tamara Kredo .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval has been obtained in each partner country (South Africa, Malawi and Nigeria) from the respective Health Research Ethics Committees or Institutional Review Boards. Information about the project will be provided to, and consent obtained from, all participants completing the online surveys and interviews and all participants taking part in the meetings. The consent forms will make explicit the voluntary nature of participation, that there will be no negative consequences if they decide not to participate and in the case of the interviews and meetings observations will ask explicitly for permission for the interview or meeting to be recorded. The online surveys will ask participants to provide the last six numbers of their ID as a unique identifier to track their capacity development needs and progress throughout the project. To help protect their confidentiality, the information they provide will be private, deidentified and no names will be used. In addition, all cookies and IP address collectors will be disabled to ensure confidentiality. All interview and meeting recordings on the digital recorders will be destroyed following safe storage and transcription, and any identifying information will be redacted from all transcripts. All study data, including recordings, will be stored electronically using password-controlled software only accessible to key project members and project analysts. Reports of study findings will not identify individual participants. We do not anticipate any specific harms or serious risks to participants. However, there is a risk of breaches of confidentiality for participants who take part in guideline panel and steering group committee project meetings. At the start of all meetings, participants will be introduced to each other. The member names of these groups will not be anonymous as they will play an ongoing role in the GELA project. At the start of each meeting, we will discuss the importance of maintaining confidentiality by everyone. As part of guideline development processes, all guideline members will need to declare conflicts of interests and sign a confidentiality agreement. We will explain, however, that while the researchers undertake to maintain confidentiality, we cannot guarantee that other meeting participants will, and there is, thus, a risk of breaches of confidentiality. We will ensure participants are aware of this risk. Participants may also feel anxiety or distress expressing negative views about project activities. Where there is this potential and where participants identify concerns, we will reassure participants of the steps that will be taken to ensure confidentiality.

Consent for publication

Competing interests.

All authors declared no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Kredo, T., Effa, E., Mbeye, N. et al. Evaluating the impact of the global evidence, local adaptation (GELA) project for enhancing evidence-informed guideline recommendations for newborn and young child health in three African countries: a mixed-methods protocol. Health Res Policy Sys 22 , 114 (2024). https://doi.org/10.1186/s12961-024-01189-5

Download citation

Received : 02 June 2023

Accepted : 20 July 2024

Published : 19 August 2024

DOI : https://doi.org/10.1186/s12961-024-01189-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Related diseases
• Newborn and child health
• Saharan Africa (SSA)
• Informed decision
• Capacity development
• Monitoring and evaluation;
• Research impact

Health Research Policy and Systems

ISSN: 1478-4505

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• DOI: 10.55041/ijsrem32266
• Corpus ID: 269494925

AN ANALYSIS OF RISK MANAGEMENT STRATEGIES OF ICICI BANK

• Nandani Yadav
• Published in INTERANTIONAL JOURNAL OF… 30 April 2024
• Business, Economics

Related Papers

Showing 1 through 3 of 0 Related Papers

Search for articles

Writing ethics case studies for upsc mains gs paper 4.

Table of contents

"Ethics is knowing the difference between what you have a right to do and what is right to do." 

– Potter Stewart

When it comes to UPSC's GS Paper 4, this distinction becomes pivotal. How would you handle a tricky ethical dilemma in a high-stakes situation? 

The UPSC's GS Paper 4 is all about navigating these grey areas, where the 'right' answer isn't always clear-cut, where you have to make tough calls, and justifying your choices. Yes, we’re talking about clarity, logic, and a sprinkle of empathy. 

So, knowing how to write a case study for UPSC can make a big difference in your performance. Let’s go step by step, from understanding the problem to crafting a solution that would make even your toughest examiner nod in approval!

Understanding the Structure of UPSC GS Paper IV

Understanding the structure of GS Paper IV is key to success in UPSC Mains. This paper, focused on ethics, integrity, and aptitude, has twelve compulsory questions, which are crucial for learning how to write a case study for UPSC.

Question Format and Marks:

• Number of Questions: There are twelve questions divided into two sections.
• Mark Distribution: Questions are worth either 10 or 20 marks. For 10-mark questions, write within 150 words. For 20-mark questions, aim for 250 words.
• Total Marks: The paper totals 250 marks.

Types of Questions:

• Conceptual Questions: These test your understanding of ethical issues, integrity, and aptitude, making up 125 marks.
• Case Studies: These test your ability to apply ethical concepts to real-life scenarios, also worth 125 marks.

Knowing this structure helps you prepare effectively for both the theoretical and practical parts of the exam.

Now that we've grasped the framework, let's dive into one of the most intriguing sections: the case studies.

Introduction to GS Paper IV  - Case Studies for UPSC

UPSC Mains GS Paper 4 is all about testing your ethical compass. Case studies are the real deal here. They’re not just academic exercises; they’re a peek into how you’d handle the pressure of real-life administrative challenges.

Let’s take a look at the essence of this paper:

• Ethics and Human Interface: This is about understanding what's right and wrong, how it affects people, and the tricky balance between personal and professional life.
• Attitude : Your mindset matters. This part checks if you're the right fit for public service.
• Aptitude and Values : This section checks if you're good at solving problems, making fair calls, and sticking to what’s right.

Why is this paper important?

• Real-life practice : You learn to use what you've studied in real situations.
• Problem-solving skills: Facing tough choices makes you a better problem solver.
• Effective Administration: Understanding ethics makes you a fairer and more effective administrator.

So, we can establish that case studies are the heart of this paper. But what is the UPSC trying to unveil here? 

Well, the case studies for UPSC are used to assess whether you can:

• Figure out what's the right thing to do.
• Manage people and resources wisely.
• Make tough choices fairly.

Great, now that we've nailed down what the UPSC is looking for, let's break down how to actually ace and master how to write a case study for UPSC.

Crafting Your UPSC Case Study Response

When you’re figuring out how to write a case study for UPSC, a clear framework makes all the difference. This approach helps you address every important part of the case systematically. 

Here’s a breakdown of the key elements to include in your answers when writing case studies for UPSC:

Understanding the Core Issue

The first step to writing a great case study answer is figuring out what the question is really about. Don't get lost in the details.

• Capture the Essence: Start by briefly summarizing the main problem. This keeps you focused on what’s most important.
• Describe Clearly: Break down the core dilemma in simple terms. Make sure you highlight the main conflict or ethical challenge.
• Example: Think about a situation like development versus the environment. This example shows how to weigh competing interests effectively.

Identifying Stakeholders

Once you've nailed down the core issue, it's time to identify who's involved. 

• List Affected Parties: Everyone affected by the situation is a stakeholder. This could be anyone from employees and government officials to local communities and even the environment itself.

For instance, in a dam project, stakeholders could include the local population, farmers, environmental activists, the construction company, the government, and even people living downstream.

• Analyze Their Interests: Understand what each stakeholder wants. What are their interests, concerns, and potential gains or losses? This analysis helps you see the situation from different angles and anticipate potential conflicts.

Ethical Issues and Dilemmas

When you’re figuring out how to write a case study for UPSC, start by spotting the ethical issues. This helps you get to the heart of the problem and make balanced choices. Here’s how to do it:

• Identify the ethical dilemmas: These are situations where your values clash, like choosing between personal benefits and the greater good. Recognizing these conflicts is key to addressing the core issues.
• Describe these dilemmas with clear examples: For instance, in a whistleblowing case, you might face a choice between 'Personal Loss vs. Public Interest' or 'Organizational Loyalty vs. Public Interest.' Using specific examples will show how you handle these tough choices.

Brainstorming Your Options

Once you've pinpointed the problem and the ethical dilemmas, it's time to think about your options. List out 3-4 possible actions and weigh their pros and cons.

For example, if you're dealing with a corruption case in a government department, your options might be:

• Report the matter to higher authorities: This is the most straightforward approach, but it might lead to retaliation or a cover-up.
• Gather evidence discreetly and approach an anti-corruption agency: This could be more effective, but it requires careful planning and could be risky.
• Confront the corrupt official directly: This might resolve the issue quickly, but it could also escalate the situation.
• Ignore the issue: While tempting, this is clearly unethical and could have long-term consequences.

Deciding the Course of Action

Now comes the vital part: choosing the best course of action. Pick the option that strikes the best balance between ethics and practicality, which is important for learning how to write a case study for UPSC.

• Pick the Best Option: Choose the most balanced and ethical choice that works in real life.
• Explain Your Decision: Use quotes and examples to back up why you chose this option. Show you’ve thought it through.
• Keep It Practical and Legal: Make sure your choice is practical and follows current rules.

Articulation and Detailing in Your Response

When it comes to case studies in UPSC, it’s super important to explain your plan clearly. Don't just say what you'll do; show how you'll do it. Be simple and easy to understand.

• Outline Your Steps: Clearly describe each step you’ll take in your chosen course of action. This helps make your plan easy to follow and understand.
• Use First-Person Perspective: Write from your own point of view. This makes your response more personal and shows your thought process.
• Include Quotes and Terms: Use relevant quotes and key terms to back up your arguments. This adds credibility and shows you know your stuff.

Remember, the goal is to convince the examiners that your plan is practical, effective, and ethically sound.

Also watch: Perfect Strategy for Mains Answer Writing | A Complete Guide | SuperKalam  

Time Management and Practice

When figuring out how to write a case study for UPSC, time management is crucial. Here’s a simple approach:

• Set a Time Limit: Dedicate around 90 minutes for case studies during your practice sessions . This helps you get used to the exam's time limits.
• Practice Regularly: Keep writing case studies using past papers and mock tests. It’s a great way to get familiar with the format and improve your skills.

Also worth reading: How To Begin Daily Writing Practice For UPSC Mains Answers  

• Review and Get Feedback: After practicing, review your answers and ask for feedback. This helps you identify what needs improvement and refine your technique.

Need fast, expert feedback on your answers?  

SuperKalam’s 1-minute Mains Answer Evaluation has you covered. 

Share your handwritten answers, get detailed feedback and a model answer quickly.

Ready to see how all these steps come together in a real case study example? Let’s break it down.

Mastering Case Studies for UPSC: A Practical Example

Q. What does each of the following quotations mean to you?

(a) “Ethics is knowing the difference between what you have the right to do and what is right to do. “- Potter Stewart

(b) “If a country is to be corruption-free and become a nation of beautiful minds, I strongly feel that there are three key societal members who can make a difference. They are father, mother and teacher.”- APJ Abdul Kalam

GS Paper 4 (2022)

Analyzing the Quotation

a) “Ethics is knowing the difference between what you have the right to do and what is right to do.” - Potter Stewart

• Meaning: Know the difference between what’s legal and what’s moral. It’s not enough to just follow the law; you must do what’s right. 
• Application: Use this in case studies to highlight that the ethical choice can differ from the legal one. Discuss situations where doing the right thing exceeds legal requirements.

(b) “If a country is to be corruption-free and become a nation of beautiful minds, I strongly feel that there are three key societal members who can make a difference. They are father, mother, and teacher.” - APJ Abdul Kalam

• Meaning: Parents and teachers are crucial in shaping ethical behavior, building a corruption-free society through strong moral values. 
• Application: In case studies, consider how upbringing and education influence decisions. Show how early moral guidance impacts responses to ethical challenges.

Example of Case Study Analysis

Let's apply these quotes to a case study:

Case Study: You discover corruption in your department. You have to decide between reporting it, which could risk your career, or staying silent to protect yourself.

Step-by-Step Approach:

1. Understanding the Core Issue: The dilemma is reporting corruption vs. personal career safety.

2. Identifying Stakeholders: List everyone involved – you, your family, colleagues, the public, and government integrity.

3. Ethical Issues and Dilemmas:

• Personal Loss vs. Public Interest: Reporting might hurt your career but benefits the public by ensuring ethical governance.
• Organizational Loyalty vs. Public Interest: Staying silent protects colleagues but harms public trust.

4. Brainstorming Options:

• Report the corruption to higher authorities.
• Gather evidence discreetly and go to an anti-corruption agency.
• Confront the corrupt officials directly.
• Ignore the issue.

5. Deciding the Course of Action:

• Chosen Option: Report to an anti-corruption agency.
• Rationale: This meets the ethical duty to serve the public interest, despite personal risk. It follows the values from both Potter Stewart and APJ Abdul Kalam – doing what’s morally right and fighting corruption.

6. Articulation and Detailing:

• Outline steps : Collect evidence, seek legal advice, report to the agency, and ensure personal safety.
• Use first-person perspective: "I would collect evidence, seek legal advice, and report to ensure my safety and solidify the case."
• Include relevant quotes : "As Potter Stewart said, ‘Ethics is knowing the difference between what you have the right to do and what is right to do.’”

Also read: Strategy for Evaluating Your Own UPSC Mains Answers Daily  

With our approach in mind, let’s wrap up with some essential readings that can help you nail how to write a case study for UPSC.

Essential Books for UPSC GS Paper 4 Case Studies

So, you've got the framework, now let's talk books! Good books can be your best friends when tackling case studies for UPSC. They offer insights, examples, and different perspectives.

Here are some of the top picks to get you started:

Ethics, Integrity and Aptitude for Civil Services Exams	Subba Rao and P.N. Roy Chowdhury
Lexicon for Ethics, Integrity & Aptitude	Niraj Kumar
Ethics in Governance: Innovations, Issues and Instrumentalities	Ramesh K Arora
Challenges to Internal Security of India	Ashok Kumar, Vipul
The Quest for Ethical Leadership in Business	Sharad Kumar
Ethics, Integrity, and Aptitude: General Studies Paper IV	M. Karthikeyan
Ethics, Integrity, and Aptitude	Santosh Ajmera & Nanda Kishore Reddy
Ethics and Human Interface	G. Subba Rao

Looking for a game-changing study resource?

Join 25,000+ aspirants on SuperKalam’s Telegram channel ! 

Access handwritten notes, mind-maps, and daily practice targets to boost your preparation for how to write a case study for UPSC!

Let's wrap things up with a quick conclusion.

So there you have it! The key to acing UPSC case studies is all about action. Don't just theorize. Show the examiners how you'd tackle real-world problems with practical, ethical solutions.

Remember, strong ethical decision-making is the foundation for a successful career in public service. By knowing how to write a case study for UPSC, you'll demonstrate your potential to be a fair, effective, and ethical leader.

So, are you ready to write winning case study responses? SuperKalam is your personal mentor, providing expert guidance and resources to help you excel in every aspect of the exam!

Let us help you reach your UPSC goals!

You might also like: Tips to Clear UPSC Exam in First Attempt  

Recent articles

Techniques to Maximize Marks in UPSC Mains Exam

Understand how to maximize marks in the UPSC Mains Exam through time management, regular practice with mock tests, and current affairs updates.

Scoring Top Optional Subjects for UPSC Mains Exam

The blog discusses the important criteria for choosing the optional subject. Understand which optional subject is best for the UPSC Mains exam.

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Machine learning and artificial intelligence for a sustainable tourism: a case study on saudi arabia.

Share and Cite

Louati, A.; Louati, H.; Alharbi, M.; Kariri, E.; Khawaji, T.; Almubaddil, Y.; Aldwsary, S. Machine Learning and Artificial Intelligence for a Sustainable Tourism: A Case Study on Saudi Arabia. Information 2024 , 15 , 516. https://doi.org/10.3390/info15090516

Louati A, Louati H, Alharbi M, Kariri E, Khawaji T, Almubaddil Y, Aldwsary S. Machine Learning and Artificial Intelligence for a Sustainable Tourism: A Case Study on Saudi Arabia. Information . 2024; 15(9):516. https://doi.org/10.3390/info15090516

Louati, Ali, Hassen Louati, Meshal Alharbi, Elham Kariri, Turki Khawaji, Yasser Almubaddil, and Sultan Aldwsary. 2024. "Machine Learning and Artificial Intelligence for a Sustainable Tourism: A Case Study on Saudi Arabia" Information 15, no. 9: 516. https://doi.org/10.3390/info15090516

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals