Objective: Surveys are a commonly used tool in quality improvement (QI) projects, but little is known about the standards to which they are designed and applied. We aimed to investigate the quality of surveys used within a QI collaborative, and to characterise the common errors made in survey design.

Methods: Five reviewers (two research methodology and QI, three clinical and QI experts) independently assessed 20 surveys, comprising 250 survey items, that were developed in a North American cystic fibrosis lung transplant transition collaborative. Content Validity Index (CVI) scores were calculated for each survey. Reviewer consensus discussions decided an overall quality assessment for each survey and survey item (analysed using descriptive statistics) and explored the rationale for scoring (using qualitative thematic analysis).

Results: 3/20 surveys scored as high quality (CVI >80%). 19% (n=47) of survey items were recommended by the reviewers, with 35% (n=87) requiring improvements, and 46% (n=116) not recommended. Quality assessment criteria were agreed upon. Types of common errors identified included the ethics and appropriateness of questions and survey format; usefulness of survey items to inform learning or lead to action, and methodological issues with survey questions, survey response options; and overall survey design.

Conclusion: Survey development is a task that requires careful consideration, time and expertise. QI teams should consider whether a survey is the most appropriate form for capturing information during the improvement process. There is a need to educate and support QI teams to adhere to good practice and avoid common errors, thereby increasing the value of surveys for evaluation and QI. The methodology, quality assessment criteria and common errors described in this paper can provide a useful resource for this purpose. © Author(s) (or their employer(s)) 2024. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The increasing prevalence of allergic disease has resulted in the recognition of allergy as a global public health concern. Yet health services worldwide appear to be ill-equipped to deliver high-quality allergy care. Clinical guidelines have been developed to describe what high-quality care looks like for most allergic diseases. However, allergy guidelines do not describe how the delivery of such care is organised across clinicians and provider organisations with varying degrees of access to allergy expertise and clinical resources. In this article, we describe how care pathways can be used to improve the organisation and delivery of allergy care in accordance with the characteristics of allergic disease and local constraints in the health service. We then describe how quality-improvement methods can support the successful realisation of allergy care pathways in practice. Realising care pathways involves a highly complex process of changing the way care is practised and organised. This could involve developing a new service, clinical training or other interventions. Quality-improvement methods were developed as a guide to navigate and support the process of change and improvement. © 2023, Allergy Society of South Africa. All rights reserved.

Background: Substantial resources are invested by Health Departments worldwide in introducing National Clinical Audits (NCAs). Yet, there is variable evidence on the NCAs’ effectiveness and little is known on factors underlying the successful use of NCAs to improve local practice. This study will focus on a single NCA (the National Audit of Inpatient Falls -NAIF 2017) to explore: (i) participants’ perspectives on the NCA reports, local feedback characteristics and actions undertaken following the feedback underpinning the effective use of the NCA feedback to improve local practice; (ii) reported changes in local practice following the NCA feedback in England and Wales. Methods: Front-line staff perspectives were gathered through interviews. An inductive qualitative approach was used. Eighteen participants were purposefully sampled from 7 of the 85 participating hospitals in England and Wales. Analysis was guided by constant comparative techniques. Results: Regarding the NAIF annual report, interviewees valued performance benchmarking with other hospitals, the use of visual representations and the inclusion of case studies and recommendations. Participants stated that feedback should target front-line healthcare professionals, be straightforward and focused, and be delivered through an encouraging and honest discussion. Interviewees highlighted the value of using other relevant data sources alongside NAIF feedback and the importance of continuous data monitoring. Participants reported that engagement of front-line staff in the NAIF and following improvement activities was critical. Leadership, ownership, management support and communication at different organisational levels were perceived as enablers, while staffing level and turnover, and poor quality improvement (QI) skills, were perceived as barriers to improvement. Reported changes in practice included increased awareness and attention to patient safety issues and greater involvement of patients and staff in falls prevention activities. Conclusions: There is scope to improve the use of NCAs by front-line staff. NCAs should not be seen as isolated interventions but should be fully embedded and integrated into the QI strategic and operational plans of NHS trusts. The use of NCAs could be optimised, but knowledge of them is poor and distributed unevenly across different disciplines. More research is needed to provide guidance on key elements to consider throughout the whole improvement process at different organisational levels. © 2023, The Author(s).

Background: Artificial intelligence (AI) for healthcare presents potential solutions to some of the challenges faced by health systems around the world. However, it is well established in implementation and innovation research that novel technologies are often resisted by healthcare leaders, which contributes to their slow and variable uptake. Although research on various stakeholders’ perspectives on AI implementation has been undertaken, very few studies have investigated leaders’ perspectives on the issue of AI implementation in healthcare. It is essential to understand the perspectives of healthcare leaders, because they have a key role in the implementation process of new technologies in healthcare. The aim of this study was to explore challenges perceived by leaders in a regional Swedish healthcare setting concerning the implementation of AI in healthcare.

Methods: The study takes an explorative qualitative approach. Individual, semi-structured interviews were conducted from October 2020 to May 2021 with 26 healthcare leaders. The analysis was performed using qualitative content analysis, with an inductive approach.

Results: The analysis yielded three categories, representing three types of challenge perceived to be linked with the implementation of AI in healthcare: 1) Conditions external to the healthcare system; 2) Capacity for strategic change management; 3) Transformation of healthcare professions and healthcare practice.

Conclusions: In conclusion, healthcare leaders highlighted several implementation challenges in relation to AI within and beyond the healthcare system in general and their organisations in particular. The challenges comprised conditions external to the healthcare system, internal capacity for strategic change management, along with transformation of healthcare professions and healthcare practice. The results point to the need to develop implementation strategies across healthcare organisations to address challenges to AI-specific capacity building. Laws and policies are needed to regulate the design and execution of effective AI implementation strategies. There is a need to invest time and resources in implementation processes, with collaboration across healthcare, county councils, and industry partnerships. © The Author(s) 2022.

Background: Advancing the description and conceptualisation of interventions in complex systems is necessary to support spread, evaluation, attribution and reproducibility. Improvement teams can provide unique insight into how interventions are operationalised in practice. Capturing this 'insider knowledge' has the potential to enhance intervention descriptions.

Objectives: This exploratory study investigated the spread of a comprehensive medication review (CMR) intervention to (1) describe the work required from the improvement team perspective, (2) identify what stays the same and what changes between the different sites and why, and (3) critically appraise the 'hard core' and 'soft periphery' (HC/SP) construct as a way of conceptualising interventions.

Design: A prospective case study of a CMR initiative across five sites. Data collection included: observations, document analysis and semistructured interviews. A facilitated workshop triangulated findings and measured perceived effort invested in activities. A qualitative database was developed to conduct thematic analysis.

Results: Sites identified 16 intervention components. All were considered essential due to their interdependency. The function of components remained the same, but adaptations were made between and within sites. Components were categorised under four 'spheres of operation': Accessibility of evidence base; Process of enactment; Dependent processes and Dependent sociocultural issues. Participants reported most effort was invested on 'dependent sociocultural issues'. None of the existing HC/SP definitions fit well with the empirical data, with inconsistent classifications of components as HC or SP.

Conclusions: This study advances the conceptualisation of interventions by explicitly considering how evidence-based practices are operationalised in complex systems. We propose a new conceptualisation of 'interventions-in-systems' which describes intervention components in relation to their: proximity to the evidence base; component interdependence; component function; component adaptation and effort. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Background: Significant efforts have been made to develop artificial intelligence (AI) solutions for health care improvement. Despite the enthusiasm, health care professionals still struggle to implement AI in their daily practice.

Objective: This paper aims to identify the implementation frameworks used to understand the application of AI in health care practice.

Methods: A scoping review was conducted using the Cochrane, Evidence Based Medicine Reviews, Embase, MEDLINE, and PsycINFO databases to identify publications that reported frameworks, models, and theories concerning AI implementation in health care. This review focused on studies published in English and investigating AI implementation in health care since 2000. A total of 2541 unique publications were retrieved from the databases and screened on titles and abstracts by 2 independent reviewers. Selected articles were thematically analyzed against the Nilsen taxonomy of implementation frameworks, and the Greenhalgh framework for the nonadoption, abandonment, scale-up, spread, and sustainability (NASSS) of health care technologies.

Results: In total, 7 articles met all eligibility criteria for inclusion in the review, and 2 articles included formal frameworks that directly addressed AI implementation, whereas the other articles provided limited descriptions of elements influencing implementation. Collectively, the 7 articles identified elements that aligned with all the NASSS domains, but no single article comprehensively considered the factors known to influence technology implementation. New domains were identified, including dependency on data input and existing processes, shared decision-making, the role of human oversight, and ethics of population impact and inequality, suggesting that existing frameworks do not fully consider the unique needs of AI implementation.

Conclusions: This literature review demonstrates that understanding how to implement AI in health care practice is still in its early stages of development. Our findings suggest that further research is needed to provide the knowledge necessary to develop implementation frameworks to guide the future implementation of AI in clinical practice and highlight the opportunity to draw on existing knowledge from the field of implementation science. ©Fábio Gama, Daniel Tyskbo, Jens Nygren, James Barlow, Julie Reed, Petra Svedberg. 

Introduction: Artificial intelligence (AI) is widely seen as critical for tackling fundamental challenges faced by health systems. However, research is scant on the factors that influence the implementation and routine use of AI in healthcare, how AI may interact with the context in which it is implemented, and how it can contribute to wider health system goals. We propose that AI development can benefit from knowledge generated in four scientific fields: intervention, innovation, implementation and improvement sciences.

Aim: The aim of this paper is to briefly describe the four fields and to identify potentially relevant knowledge from these fields that can be utilized for understanding and/or facilitating the use of AI in healthcare. The paper is based on the authors' experience and expertise in intervention, innovation, implementation, and improvement sciences, and a selective literature review.

Utilizing knowledge from the four fields: The four fields have generated a wealth of often-overlapping knowledge, some of which we propose has considerable relevance for understanding and/or facilitating the use of AI in healthcare.

Conclusion: Knowledge derived from intervention, innovation, implementation, and improvement sciences provides a head start for research on the use of AI in healthcare, yet the extent to which this knowledge can be repurposed in AI studies cannot be taken for granted. Thus, when taking advantage of insights in the four fields, it is important to also be explorative and use inductive research approaches to generate knowledge that can contribute toward realizing the potential of AI in healthcare. © 2022 Nilsen, Reed, Nair, Savage, Macrae, Barlow, Svedberg, Larsson, Lundgren and Nygren. 

Background: The uptake of artificial intelligence (AI) in health care is at an early stage. Recent studies have shown a lack of AI-specific implementation theories, models, or frameworks that could provide guidance for how to translate the potential of AI into daily health care practices. This protocol provides an outline for the first 5 years of a research program seeking to address this knowledge-practice gap through collaboration and co-design between researchers, health care professionals, patients, and industry stakeholders.

Objective: The first part of the program focuses on two specific objectives. The first objective is to develop a theoretically informed framework for AI implementation in health care that can be applied to facilitate such implementation in routine health care practice. The second objective is to carry out empirical AI implementation studies, guided by the framework for AI implementation, and to generate learning for enhanced knowledge and operational insights to guide further refinement of the framework. The second part of the program addresses a third objective, which is to apply the developed framework in clinical practice in order to develop regional capacity to provide the practical resources, competencies, and organizational structure required for AI implementation; however, this objective is beyond the scope of this protocol.

Methods: This research program will use a logic model to structure the development of a methodological framework for planning and evaluating implementation of AI systems in health care and to support capacity building for its use in practice. The logic model is divided into time-separated stages, with a focus on theory-driven and coproduced framework development. The activities are based on both knowledge development, using existing theory and literature reviews, and method development by means of co-design and empirical investigations. The activities will involve researchers, health care professionals, and other stakeholders to create a multi-perspective understanding.

Results: The project started on July 1, 2021, with the Stage 1 activities, including model overview, literature reviews, stakeholder mapping, and impact cases; we will then proceed with Stage 2 activities. Stage 1 and 2 activities will continue until June 30, 2026.

Conclusions: There is a need to advance theory and empirical evidence on the implementation requirements of AI systems in health care, as well as an opportunity to bring together insights from research on the development, introduction, and evaluation of AI systems and existing knowledge from implementation research literature. Therefore, with this research program, we intend to build an understanding, using both theoretical and empirical approaches, of how the implementation of AI systems should be approached in order to increase the likelihood of successful and widespread application in clinical practice. © Petra Svedberg, Julie Reed, Per Nilsen, James Barlow, Carl Macrae, Jens Nygren.

Research on organizational interventions needs to meet the objectives of both researchers and participating organizations. This duality means that real-world impact has to be considered throughout the research process, simultaneously addressing both scientific rigour and practical relevance. This discussion paper aims to offer a set of principles, grounded in knowledge from various disciplines that can guide researchers in designing, implementing, and evaluating organizational interventions. Inspired by Mode 2 knowledge production, the principles were developed through a transdisciplinary, participatory and iterative process where practitioners and academics were invited to develop, refine and validate the principles. The process resulted in 10 principles: 1) Ensure active engagement and participation among key stakeholders; 2) Understand the situation (starting points and objectives); 3) Align the intervention with existing organizational objectives; 4) Explicate the program logic; 5) Prioritize intervention activities based on effort-gain balance; 6) Work with existing practices, processes, and mindsets; 7) Iteratively observe, reflect, and adapt; 8) Develop organizational learning capabilities; 9) Evaluate the interaction between intervention, process, and context; and 10) Transfer knowledge beyond the specific organization. The principles suggest how the design, implementation, and evaluation of organizational interventions can be researched in a way that maximizes both practical and scientific impact. © 2020 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.

Objectives: To improve the provision of health care, academics can be asked to collaborate with clinicians, and clinicians with patients. Generating good evidence on health care practice depends on these collaborations working well. Yet such relationships are not the norm. We examine how social science research and health care improvement practice were linked through a programme designed to broker collaborations between clinicians, academics, and patients to improve health care – the UK National Institute for Health Research Collaboration for Leadership in Applied Health Research and Care for Northwest London. We discuss the successes and challenges of the collaboration and make suggestions on how to develop synergistic relationships that facilitate co-production of social science knowledge and its translation into practice. Methods: A qualitative approach was used, including ethnographic elements and critical, reflexive dialogue between members of the two collaborating teams. Results: Key challenges and remedies were connected with the risks associated with new ways of working. These risks included differing ideas between collaborators about the purpose, value, and expectations of research, and institutional opposition. Dialogue between collaborators did not mean absence of tensions or clashes. Risk-taking was unpopular – institutions, funders, and partners did not always support it, despite simultaneously demanding ‘innovation’ in producing research that influenced practice. Conclusions: Our path was made smoother because we had funding to support the creation of a ‘potential space’ to experiment with different ways of working. Other factors that can enhance collaboration include a shared commitment to dialogical practice, a recognition of the legitimacy of different partners’ knowledge, a long timeframe to identify and resolve problems, the maintenance of an enabling environment for collaboration, a willingness to work iteratively and reflexively, and a shared end goal. © The Author(s) 2020.