Innovation bundles and platforms - a qualitative analysis of health system responses to the COVID-19 pandemic.

Background: Health systems underwent substantial changes to respond to COVID-19. Learning from the successes and failures of health system COVID-19 responses may help us understand how future health service responses can be designed to be both effective and sustainable. This study aims to identify the role that innovation played in crafting health service responses during the COVID-19 pandemic.

A qualitative analysis of health service problems and the strategies used to manage them in the COVID-19 pandemic: exploiting generic and context-specific approaches.

Background: The COVID-19 pandemic disrupted health systems around the globe. Lessons from health systems responses to these challenges may help design effective and sustainable health system responses for future challenges. This study aimed to 1/ identify the broad types of health system challenges faced during the pandemic and 2/ develop a typology of health system response to these challenges.

Evaluating Artificial Intelligence in Clinical Settings-Let Us Not Reinvent the Wheel.

Given the requirement to minimize the risks and maximize the benefits of technology applications in health care provision, there is an urgent need to incorporate theory-informed health IT (HIT) evaluation frameworks into existing and emerging guidelines for the evaluation of artificial intelligence (AI). Such frameworks can help developers, implementers, and strategic decision makers to build on experience and the existing empirical evidence base. We provide a pragmatic conceptual overview of selected concrete examples of how existing theory-informed HIT evaluation frameworks may be used to inform the safe development and implementation of AI in health care settings.

Improving health system responses when patients are harmed: a protocol for a multistage mixed-methods study.

Introduction: At least 10% of hospital admissions in high-income countries, including Australia, are associated with patient safety incidents, which contribute to patient harm ('adverse events'). When a patient is seriously harmed, an investigation or review is undertaken to reduce the risk of further incidents occurring. Despite 20 years of investigations into adverse events in healthcare, few evaluations provide evidence of their quality and effectiveness in reducing preventable harm.

Achieving large-scale clinician adoption of AI-enabled decision support.

Computerised decision support (CDS) tools enabled by artificial intelligence (AI) seek to enhance accuracy and efficiency of clinician decision-making at the point of care. Statistical models developed using machine learning (ML) underpin most current tools. However, despite thousands of models and hundreds of regulator-approved tools internationally, large-scale uptake into routine clinical practice has proved elusive.

How should artificial intelligence be used in Australian health care? Recommendations from a citizens' jury.

Objective: To support a diverse sample of Australians to make recommendations about the use of artificial intelligence (AI) technology in health care.

Study Design: Citizens' jury, deliberating the question: "Under which circumstances, if any, should artificial intelligence be used in Australian health systems to detect or diagnose disease?"

Setting, Participants: Thirty Australian adults recruited by Sortition Foundation using random invitation and stratified selection to reflect population proportions by gender, age, ancestry, highest level of education, and residential location (state/territory; urban, regional, rural). The jury process took 18 days (16 March - 2 April 2023): fifteen days online and three days face-to-face in Sydney, where the jurors, both in small groups and together, were informed about and discussed the question, and developed recommendations with reasons.

Automating the Identification of Safety Events Involving Machine Learning-Enabled Medical Devices.

With growing use of machine learning (ML)-enabled medical devices by clinicians and consumers safety events involving these systems are emerging. Current analysis of safety events heavily relies on retrospective review by experts, which is time consuming and cost ineffective. This study develops automated text classifiers and evaluates their potential to identify rare ML safety events from the US FDA's MAUDE.

Assessing the Safety of a New Clinical Decision Support System for a National Helpline.

We assessed the safety of a new clinical decision support system (CDSS) for nurses on Australia's national consumer helpline. Accuracy and safety of triage advice was assessed by testing the CDSS using 78 standardised patient vignettes (48 published and 30 proprietary). Testing was undertaken in two cycles using the CDSS vendor's online evaluation tool (Cycle 1: 47 vignettes; Cycle 2: 41 vignettes).

Using Clinical Simulation to Evaluate AI-Enabled Decision Support.

Clinical simulation is a useful method for evaluating AI-enabled clinical decision support (CDS). Simulation studies permit patient- and risk-free evaluation and far greater experimental control than is possible with clinical studies. The effect of CDS assisted and unassisted patient scenarios on meaningful downstream decisions and actions within the information value chain can be evaluated as outcome measures.

How Well Do AI-Enabled Decision Support Systems Perform in Clinical Settings?

Real-world performance of machine learning (ML) models is crucial for safely and effectively embedding them into clinical decision support (CDS) systems. We examined evidence about the performance of contemporary ML-based CDS in clinical settings. A systematic search of four bibliographic databases identified 32 studies over a 5-year period.

Automation in Contemporary Clinical Information Systems: a Survey of AI in Healthcare Settings.

Aims And Objectives: To examine the nature and use of automation in contemporary clinical information systems by reviewing studies reporting the implementation and evaluation of artificial intelligence (AI) technologies in healthcare settings.

Method: PubMed/MEDLINE, Web of Science, EMBASE, the tables of contents of major informatics journals, and the bibliographies of articles were searched for studies reporting evaluation of AI in clinical settings from January 2021 to December 2022. We documented the clinical application areas and tasks supported, and the level of system autonomy.

Clinical decision support versus a paper-based protocol for massive transfusion: Impact on decision outcomes in a simulation study.

Background: Management of major hemorrhage frequently requires massive transfusion (MT) support, which should be delivered effectively and efficiently. We have previously developed a clinical decision support system (CDS) for MT using a multicenter multidisciplinary user-centered design study. Here we examine its impact when administering a MT.

Artificial intelligence in medicine: has the time come to hang up the stethoscope?

The question of whether the time has come to hang up the stethoscope is bound up in the promises of artificial intelligence (AI), promises that have so far proven difficult to deliver, perhaps because of the mismatch between the technical capability of AI and its use in real-world clinical settings. This perspective argues that it is time to move away from discussing the generalised promise of disembodied AI and focus on specifics. We need to focus on how the computational method underlying AI, i.

Effects of machine learning-based clinical decision support systems on decision-making, care delivery, and patient outcomes: a scoping review.

Objective: This study aims to summarize the research literature evaluating machine learning (ML)-based clinical decision support (CDS) systems in healthcare settings.

Materials And Methods: We conducted a review in accordance with the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta Analyses extension for Scoping Review). Four databases, including PubMed, Medline, Embase, and Scopus were searched for studies published from January 2016 to April 2021 evaluating the use of ML-based CDS in clinical settings.

The need to strengthen the evaluation of the impact of Artificial Intelligence-based decision support systems on healthcare provision.

Despite the renewed interest in Artificial Intelligence-based clinical decision support systems (AI-CDS), there is still a lack of empirical evidence supporting their effectiveness. This underscores the need for rigorous and continuous evaluation and monitoring of processes and outcomes associated with the introduction of health information technology. We illustrate how the emergence of AI-CDS has helped to bring to the fore the critical importance of evaluation principles and action regarding all health information technology applications, as these hitherto have received limited attention.

Evaluation of machine learning-based models for prediction of clinical deterioration: A systematic literature review.

Background And Objective: Early identification of patients at risk of deterioration can prevent life-threatening adverse events and shorten length of stay. Although there are numerous models applied to predict patient clinical deterioration, most are based on vital signs and have methodological shortcomings that are not able to provide accurate estimates of deterioration risk. The aim of this systematic review is to examine the effectiveness, challenges, and limitations of using machine learning (ML) techniques to predict patient clinical deterioration in hospital settings.

More than algorithms: an analysis of safety events involving ML-enabled medical devices reported to the FDA.

Objective: To examine the real-world safety problems involving machine learning (ML)-enabled medical devices.

Materials And Methods: We analyzed 266 safety events involving approved ML medical devices reported to the US FDA's MAUDE program between 2015 and October 2021. Events were reviewed against an existing framework for safety problems with Health IT to identify whether a reported problem was due to the ML device (device problem) or its use, and key contributors to the problem.

Multicenter, multidisciplinary user-centered design of a clinical decision-support and simulation system for massive transfusion.

Background: Managing critical bleeding with massive transfusion (MT) requires a multidisciplinary team, often physically separated, to perform several simultaneous tasks at short notice. This places a significant cognitive load on team members, who must maintain situational awareness in rapidly changing scenarios. Similar resuscitation scenarios have benefited from the use of clinical decision support (CDS) tools.

Using automated methods to detect safety problems with health information technology: a scoping review.

Objective: To summarize the research literature evaluating automated methods for early detection of safety problems with health information technology (HIT).

Materials And Methods: We searched bibliographic databases including MEDLINE, ACM Digital, Embase, CINAHL Complete, PsycINFO, and Web of Science from January 2010 to June 2021 for studies evaluating the performance of automated methods to detect HIT problems. HIT problems were reviewed using an existing classification for safety concerns.

Using artificial intelligence for personal protective equipment guidance for healthcare workers in the COVID-19 pandemic and beyond.

Background: Current procedures for effective personal protective equipment (PPE) usage rely on the availability of trained observers or 'buddies' who, during the COVID-19 pandemic, are not always available. The application of artificial intelligence (AI) has the potential to overcome this limitation by assisting in complex task analysis. To date, AI use for PPE protocols has not been studied.

Digital health for climate change mitigation and response: a scoping review.

Objective: Climate change poses a major threat to the operation of global health systems, triggering large scale health events, and disrupting normal system operation. Digital health may have a role in the management of such challenges and in greenhouse gas emission reduction. This scoping review explores recent work on digital health responses and mitigation approaches to climate change.

The Role of Formative Evaluation in Promoting Digitally-based Health Equity and Reducing Bias for Resilient Health Systems: The Case of Patient Portals.

Objectives: Patient portals are increasingly implemented to improve patient involvement and engagement. We here seek to provide an overview of ways to mitigate existing concerns that these technologies increase inequity and bias and do not reach those who could benefit most from them.

Methods: Based on the current literature, we review the limitations of existing evaluations of patient portals in relation to addressing health equity, literacy and bias; outline challenges evaluators face when conducting such evaluations; and suggest methodological approaches that may address existing shortcomings.

How Well Do Computerised Decision Support Systems Cover Nursing Standards of Practice? A Literature Review.

The use of computerized decision support systems (DSS) in nursing practice is increasing. However, research about who uses DSS, where are they implemented, and how they are linked with standards of nursing is limited. This paper presents evidence on users and settings of DSS implementation, along with specific nursing standards of practice that are facilitated by such DSS.

Ethical Guidance for Hard Decisions: A Critical Review of Early International COVID-19 ICU Triage Guidelines.

This article provides a critical comparative analysis of the substantive and procedural values and ethical concepts articulated in guidelines for allocating scarce resources in the COVID-19 pandemic. We identified 21 local and national guidelines written in English, Spanish, German and French; applicable to specific and identifiable jurisdictions; and providing guidance to clinicians for decision making when allocating critical care resources during the COVID-19 pandemic. US guidelines were not included, as these had recently been reviewed elsewhere.