In my previous post, The Dawn of AI in Healthcare: Promise and Peril, I explored the burgeoning role of Artificial Intelligence in medicine and the critical need for robust regulatory frameworks and organisational AI governance. Building upon that foundation, this blog delves deeper into the profound impact of AI on patient experience, specifically focusing on the important trust relationship between patients and clinicians.

The promise of AI in patient experience

AI promises a revolution in patient care, moving towards a truly personalised and seamless healthcare experience. Imagine a world where your needs are intuitively understood across all touchpoints, from initial consultation to follow-up care, and where clinical interventions achieve high levels of efficacy. Imagine AI-powered drug discovery that identifies the most effective medications for a specific patient profile. Or consider AI-powered wearable devices and remote monitoring systems that track your physiology including vital signs and alert you and healthcare providers to potential problems before they escalate. AI can also be used to create bots and mobile apps, that can answer common patient questions, schedule appointments and provide post-operative care instructions. This will provide patients with 24/7 access to information and help free up clinician time. This vision of an omnichannel, AI-powered healthcare ecosystem holds immense potential for enhancing the patient experience.

The importance of clinician-patient trust

The clinician-patient relationship is built on a bedrock of trust, a bond considered sacrosanct in healthcare. As patients, we entrust our very lives to the expertise and care of doctors, nurses, midwives and allied health professionals. This trust is not merely a preference; it’s a fundamental requirement for effective healthcare delivery. AI introduces a third-party into that relationship, whose influence has the potential to undermine that trust.

AI’s impact on trust: ethical considerations

Introducing AI into clinical settings inevitably raises complex ethical questions. Issues such as algorithmic bias stemming from poorly trained models and the critical need for clear accountability become paramount. Patients rightfully demand transparency, seeking to understand the rationale behind AI-driven diagnoses and treatment plans, mirroring their expectations of clinician explanations.

Data privacy, security and informed consent

Beyond transparency and accountability, AI necessitates a renewed focus on data privacy and security. Healthcare providers have long navigated the complexities of informed consent regarding procedures, financial matters and research participation. The integration of AI potentially impacts data privacy and security via the use of patient data within AI algorithms, necessitating robust privacy safeguards, including patient consent considerations.

Preserving trust in an AI-driven healthcare landscape

The erosion of patient trust can have severe consequences. If data security is compromised, if AI’s decision-making processes remain opaque, or if clinician oversight is diminished, patient confidence will falter. Ultimately, patient experience hinges on trust in their clinicians and the perceived quality of care. AI can be a powerful tool to enhance these elements, but only if implemented thoughtfully.

From black box to transparent system

Treating AI as a “digital black box” is a surefire way to undermine trust. Irrespective of industry, organisations must prioritise transparency. Building upon the internal governance structures discussed in my previous blog, investing in sophisticated observability tools is crucial. These tools provide comprehensive visibility into AI systems, encompassing data usage, security protocols and real-time performance within clinical and patient settings. By illuminating the inner workings of AI, we can identify anomalies and foster a deeper understanding, thereby strengthening trust among clinicians and patients alike.

Meeting AI’s data demands

With AI, comes huge demand for data that is secure, reliable, accurate and trusted. As discussed in my first blog, ensuring observability across all system components is vital. When it comes to AI, enterprise-wide observability of data flows provides organisations with important tools to prevent bottle-necks in algorithm-driven data flows and proactively address any unexpected data issues. I have also commented that organisations now have little justification for not investing in observability solutions that provide scalable and affordable tools for optimising network and application performance, managing digital experiences, proactively detecting and resolving issues, and providing comprehensive, end-to-end monitoring and management of enterprise systems. When using AI, organisations should consider these enterprise observability tools as an insurance policy against a poor AI experience or worse, a loss of trust.

Conclusion

AI holds immense promise for transforming healthcare, driving efficiency and improving patient outcomes. However, it also presents a unique challenge to the established trust relationship between clinicians and patients. While robust AI governance is essential, cultivating transparency and explainability through advanced observability tools is equally critical. By prioritising these elements, we can harness the power of AI to enhance patient experience while preserving the trust relationship that underpins effective healthcare.

Dr. Thatcher is CEO and Founder of Strategance Group, a firm specialising in digital strategy, risk and governance services to assist organisations with their digital investments. Dr.Thatcher is a published author and has held senior executive roles in large public sector and private sector organisations. Notable roles included Chief Technology Officer of the Australian Digital Health Agency, Chief Health Information Officer for Queensland Health; CEO of eHealth Queensland; and Chief Information Officer and Executive Director Facilities for the Mater Health Group. Dr. Thatcher was also formerly a Professor of Digital Practice in the QUT Graduate School of Business.

My previous blog, When Every Second Counts: A Healthcare Perspective, discussed the importance of optimising clinician digital experience in an effort to improve the efficiency and quality of care. Continuing this theme of efficiency and quality in healthcare, I would like to discuss the emergence of Artificial Intelligence (AI). Like many other sectors, healthcare is set to be revolutionised by AI, offering the promise of more efficient care delivery and significantly improved patient outcomes. AI excels at pattern recognition, making it a powerful tool for diagnosing diseases, identifying optimal treatment pathways and personalising care.

Imagine a future where medical imaging is analysed by AI, detecting subtle signs of disease that might be missed by the human eye. This is already becoming a reality. Furthermore, advancements in personalised medicine, fuelled by AI, will enable tailored treatments based on an individual’s unique genetic makeup, medical history and physiological characteristics. More prosaically, AI can also be leveraged in IT Operations to automate and remediate incidents – even before humans are aware of any problem that may affect the delivery of healthcare services.

However, this promising future comes with challenges. The rapid advancement of AI technology often outpaces regulatory frameworks designed to govern its use. Data privacy and the potential for bias in AI algorithms are critical concerns that demand ongoing attention.

The importance of data governance

Data governance plays a pivotal role in ensuring the safe and responsible use of AI in healthcare. Clear guidelines are needed on how AI algorithms utilise patient data while simultaneously safeguarding sensitive information. This presents a delicate balancing act, as access to large volumes of high-quality data is crucial for training effective AI models.

The use of general-purpose Large Language Models (LLMs) in healthcare introduces unique privacy concerns. These models often lack the necessary safeguards to handle sensitive health information. To mitigate these risks, the healthcare sector should consider developing dedicated, health data LLMs. These specialised models would operate on de-identified data and be accessible only to certified healthcare organisations, ensuring both the advancement of AI-driven solutions and the protection of patient privacy.

Regulatory developments in Australia

The Australian government is actively working to establish a regulatory framework for AI in healthcare. In 2024, the Department of Health and Aged Care initiated the “Safe and Responsible Artificial Intelligence in Health Care Legislation and Regulation Review,” calling for public submissions. However, it will take time before regulatory changes are implemented.

The Australian Health Practitioner Regulation Agency (AHPRA) has published guidelines for health practitioners on the ethical and responsible use of AI, emphasising principles like accountability, transparency and informed consent. Additionally, the Therapeutic Goods Administration (TGA) oversees AI tools classified as medical devices, ensuring they meet safety and efficacy standards. However, many AI applications in healthcare fall outside the TGA’s scope, highlighting the need for a more comprehensive regulatory approach.

Furthermore, the Australian government has introduced a voluntary standard for AI, with 10 key guardrails. Nine of these guardrails are set to become legislated as mandatory for high-risk AI applications, a category that includes many AI applications in healthcare.

Key guardrails for AI in healthcare

Some of the most relevant AI guardrails for healthcare include:

• Protecting AI systems by implementing data governance measures to manage data quality and provenance.
• Testing AI models to evaluate model performance and continuously monitor AI systems post-deployment.
• Ensuring human oversight to enable meaningful human control or intervention across the AI system’s lifecycle.
• Enhancing transparency to inform end-users about AI-enabled decisions, AI interactions and AI-generated content.

Another critical guardrail addresses the AI technology supply chain: organisations must be transparent about data, models, and systems across the AI / technology supply chain. To do so effectively requires real-time observability of technology systems. This observability supports:

• Risk mitigation: Identifying and addressing risks associated with data sharing and model development.
• Trust and collaboration: Fostering transparency to build trust and facilitate cooperation among AI stakeholders.
• Regulatory compliance: Ensuring adherence to relevant regulations and ethical guidelines.

This real-time observability of AI should be embedded in technology operations to allow the organisation to monitor AI use of corporate data. For example, an alert could be configured whenever LLMs were called using certain datasets.

Proactive management of AI in organisations is critical to maintaining trust with stakeholders. Management of AI requires organisations to think carefully about AI and data governance, including principles, policies and accountabilities.

Conclusion

AI holds immense potential to transform healthcare, but its successful implementation requires a multi-faceted approach. By prioritising data privacy, ensuring human oversight, and fostering a transparent and collaborative ecosystem, healthcare can harness the power of AI to improve patient care while safeguarding patient safety and trust.

Dr. Thatcher is CEO and Founder of Strategance Group, a firm specialising in digital strategy, risk and governance services to assist organisations with their digital investments. Dr.Thatcher is a published author and has held senior executive roles in large public sector and private sector organisations. Notable roles included Chief Technology Officer of the Australian Digital Health Agency, Chief Health Information Officer for Queensland Health; CEO of eHealth Queensland; and Chief Information Officer and Executive Director Facilities for the Mater Health Group. Dr. Thatcher was also formerly a Professor of Digital Practice in the QUT Graduate School of Business.

I have spent my entire working life in the IT profession. In the earlier part of my career I designed, built and implemented enterprise systems for Finance and HR/Payroll, Supply Chain systems, Energy and Mining systems, Policing and Legal Aid systems, and Agricultural systems (to name a few). Without doubt, these systems had varying degrees of complexity but none of them compared to the systems I was involved in when I entered healthcare in the second half of my career.

As a CIO in large health systems, both public and private, and in my federal role with the Australian Digital Health Agency, I had oversight of implementing more clinical systems than I care to count. Outside of my executive roles, I have also been a full-time Professor of Digital Practice and undertaken significant reviews on behalf of governments in Australia.

Unique Challenges of IT in Healthcare

Throughout my 20 years in healthcare, there is one thing that stands out as differentiating the healthcare sector from other sectors I have worked in – and that is the human impact of technology in healthcare. Healthcare is a sector where seconds matter. Timely access to information by clinicians is critical and therefore performance of clinical systems and networks is critical to patient outcomes and the overall efficiency of delivering care.

The Complexity of Implementing Clinical Systems

Healthcare professionals, including clinicians and administrative staff are inextricably bound by a desire to heal and help those who are often facing some of life’s greatest challenges. I have had the privilege of working in hospitals for many years, experiencing firsthand the tribulations of both patients and staff.

Implementing technology solutions, particularly those involving major disruption or transformation of clinical workflows, is challenging. Let me elaborate…

• Clinical systems nearly always require localisation and configuration. Such configurations, particularly relating to complex clinical workflows are perfectly imperfect, i.e. they rely on clinicians to interact with software ‘experts’ to build these workflows and rarely is the result anywhere near perfect. Often the workflows need ongoing tweaking and refinement over a long period. This understandably results in frustration for clinicians.
• The clinical workforce has varying degrees of digital competency. For example, asking a nurse for the first time to use an electronic medical record (EMR) to undertake nursing assessments, charting and administering medications can be terrifying when they’ve spent their careers caring for people and not always learning technology. The time taken to become proficient in the use of these systems is often viewed as taking away from time spent with patients.
• Many hospitals still manage hybrid records. While many Australian hospitals have ventured into the world of paperless EMRs, many still have not and need to deal with the hybrid nature of both digital and paper medical records. Beyond EMRs and other specialist clinical information systems, clinicians are faced with a rapidly evolving digital health landscape including the integration of AI into decision-making and advancements in personalised medicine – both of which are challenging the traditional roles of clinicians.

Key Technical Factors in System Performance

Whilst technology in healthcare is here to stay, clinicians will often echo common sentiments regarding the need for the technology to ‘just work’. No doubt we have all experienced the frustration of technology not working or not behaving as usual, which we otherwise take for granted. Imagine time-poor clinicians’ frustrations when they are being blocked in their work by slow or non-functioning software.

Throughout my experience implementing clinical information systems, I would say the most common criticism I faced from clinicians was that systems were too slow, too many clicks, too much time waiting. For a clinician, every second counts. A clinician waiting while the spinning wheel of disappointment cycles on the screen, does not endear them to the software, no matter how clever the software.

There are of course many technical components that contribute to the performance of end-user software, from network routing / latency to database performance, to application complexity and data integrations, not to mention the performance of end-point devices being used to access the software.

Why Observability is Critical in Healthcare IT

Ensuring observability across all components is increasingly vital for meeting users’ expectations of enterprise software. In the past, effective observability and digital experience optimization tools were limited or inadequate, but that is no longer true. Today, scalable and affordable solutions exist for optimizing network and application performance, managing digital experiences, proactively detecting and resolving issues, and providing comprehensive, end-to-end monitoring and management of enterprise systems. Organizations now have no justification for neglecting these investments, especially in critical fields like healthcare, where every minute counts in patient care, and where both clinician and patient experiences should be optimized to the highest standard.

Dr. Thatcher is CEO and Founder of Strategance Group, a firm specialising in digital strategy, risk and governance services to assist organisations with their digital investments. Dr.Thatcher is a published author and has held senior executive roles in large public sector and private sector organisations. Notable roles included Chief Technology Officer of the Australian Digital Health Agency, Chief Health Information Officer for Queensland Health; CEO of eHealth Queensland; and Chief Information Officer and Executive Director Facilities for the Mater Health Group. Dr. Thatcher was also formerly a Professor of Digital Practice in the QUT Graduate School of Business.