Ability to identify bone fractures tells us something important about AI

As an IT researcher, I’ve been following with great interest the stories in the media about the unfortunate people falling and injuring themselves in Vestre Viken, and the futuristic technology that awaits them when they arrive at hospital.

Since last year, the Vestre Viken Health Trust has been permitting its hospitals to use artificial intelligence (AI)  to identify which of these patients are suffering from bone fractures. Recently, the radio news programme NRK Dagsnytt reported that use of the AI triage tool had in total saved these patients as many as 115 days sitting in hospital waiting rooms.

As the report was going out, patient number 10,000 had just undergone an AI-assisted examination.

Lessons learned have value far beyond the health sector

The procedure adopted by the Vestre Viken Health Trust clearly represents one of the biggest achievements in the practical use of AI in Norway to date.

The lessons learned from this success should have value far beyond the health sector. In short – and at least for now – delegate the easier, lower-risk tasks to AI algorithms and leave the rest to human specialists.

History repeating itself

It happens every time. When a new technology bursts onto the scene, many jump to the conclusion that we humans will become surplus to requirements. However, history keeps telling us that the tasks which become automated are always those that either follow set rules or patterns, or do not require high levels of human judgement. At present, this is very much the case with our application of AI.

The Vestre Viken Health Trust has applied this principle in a remarkable way. Its use of the AI triage tool not only shortens waiting times for patients who are not suffering from fractures. It also relieves the workload on hospital radiologists.

This experience should be of interest to many businesses in a variety of sectors, not least in the wake of a recent survey conducted by the Confederation of Norwegian Enterprise (NHO). 

The survey reveals that one of the barriers to the more widespread use of AI in Norwegian businesses is “a lack of insight into how AI tools can solve problems”.

Three key benefits

It is precisely this type of insight that the Vestre Viken Health Trust has succeeded in acquiring. Its hospitals have chosen to delegate to the AI algorithms a specific task that is important but, above all else, also simple and which needs to be performed frequently.

This has resulted in three key benefits:

• Patients who do not present with fractures obtain an answer within a few minutes and can go home, perhaps with bandages to support a sprain. They avoid having to wait for hours while doctors plough through a long ‘queue’ of X-ray images. 
• Doctors are left with more time to concentrate on more complicated cases. 
• It is easier to explain how an AI tool comes to a decision when it is asked to identify a bone fracture than when it is delegated a more complex task. Such a strategy reduces the problem that we might characterise as a lack of trust in the ‘black box’ that AI represents. The difficulty that experts face when an AI tool is delegated complex tasks, is that it is impossible to know the weights that the algorithms assign to the data they receive.

Moreover, hospital doctors review all X-ray images the day after they are taken with the aim of checking that the AI tool has not misidentified patients with fractures.

Over-optimistic prophecies

The task delegated to the AI tool in Vestre Viken is a straightforward triage process, sorting patients into one of two categories – ‘fracture’ or ‘no fracture’.

This is a far cry from the complex categorisation processes involved in disease diagnosis, and thus also from the prophecies of many over-optimistic promoters of AI tools.

In 2016, the British-Canadian Geoffrey Hinton, who is a computer scientist, cognitive psychologist and the so-called ‘Godfather of AI’, predicted that in five years we would no longer have any need for radiologists. According to Hinton, we may as well stop training them because AI would soon be able to do a better job than they could.

However, recent research into AI-assisted diagnostics, including cases of brain and early-stage breast cancer, has dampened this earlier optimism.

Disagreeing with AI-assisted diagnoses

Making diagnoses in such cases is a complex process, and it is virtually impossible to supply AI algorithms with all the information that doctors normally use to assess a patient. If a doctor is unsure, he or she can always consult a patient’s records and search for any earlier indications of the disease.

The diagnostic AI tools that have been investigated by research scientists are not permitted to take decisions in isolation. The idea is that they supply suggestions which are then quality assured by specialist doctors.

However, when doctors are attempting to identify the cause of a troubling symptom, which is a far more complex process than looking for the difference between a bone with fracture and one without, they frequently arrive at different conclusions to those arrived at by the AI tool.

Nor will the doctors be aware of exactly what aspects of the available data the AI tool has emphasised. In such cases, they will refrain from giving any weight to the tool’s recommendations

Just like in business

These problems are in some ways similar to those being grappled with by many businesses currently attempting to test their AI tools.

My PhD studies involve looking into what business experts are looking for in terms of explanations that will allow them to have more confidence in the answers that AI tools offer for their problems.

At the same time, AI researchers are working very hard to throw some light on what the algorithms inside all these AI ‘black boxes’ are thinking. Until the time when they reach their conclusions, I will offer the following advice to business leaders currently in the process of experimenting with AI: Do what the hospitals in Vestre Viken are doing and – at least for now – delegate your easier, lower-risk tasks to AI algorithms and leave the rest to your expert employees.

This article was first published in the financial daily Dagens Næringsliv on 24 January 2024 and is reproduced here with the permission of the paper.