Augmented reality tech to deliver faster ultrasound scan results, help with surgical planning

SINGAPORE – Patients going for ultrasound scans of injuries or tumours can sometimes wait for weeks for their results, but 5G-enabled devices will soon be able to speed up the process to within a single visit.

Doctors can soon see scans in real time through an augmented reality (AR) visor that helps them to quickly diagnose injuries or get an update on the status of tumours.

The quick scan, which will be trialled at the National University Health System (NUHS) this year, is achieved by swiping a wireless ultrasound probe across a patient’s body. Captured images are streamed to powerful computers that show the results on the doctor’s AR headset – a Microsoft HoloLens 2 – in an instant.

NUHS assistant group chief technology officer Gao Yujia said 5G-enabled tools could help speed up the thousands of routine thyroid cancer check-ups each year, which can take weeks to schedule. If necessary, patients will be transferred to specialised departments.

Dr Gao, 36, who is also a surgeon with the National University Hospital, said: “Around 80 per cent of thyroid ultrasound scans are done for regular surveillance. So this frees up resources for more urgent scans and also reduces the anxiety for patients as they don’t have to wait so long for their results.”

The ultrasound probe was one of several 5G tools presented by Microsoft and Singtel at a media showcase of a new edge computing service for enterprises.

Edge computing refers to processors and data storage being brought closer to the devices they serve, instead of the data being accessed through the cloud from data centres far away, which could cause delays. This is critical to enabling autonomous vehicles to drive safely and giving doctors the confidence to perform surgery from remote locations.

Enterprises can tap Singtel’s multi-access edge compute (MEC) service to develop 5G-enabled applications, such as virtual reality, digital in-store experiences and remote patient care.

AR technology has already been used on about 100 occasions at NUHS to plan for and assist with surgery and educate patients over the past two years, said Dr Gao.

He uses an AR headset during surgery, glancing at a 3D rendering of the patient’s organs to assist with the procedure. This is believed to be the first use of AR in the operating theatre in the region.

While operating, he does not need to handle any electronic tools to interact with the AR projection, as the HoloLens is able to understand hand gestures, like pinching to zoom in and rotate a 3D model of a liver.

In patient consultations, NUHS doctors and patients don the headsets as they go over surgical procedures, which are presented using 3D models to help patients visualise the operation and understand the potential risks, said Dr Gao.

The quick scans will also soon be used to trace veins that are hard to see, increasing a doctor’s accuracy when drawing blood. A projection of the patient’s veins will be displayed over his arm on the transparent AR visor, helping the doctor to locate them easily.

This is helpful for cancer patients, who may have thinner veins, and patients like young children, who might be uncomfortable with being jabbed, said Dr Gao.

Plans are under way to roll out the healthcare solution commercially later in 2023, said Singtel, adding that NUHS is among the first users of its MEC service for the healthcare sector.

The telco also showcased a concept for a conversational 3D avatar that can be used for concierge services.

In another possible use case, companies that deploy surveillance cameras can also use the service, which is able to scan security footage to detect and flag accidents or breaches in real time.