Researchers at the Fraunhofer Institute for Manufacturing Engineering and Automation IPA are working on a system that could radically simplify the documentation of ultrasound examinations. Using a 3D camera and AI-based image processing, the system automatically records where and at what angle an ultrasound image was taken in relation to the patient's body. This largely automates a time-consuming and error-prone manual process.
Several million ultrasound examinations are performed in Germany every year. Ultrasound is a widely used diagnostic technique, especially in children, as it avoids exposure to X-rays. However, reporting ultrasound scans is a bottleneck in practice. When doctors find a cyst or tumour, for example, they not only have to measure the abnormality, but also manually record its location. This is usually done by transferring the position of the ultrasound probe to a two-dimensional icon display on the screen of the ultrasound system.
This manual process takes a lot of time in everyday hospital practice, an estimated quarter of the treatment time, and is also prone to inaccuracies. This can be particularly problematic in follow-up examinations, as subtle differences in measurement position or angle can influence the interpretation of images.
Automatic spatial documentation
Within the so-called SonoMap project, Fraunhofer IPA is now developing an alternative. ‘With a 3D camera, we can not only automatically determine where the ultrasound image was taken, but also at what angle,’ says Oliver Gölz, researcher at Fraunhofer IPA. According to him, this additional information makes a visible difference. ‘The side and angle from which the scan is taken influence the image. By including this information in a 3D visualisation, tumours or cysts can be found more quickly and accurately in follow-up examinations.’
The system works as follows: a 3D camera detects the ultrasound probe and simultaneously maps the patient's body surface. AI-based algorithms then determine the exact position and orientation of the probe. This information is linked to a simplified 3D model of the body, which automatically shows where and how the image was taken.
The result is an interactive 3D visualisation that can be saved and viewed from different perspectives. ‘Doctors will soon only need to save the ultrasound image; the spatial documentation will be generated automatically,’ says Gölz. ‘This makes the process faster and potentially more accurate, leaving more time for the patient.’
The road to practical application
According to Fraunhofer IPA, there are currently no comparable solutions on the ultrasound market. A technical demonstration of the system has now been completed. The next step is a clinical study, for which Gölz and his team have submitted an application. Once this has been completed, the researchers want to collaborate with industrial partners to integrate the technology into existing ultrasound systems.
At the same time, attention is being paid to data protection. The researchers are working to ensure that the 3D camera only captures secure images and that patient data is adequately protected.
Gölz looks back with satisfaction on the progress made so far. ‘I am particularly enthusiastic because this technology allows us to respond directly to problems encountered in everyday hospital practice and reduce the workload for doctors.’ If the system lives up to its promise, automatic spatial documentation could become a new standard in ultrasound diagnostics, particularly in paediatrics.
3D ultrasound assists in heart surgery
Accurate imaging is also essential in minimally invasive heart surgery. The VeriSight Pro, from Philips, is an ultra-small ultrasound probe measuring just 3 millimetres that is attached to the tip of a catheter. This probe can be inserted into the blood vessels to provide live images of the heart from the inside, in both 2D and 3D. This gives doctors a more detailed and reliable image than traditional ultrasounds via the oesophagus or chest.
According to Philips, 3D imaging helps doctors make decisions with greater confidence and treat more patients. In the Netherlands, this approach has already been used for some time in children with congenital heart defects. Philips worked closely with doctors on the design to ensure that the probe is intuitive to operate and supports the interpretation of 3D images during complex procedures.
