AI uncertainty maps could improve brain tumor monitoring

Artificial intelligence can measure brain tumors on MRI scans in three dimensions, potentially offering a more accurate picture of tumor progression than conventional measurements. However, automated segmentation is not infallible. Researchers at the University of California, San Francisco (UCSF) have therefore developed a deep learning framework that not only identifies meningiomas and calculates their volume, but also shows clinicians how certain the AI is about its assessment.

The approach uses Evidential Deep Learning (EDL) to generate interpretable uncertainty maps alongside tumor segmentations. According to the researchers, making uncertainty visible could increase confidence in AI assisted imaging and support safer use in clinical practice.

Beyond conventional 2D measurements

MRI is an important tool for diagnosing and monitoring brain tumors. In current clinical practice, tumor size is often assessed subjectively or through simplified two dimensional measurements. This can be problematic for tumors with irregular shapes or slow growth, such as meningiomas. Three dimensional volumetric analysis can provide a more complete assessment of tumor burden and subtle changes over time.

Deep neural networks can automate this process by delineating tumors throughout an MRI scan. Yet the reliability of these segmentations can vary considerably. Tumor location and shape, image quality and changes following treatment can all make tumor boundaries more difficult to determine.

Most AI systems do not explicitly communicate this uncertainty. The UCSF researchers wanted to develop a method that quantifies it, allowing clinicians to distinguish between areas where the algorithm is confident and regions where its assessment is less certain. The team focused on meningiomas, the most common type of primary brain tumor. Some have clearly defined borders, while others lie close to anatomical structures that make their boundaries difficult to distinguish.

AI shows where it is uncertain

The researchers trained their deep learning framework using 1,655 MRI scans from 788 patients. The dataset also included postoperative scans. These are particularly challenging because changes caused by treatment can resemble tumor tissue, resulting in greater uncertainty. The researchers subsequently evaluated different combinations of AI models on an independent test set containing 68 MRI scans from 43 patients. They compared the uncertainty maps generated by the system with ambiguous regions identified by neuroradiologists.

The framework achieved high segmentation accuracy, while areas identified by the AI as uncertain corresponded well with regions that specialists also considered ambiguous. The system additionally generated calibrated estimates of tumor volume, meaning its reported level of uncertainty closely reflected the reliability of its measurements. External validation involving 353 patients showed that the approach also performed well on data from outside the institution where the model was developed.

Building trust in medical AI

According to Andreas Rauschecker, UCSF assistant professor of radiology and co chief of Intelligent Imaging Research, some degree of uncertainty will always remain because AI can make segmentation errors. Quantifying that uncertainty provides clinicians with additional information when interpreting automated measurements.

This could be particularly relevant when monitoring tumors over time. Small changes in tumor volume can influence clinical decisions, while unclear tumor boundaries or differences in image quality can affect measurements. An uncertainty aware system could help clinicians determine when an automated result is reliable and when closer human review is warranted. The researchers believe the method could also be applied beyond meningiomas. Calibrated uncertainty estimation may have value for other medical imaging applications in which AI is used to delineate lesions or anatomical structures.

Further validation is still needed. Although the model performed well on an external dataset, it was trained exclusively on data from UCSF. Future studies should therefore include datasets from multiple medical centers and assessments from several specialists. Comparing AI uncertainty with differences between human observers could further clarify how these systems should be integrated into clinical workflows.

Ultimately, explicitly showing what an AI model does and does not know could be an important step toward more transparent and trustworthy use of automated medical imaging.

Tumor monitoring innovation

Last year, researchers at Erasmus MC’s BrainEcho Lab developed an ultrasound device that can monitor brain activity in real time while a patient is moving. Led by neuroscientist Pieter Kruizinga and neurosurgery resident Sadaf Soloukey, the technology can capture up to 10,000 images per second. The system uses a custom helmet and ultrasound probe and was tested in a patient whose skull had partly been replaced with PEEK, a plastic that allows ultrasound waves to pass through more easily than bone. Researchers successfully monitored his brain activity while he walked and performed simple tasks.

Beyond neuroscience research, the technology could eventually support brain surgery by visualizing blood flow in active brain regions and enable long term monitoring after surgery. It may also offer opportunities for neurorehabilitation. However, further research is required before the technique can be used for clinical decision making during surgery.