How a digital twin can save lives

Sending artificial intelligence to work in the “text mines”

Colon cancer continues to be treated largely on the basis of these guidelines. In some patients, however, tumors continue to spread in the body. Particularly in these cases, alternative treatment methods that are not yet included in the guidelines may be of interest. For the doctors treating these patients, this would mean scouring scientific publications on clinical studies in order to track down more-expedient therapies.

This is a complex undertaking – and another area where artificial intelligence can provide targeted support. In the BMBF-funded Electronic Patient Path (EPP) research project, Jil Sander, business unit manager at the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS, is part of a team working to create a suitable text-mining solution: “AI searches public abstracts of medical publications for therapies whose effectiveness for patient groups can be estimated on the basis of certain biomarkers. The relevant publications are then recommended to the doctor as reading material for potential treatments for colorectal cancer.” A biomarker might be a gene that has mutated in the tumor cells of some, but not all, patients. And it may be that chemotherapy works well for the group with the mutated gene but has hardly any effect in the second group. In this way, doctors can form specific patient groups and search for the best treatment options for a particular group.

But the program doesn’t just search for keywords – a simple search algorithm could do that. Instead, it learns to use context in order to identify certain classifiers, known as entities, and their relationships. This makes it possible, for instance, to identify whether a certain therapy has actually been used for a clinical picture – rather than merely listing the therapies and clinical pictures within a text. The system can also recognize entirely new entities, such as novel treatment options that have not yet been recorded in any database and thus have not yet been keyworded. But the project encompasses more than just these text-mining solutions. Its goal is to develop an Electronic Patient Path (EPP) – an integrated system for therapy guidance for colorectal cancer that extends beyond the guidelines. This is why the scientific partners – Ruhr-Universität Bochum, University of Bonn, Hamm-Lippstadt University of Applied Sciences and Fraunhofer IAIS – are collaborating with a clinical partner, the Universität zu Lübeck and the University Medical Center Schleswig-Holstein, Lübeck campus. The final results are still out. In the long term, however, it is hoped that the program goals will help doctors pinpoint the most relevant new studies and thereby drive up the treatment success rate for cancers – by, for example, continuously refining the EPP method in the MED2ICIN lighthouse project, both for colorectal cancer and for other clinical pictures. But here, too, the artificial intelligence behind the integrated system is meant to support doctors, not replace them.

The head and neck region is particularly challenging

Now, however, artificial intelligence can also automatically detect the tumor. As Wesarg explains, “Our software tool localizes and labels the tumor in the computer tomography images, presents it in 3D and analyzes the corresponding image data.” The system is based on neural networks and was trained with data in which the tumor was labeled manually. It then used this data to generate a model. Additional data is added from the head and neck atlas, such as the information that the larynx is completely healthy-looking, so the system doesn’t need to look for the tumor there. The results of the head and neck atlas thereby provide a preselection.

How is brightness distributed within the tumor? Is there anything that isn’t noticeable when a human looks at it? The tool uses various descriptive parameters to answer these questions. In total, with the appropriate software, more than a hundred parameters of this kind can be extracted from the images of a head and neck tumor.

Faster, cheaper and gentler than a biopsy

Initial results show that, with this approach, CT images can even provide information that once could only be obtained through a surgical procedure followed by laboratory analysis of the extracted tumor tissue. “So it’s conceivable, for instance, that a correlation could be found between the intensity pattern within the tumor region and a cell abnormality detected in the lab. With enough patients, it could one day be possible to infer – with statistical certainty – pathological cell changes on the basis of the appearance of the tumor in the image data.” Thus, so the theory goes, it will soon be possible to use artificial intelligence to draw conclusions regarding tissue markers, obviating the need for an actual biopsy. This is easier not only on patients but also on health insurance companies’ budgets. On top of that, the results are available much more quickly than they would be for a biopsy with a lab analysis of the extracted tissue.

Parts of this technology are already being used in initial test runs at the HNO clinic of the University Hospital of Düsseldorf. The doctors there are using the technology to retroactively analyze patient data and review cohort assignments. In the months ahead, this test is expected to reveal how the AI findings correlate with empirical knowledge, thus marking the first step toward cohorting – and onward toward treatment tailored to the individual patient.

The long-term aim is to personalize medical care – to identify the therapy with the highest probability of success for each patient. To achieve this, the algorithms developed for the head and neck region could also be extended to other types of cancer. For this, however, the algorithm needs to have the relevant information as to which structures it should look for in the image data. This is because tumors in the head and neck region have different markers than, say, lung tumors.

In collaboration with MedCom GmbH in Darmstadt, a Fraunhofer IGD spin-off, the researchers also want to begin this process as early as the initial diagnosis. In the BMBF ECHOMICS project, they are using artificial intelligence to analyze ultrasound images of the lymph nodes in a process analogous to a biopsy. This is because a permanent enlargement of the lymph nodes may indicate the presence of a tumor in the body. This would enable doctors to detect tumors much sooner than is currently possible, thus facilitating swifter treatment and improving the chances of success.

Analyzing the “inherited” immune system

When radiation or chemotherapy is not successful in treating leukemia or lymphoma, there is usually only one chance for recovery: a bone marrow or blood stem cell transplant. But here, too, the chances of success are very small. Currently, the majority of patients die despite the transplant. This is because chemotherapy kills all blood stem cells – and thus also all the white blood cells that make up our immune system. In a transplant, the patient therefore “inherits” the donor’s immune system. Yet this may turn against the patient. And it is also possible that the new immune system doesn’t recognize latent pathogens in the patient’s body, so that the corresponding disease erupts again.

The researchers working on the XplOit project at the Fraunhofer Institute for Biomedical Engineering IBMT now aim to use artificial intelligence to predict whether the new immune system will give rise to these kinds of problems. Will the patient tolerate the transplant? What is the probability that they will survive? And, if they do, what is the probability that the cancer will spread again? “Our XplOit platform enables earlier detection and treatment of life-threatening complications than is currently possible,” says project coordinator Stephan Kiefer from Fraunhofer IBMT.

To teach the neural networks what they have to look for, the scientists trained them with representative data they received from their project partners – the Clinic for Bone Marrow Transplants, the Essen University Hospital Institute of Virology, and the Clinic for Internal Medicine I and Institute of Virology at Saarland University. With their simulation tools now trained, the researchers have been working on clinical validation since March 2019. For one year, they will feed in the data of patients currently being treated, compare the results of the simulation, the doctors’ assessments and the actual course of the disease, and thus refine the simulations. In other words, they are retraining the neural networks. “We are certain that these models can offer physicians solid indications of probable complications,” says Kiefer, summarizing their findings to date. Then, so Kiefer hopes, significantly more people might survive this type of transplant.

Early diagnosis by optical coherence tomography

One of the keys to the treatment of cancer – perhaps the most important of all – is the early detection of tumors. Optical coherence tomography (OCT) could play a major role here. Its resolution is between ten and 100 times that of ultrasound. But there is a drawback: the lack of suitable expertise in the day-to-day clinical work in many medical disciplines. Researchers at the Fraunhofer Institute for Production Technology IPT and at Tokyo Women’s Medical University (TWMU) are now hoping to change that with their OCTmapp research alliance. “It is hoped that artificial intelligence will assist doctors with analysis, as well as helping them better understand signal generation – especially the formation of artifacts,” says Niels König, department head at Fraunhofer IPT.

In June, the researchers at Fraunhofer IPT took two OCT systems to Japan, where the preclinical study – testing the technology in relevant applications – will take place. Their aim is to determine what is feasible and how the technology can be used for joint research with Japanese partners. König expects that it will be possible to integrate the system into TWMU’s Hyper Smart Cyber Operating Theater (HyperSCOT) in around five years – and, in the long term, that it will help diagnose cancer e