HERMES

Current Status

The project activities started on 1st September  2020. During the first phase, the user needs have been consolidated though continuous interactions with the involved End Users as well as the partecipation to “on field trials” such as VARDIREX’s joined italian military forces held on 23rd, 24th and 25th September which saw a close collaboration with the 9th Alpine Regiment. 

The main outcomes of the project are rapresented by the three pilots campaign which involves the three different users group. The pilots campaign has started on April 12nd 2021.

The first User Group related to the first pilot campaign is leaded by Bio-Medical Campus University of Rome (UCBM).

The other key actors of this pilot campaign #1 are the UCBM Hospital network (Villa Serena Pescara, St. Maria Goretti Hospital Latina, Papardo Hospital Messina), as point-of-care, Telespazio as a Satellite Communications service provider, e-GEOS as Database service provider and EXPERT-LAB as geospatial information provider.

The objective of this Pilot Campaign is to improve early identification of Covid-19 outbreaks in symptomatic patients and to create an extended population monitoring system to detect potential epidemic outbreaks.

Conclusions:

Pilot # 1 analyzed possibility and usefulness of having a tool for sharing biomedical images in the presence of a scenario such as that of Covid-19. The peculiarities of this scenario is characterized by the presence of a substantially new pathology with a high impact in terms of number of affected patients,  which induces significant limitations to mobility, and which can also be diagnosed through analysis of biomedical images. In such a scenario, a tool such as HERMES has proven to be useful especially in the early stages of the spread of the epidemic event. In fact, in the incipient phase of the phenomenon, only a very small number of specialists have the skills to carry out the diagnosis, skills which, moreover, are refined only through the examination of a significant number of cases. This entails the opportunity to have specialized centers to which biomedical images converge for their examination. Centers in which it is essential, as verified with the Hermes project, to have Artificial Intelligence tools to support the diagnosis phase in light of the significant increase in workloads.

Specifically, the experience of Pilot # 1 allowed to confirm the preliminary data reported in the literature on the effectiveness of diagnosing Covid-19 through CT image analysis and how in this case Artificial Intelligence represents valid tool having rates of correct diagnosis above 85%

Although chest CT scans have shown high sensitivity in diagnosing COVID-19, compared to RT-PCR tests, chest CT scans alone are not sufficient to detect COVID-19 but can be utilized as a complement to other tools for diagnosing patients with COVID-19. Chest CT is really important as a staging modality, able to identify the phase of the disease and to define its prognosis. Another key role of chest CT is to assess the response to therapy and the complications of COVID-19 on patients' lungs.

Similarly, the experiment confirmed that the automatic diagnostic capacity of Covid-19 based on RX images is modest. In the experimentation it stood at 60%, slightly lower than the values reported in the literature (around 67%). This data is justified by the reduced quality of the X-ray images when compared with CT and by the intrinsic difficulty for the diagnosis.

The use of AI systems, offering an automatic  evaluation of chest CT or chest X-ray, in suspected COVID-19 patients, are able to help radiologists to reduce their workload, to produce a quick and accurate discrimination between normal and suspected COVID-19 individual, to evaluate the phase and assess the prognosis of COVID-19. During the pilot the effectiveness of the AI tools to provide adequate diagnosis support will be investigate comparing the output of the AI tools with the diagnosis provided by the radiologist. The radiologists consider useful the presence of an AI tool to support analysis of CT images, even if they stress that the final diagnosis must be exclusive responsibility of the radiologist. Morever, AI is essential for managing sudden work peaks and as a support tool especially for less experienced radiologists by accelerating their learning curve.

Video of pilot#1 is available at this link: https://youtu.be/Ap0SgEfUj2Q

The second User Group is leaded by Italian Red Cross (IRC). The other key actors are Telespazio as a Satellite Communications service provider, e-GEOS as the Database service provider, EXPERT-LAB as IoT Healthcare sensor provider and VoiceWise as voice screening solution provider.

Every pre-triage tent or mobile hospital is equipped with one or more IoT Wearable Healthcare Sensors kit. This is a modular system that could manage multiple devices to measure biometric parameters such as blood pressure, blood oxygen saturation, heart rate, and automatically send data to the Cloud Data Center.

Furthermore, the voice screening solution uses a smartphone with VoiceWise APP in order to perform the voice sample acquisition and analysis of voice samples.

The objective of this Pilot is to provide efficient support for screening activities to early identify possible Covid-19 outbreaks.

Conclusions:

The AI/ML model has evolved over time, with version 1.7 running on most of December’s samples and several sub-versions of v1.8 on the way for possible future expansions. Due to the lower-quality and heterogeneity of on-site data collected in Fiumicino and Cagliari, only controlled recordings from the Hospital of San Matteo, Pavia are used in the training phase. A much greater adherence of positive subjects and the construction of noise-free, coherent hubs could make the data collected on-site usable for training purposes. The test accuracy of the AI/ML model on Pavia data (selected dataset) remains 82.5% as stated before, given that Pavia historically has had more accurate recordings and more positive test cases. The accuracy on-site for December 2021 is 96%. The audio quality of the on-site samples is deeply different between the two locations: Fiumicino is a tent, and it has been acoustically treated by a Voicewise sound engineer, while the van from Cagliari produces noisier files, especially regarding static and machinery noise.

The final accuracy of the whole HERMES on-site trial is 97%, evaluated on all usable samples, for a total of 241 instances. Although most of the percentages reported exceed the KPIs, it is necessary to state that data are heavily unbalanced towards the “negative” class, due to environmental conditions. Plus, most of the positive are generally asymptomatic at the moment of recording, and the Omicron variant, which is the most common within the trial data for timing reasons, bears a relevantly different symptomatology than previous variants, especially early ones (Phase 1), and can be considered to be generally milder and less aggressive on the lungs. With these premises, it is reasonable that the AI/ML models started showing a trend of biasing towards the “negative” class. In fact, a higher specificity is reported, which some studies deem preferable for large-scale preliminary COVID tests, due to the factual prevalence of COVID-negatives and to the possible consequences of declaring a large number of false positives. However, these results also prove that the AI/ML models, originally sensitive to symptomatic positive subjects, are shifting towards a high specificity. We still consider the identification of positives subjects crucial, and the main problem in these regards is the scarcity of subjects along with the ever-changing symptomatology.

Nevertheless, Voicewise is currently working on the knowledge-based exploiting the existing datasets, with experiments on a stricter pre-processing of audio files, on the use of more homogeneous subsets of the training set and the employment of different Machine-Learning models, possibly shallower in order to induce a “slack” which allows for a more generalized inference instead of near-perfect results on the current test set. Although the whole experiment suggested the necessity of separate studies for asymptomatic voices, and COVID variants and/or stages, we still argue that a quick collection of a large number of data is the key point in such a situation, possibly forgoing other verbose and less immediate requirements.

As a conclusion the overall KPI is obtained, and we strongly feed that we can enhance the KPI in the near future by upgrading the AI/ML model covering the new Omnicom variation and further new variations/mutations

Video of pilot#2 (FCO) is available at this link: https://youtu.be/qcMI-jfaXaM

The third User Group is leaded by University of Chieti D’Annunzio. The other key actors of this Pilot #3, Telespazio as a Satellite Communications service provider, e-GEOS as Database service provider and EXPERT-LAB as IoT Tracking sensor solution.

The objective of this Pilot #3 is the tracking of the patients’ medical swabs for eHealth (digital healthcare) applications, based on a modular system that could support tracking of the delivered medical swabs from patients’ healthcare facilities to UdA Analysis Lab Covid-Test Centre for their test analysis reports.

The healthcare staff from Analysis Lab will access to Hermes dashboard to track the medical swabs and timely plan the number of samples to be analyzed.

Video of pilot#3 is available at this link: https://youtu.be/J2uUWdFe_IE

Conclusions: 
(Prof. Liborio Stuppia - DirectorLab of Molecular Genetics – COVID19 - Center for Advanced Sciences and Technology G. d’Annunzio University of Chieti-Pescara).

The original project foresaw that the tracking device would be delivered by 118 operators, who would manage the organization of the shipments. After a short time, it became clear that, due to their excessive commitments, the 118 operators were not able to efficiently manage the tracking of the swabs, and it was therefore decided to identify a single hospital department, namely the Breast-Unit of the hospital of Ortona to directly organize the tracking operations.

Due to the transition from one organizational system to the next, the number of tracings performed in the summer months was drastically reduced, also as a result of the holiday period.

Moreover, with a new organizational method, the problem arose that, as only one device was available, each time a delivery was made there was the necessity of the return of the device to the Breast-Unit, which was about 30 km away, making it impossible to operate on a daily basis.

Following the purchase of a second device, we tried to increase the weekly shipments, but the main problem always remained the difficulty of retrieving the device to make a new shipment from the identified point of Ortona. At the conclusion of the work, we could see that the positive aspects highlighted during the trial of the project are the following:

• Reflection on the excellent potential of the system and its usefulness

• Ease of use

• Very little training required

• The system fits very well with the logistics of the process.

On the other hand, the only possible criticality would remain the unavailability of the transporter that implied the impossibility to follow the logistic procedure of use made explicit during the project phase.

In spite of the problems encountered, it should be noted the great interest of our laboratory in the use of technology for other and interesting purposes, and we believe that the tested system allows to digitize well the logistic process, ensuring all the information that today is not properly managed. Moreover, the use lends itself well to various applications, even in other similar areas for the health sector.