Letter to the Editor: “COVID-19, AI enthusiasts, and toy datasets: radiology without radiologists”

by Sérgio D. Dias, Pedro E. M. Lopes (pemlopes@fastcompchem.pt)

COVID-19, AI enthusiasts, and toy datasets: radiology without radiologists

Dear Editor,

We would like to follow up on the letter by Tizhoosh and Fratesi with the title “COVID-19, AI enthusiasts, and toy datasets: radiology without radiologists” [1] since it directly relates to our work at FastCompChem, Lda and has implications in the relationship between medicine and technology, The authors address the importance of using well-curated data to train deep learning algorithms for dealing with COVID-19 and affirm the importance of including radiologists in this process. Recent advances in the use of artificial intelligence technologies, especially in medicine, raise many questions that need addressing from the medical and technological communities.

We must start by saying that our view aligns in general with the opinion of Tizhoosh and Fratesi. We develop our work at FastCompChem accordingly. FastCompChem is a startup with the prime goal of developing new technologies for Computer Aided Drug Design using proprietary quantum methodologies. With the onset of the COVID-19 pandemic and with the news, mainly from China [2-5], that radiological techniques were successfully being applied to diagnose COVID-19, we decided to develop deep learning algorithms to diagnose the infection from Chest X-Ray images (CXR). Important in our decision was the work of Jacobi et al. [6]. Our work followed two major lines: (1) to develop a deep learning algorithm to diagnose COVID-19 from toy datasets of CXR images and (2) to conduct a retrospective clinical study in collaboration with medical institutions of the Portuguese SNS to test and validate the deep learning algorithm.

This brings us back to the letter of Tizhoosh and Fratesi. In our own experience, toy datasets are still of value, even when not following all accepted standards as long as they include the patterns associated with the disease, validated by radiologists. In our case we used the compilation of Cohen et al. [7] as source of radiographs. The dataset is certainly not the well curated datasets from the UK’s NHS [8], but it was quickly available allowing the initial development effort. We share the opinion of Tizhoosh and Fratesi regarding professionally curated datasets and soon a retrospective study will begin in the Portuguese National Health System (SNS) to evaluate the real accuracy of our deep learning algorithm and to determine whether it needs to be retrained or adjusted.

We are at a point in time when technology and medicine can join efforts to bring unprecedented benefits to patients and medical professionals. Technology can ease the burden of clinicians through automation of acceptable tasks and can help reach details not accessible to human senses. For instance, medical imaging devices are able to produce 12-16 bit/pixel images, corresponding to 4,096 to 65,536 shades of gray [9]. Since the human eye is able to detect only 700-900 shades of gray, there is likely a wealth of information that is hidden from the human eye but can be detected by computer graphics algorithms and eventually analysed with the help of deep network technologies. We advocate a two-way communication between the clinical and technological worlds. Clinicians should be aware of the latest technologies being developed, while the technology world should engage with clinicians and learn which are their biggest challenges and needs. It is important that new technologies are developed strategically by the combined efforts of medical and technology professionals.

In summary, we fully agree with the ideas of Tizhoosh and Fratesi. However, we would not discard the use of toy datasets if that is the only avenue of development, including training of deep learning algorithms. The true validation can only occur in controlled clinical trials under the supervision of radiologists though. Furthermore, many important technologies are being developed by the technology world that would greatly benefit the medical profession and patient outcomes. The medical professionals should be aware of those developments while the technical partners should engage with clinicians to understand their needs. It is a win-win scenario that will foster innovation, benefiting everybody, especially patients.