System for quantitative diagnosis of COVID-19 associated Pneumonia based on Superpixels with deep learning and chest CT
DOI:
https://doi.org/10.55905/oelv21n9-022Keywords:
lung infection, computer vision system, respiratory syndrome, Coronavirus, machine learningAbstract
COVID-19 is a disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) that can lead to complications such as acute respiratory distress syndrome, acute heart injury and secondary infections in a relatively high proportion of patients and, consequently, lead to significant mortality. The definitive diagnosis of COVID-19 is performed by real-time Polymerase Chain Reaction (RT-PCR). However, as RT-PCR results usually take longer to be completed than computed tomography (CT), the latter has had an important role in the detection of patients infected with COVID-19. A rough estimate of the extent of lung involvement by the disease is also important and considered an additional criterion for deciding on discharge or hospitalization. Recent research has adopted deep neural networks and other machine learning approaches to detect the presence of lung infection caused by COVID-19. However, the extent of lung involvement (volume) caused by the disease has been only superficially investigated. In this work, we created an end-to-end computer vision system to automatically quantify the Percentage of Infection (POI) in chest CT images of COVID-19 cases confirmed by laboratorial analysis. We evaluated the performance of three well-known deep neural networks, trained using three different strategies: (1) no transfer learning with randomly initialized weights; (2) transfer learning using with ImageNet weights without fine-tuning; and (3) transfer learning with weight fine-tuning. Data augmentation and dropout were used during training to reduce overfitting and increase the generalization capacity of the models. Our approach consists of segmenting a chest CT with the SLIC Superpixels method and classifying each segment (superpixel) into a specific class (COVID or non-COVID). We used the weights of the best deep neural network to classify the superpixels and quantify the COVID-19 infection by calculating the POI on chest CT. The results indicate that deep learning models can be successfully used to support radiologists in the quantitative diagnosis of lung infection caused by COVID-19, reaching an accuracy of up to 98.4% with Inception-Resnet-v2 architecture.
References
Xu, X. et al. A deep learning system to screen novel coronavirus disease 2019 pneumonia. Engineering DOI: https:
//doi.org/10.1016/j.eng.2020.04.010 (2020).
WHO. Coronavirus disease 2019 (covid-19) situation report – 51. Tech. Rep., World health organization (2020).
Qureshi, S. A. & Rehman, A. u. Optical techniques, computed tomography and deep learning role in the diagnosis of covid-19 pandemic towards increasing the survival rate of vulnerable populations. Photodiagnosis Photodyn. Ther. 31, DOI: https://doi.org/10.1016/j.pdpdt.2020.101880 (2020).
Shoji, H. et al. Structured thoracic computed tomography report for covid-19 pandemic. Einstein (Sao Paulo) 18, DOI: http://www.doi.org/10.31744/einstein_journal/2020ed5720 (2020).
Shan, F. et al. Lung infection quantification of covid-19 in ct images with deep learning, DOI: https://doi.org/10.1002/mp. 14609 (2020).
Zhang, H.-t. et al. Automated detection and quantification of covid-19 pneumonia: Ct imaging analysis by a deep learning-based software. Eur. J. Nucl. Medicine Mol. Imaging 47, DOI: https://doi.org/10.1007/s00259-020-04953-1 (2020).
Zhao, W., Zhong, Z., Xie, X., Yu, Q. & Liu, J. Relation between chest ct findings and clinical conditions of coronavirus disease (covid-19) pneumonia: A multicenter study. Am. J. Roentgenol. 214, 1–6, DOI: https://doi.org/10.2214/AJR.20. 22976 (2020).
Bressem, K. et al. Comparing different deep learning architectures for classification of chest radiographs. Sci. Reports 10, DOI: https://doi.org/10.1038/s41598-020-70479-z (2020).
Asif, S., Wenhui, Y., Jin, H., Tao, Y. & Jinhai, S. Classification of covid-19 from chest x-ray images using deep convolutional neural networks. medRxiv DOI: https://doi.org/10.1101/2020.05.01.20088211 (2020). https://www.medrxiv. org/content/early/2020/06/18/2020.05.01.20088211.full.pdf.
Sarker, L., Islam, M., Hannan, T. & Ahmed, Z. Covid-densenet: A deep learning architecture to detect covid-19 from chest radiology images. Preprints DOI: https://doi.org/10.20944/preprints202005.0151.v1 (2020).
Hassantabar, S., Ahmadi, M. & Sharifi, A. Diagnosis and detection of infected tissue of covid-19 patients based on lung x-ray image using convolutional neural network approaches. Chaos, Solitons & Fractals 140, 110170, DOI: https://doi.org/10.1016/j.chaos.2020.110170 (2020).
Rahimzadeh, M. & Attar, A. A modified deep convolutional neural network for detecting covid-19 and pneumonia from chest x-ray images based on the concatenation of xception and resnet50v2. Informatics Medicine Unlocked 19, 100360, DOI: https://doi.org/10.1016/j.imu.2020.100360 (2020).
Bassi, P. & Attux, R. A deep convolutional neural network for covid-19 detection using chest x-rays. CoRR (2020).
Loey, M., Smarandache, F. & Khalifa, N. E. Within the lack of chest covid-19 x-ray dataset: A novel detection model based on gan and deep transfer learning. Symmetry 12, 651, DOI: https://doi.org/10.3390/sym12040651 (2020).
Liew, C., Quah, J., Goh, H. & Venkataraman, N. A chest radiography-based artificial intelligence deep-learning model to predict severe covid-19 patient outcomes: the cape (covid-19 ai predictive engine) model. Medrxiv DOI: https:
//doi.org/10.1101/2020.05.25.20113084 (2020).
Meng, Q. et al. Role of novel deep-learning-based ct used in management and discharge of covid-19 patients at a “square cabin” hospital in china. Res. Sq. DOI: https://doi.org/10.21203/rs.3.rs-28201/v1 (2020).
Butt, C., Gill, J., Chun, D. & Babu, B. Deep learning system to screen coronavirus disease 2019 pneumonia. Appl. Intell.
DOI: https://doi.org/10.1007/s10489-020-01714-3 (2020).
Song, Y. et al. Deep learning enables accurate diagnosis of novel coronavirus (covid-19) with ct images. Medrxiv DOI: https://doi.org/10.1101/2020.02.23.20026930 (2020).
Weikert, T. et al. Treatment intensity stratification in covid-19 by fully automated analysis of pulmonary and cardiovascular metrics on initial chest ct using deep learning. Res. Sq. DOI: https://doi.org/10.21203/rs.3.rs-35878/v1 (2020).
Li, L. et al. Artificial intelligence distinguishes covid-19 from community acquired pneumonia on chest ct. Radiology 296, 200905, DOI: https://doi.org/10.1148/radiol.2020200905 (2020).
Yan, L. et al. An interpretable mortality prediction model for covid-19 patients. Nat. Mach. Intell. 1–6, DOI: https:
//doi.org/10.1038/s42256-020-0180-7 (2020).
Uddin, M. I., Ali Shah, S. & Al-Khasawneh, M. A novel deep convolutional neural network model to monitor people following guidelines to avoid covid-19. J. Sensors 2020, 1–15, DOI: https://doi.org/10.1155/2020/8856801 (2020).
Mei, X. et al. Artificial intelligence–enabled rapid diagnosis of patients with covid-19. Nat. Medicine 26, 1–5, DOI: https://doi.org/10.1038/s41591-020-0931-3 (2020).
yu, Z. et al. Rapid identification of covid-19 severity in ct scans through classification of deep features. Biomed Eng Online
DOI: https://doi.org/10.21203/rs.3.rs-30802/v1 (2020).
Tuncer, T., Dogan, S. & Özyurt, F. An automated residual exemplar local binary pattern and iterative relieff based corona detection method using lung x-ray image. Chemom. Intell. Lab. Syst. 203, 104054, DOI: https://doi.org/10.1016/j.chemolab. 2020.104054 (2020).
Farhat, H., Sakr, G. E. & Kilany, R. Deep learning applications in pulmonary medical imaging: recent updates and insights on covid-19. Mach. vision applications 31, 53–53, DOI: https://doi.org/10.1007/s00138-020-01101-5 (2020). 32834523[pmid].
Hussain, A., Bouachir, O., Al-Turjman, F. & Aloqaily, M. Ai techniques for covid-19. IEEE Access PP, 1–1, DOI: https://doi.org/10.1109/ACCESS.2020.3007939 (2020).
Hartigan, J. A. & Wong, M. A. Algorithm as 136: A k-means clustering algorithm. J. Royal Stat. Soc. Ser. C (Applied Stat.
, 100–108 (1979).
Achanta, R. et al. Slic superpixels compared to state-of-the-art superpixel methods. IEEE transactions on pattern analysis machine intelligence 34, DOI: https://doi.org/10.1109/TPAMI.2012.120 (2012).
Soares, E., Angelov, P., Biaso, S., Higa Froes, M. & Kanda Abe, D. Sars-cov-2 ct-scan dataset: A large dataset of real patients ct scans for sars-cov-2 identification. medRxiv DOI: https://doi.org/10.1101/2020.04.24.20078584 (2020). https://www.medrxiv.org/content/early/2020/05/14/2020.04.24.20078584.full.pdf.
Tetila, E. C. Covid20k2c superpixels dataset: Banco de imagens de superpixels criado a partir de to- mografias computadorizadas para infecção por sars-cov-2 (covid-19). github https://github.com/EvertonTetila/ COVID20K2C-Superpixels-Dataset (2020).
Pedregosa, F. et al. Scikit-learn: Machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011).
Huang, G., Liu, Z., Van Der Maaten, L. & Weinberger, K. Q. Densely connected convolutional networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2261–2269 (2017).
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. Rethinking the inception architecture for computer vision. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), DOI: https://doi.org/10.1109/CVPR.2016.308 (2016).
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770–778 (2016).
Bottou, L. & Bousquet, O. The tradeoffs of large scale learning. In Platt, J., Koller, D., Singer, Y. & Roweis, S. (eds.)
Advances in Neural Information Processing Systems, vol. 20, 161–168 (Curran Associates, Inc., 2008).
Swain, M. & Ballard, D. Color indexing. Int. J. Comput. Vis. 7, 11–32 (2004).
Dalal, N. & Triggs, B. Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 1, 886–893 vol. 1 (2005).
Haralick, R. Haralic rm.statistical and structural approaches to texture. proc ieee 67:786-804. Proc. IEEE 67, 786 – 804, DOI: https://doi.org/10.1109/PROC.1979.11328 (1979).
Ojala, T., Pietikainen, M. & Maenpaa, T. Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on Pattern Analysis Mach. Intell. 24, 971–987 (2002).
Ming-Kuei Hu. Visual pattern recognition by moment invariants. IRE Transactions on Inf. Theory 8, 179–187 (1962).
Ferreira, A. S., Freitas, D. M., da Silva, G. G., Pistori, H., & Folhes, M. T. (2017). Weed detection in soybean crops using ConvNets. Computers and Electronics in Agriculture, 143, 314-324. http://git.inovisao.ucdb.br/inovisao/pynovisao
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
