COVID-19 computed tomography reporting systems: inter- rater and inter-system agreement study

Abstract

Aim: The aim of this study was to investigate agreements between COVID-19 reporting systems and radiologists.
Materials and Methods: A total of 100 laboratory-confirmed COVID-19 cases (49 males, 51 females; age range 19-88 years) were
retrospectively collected. Firstly, computed tomography (CT) images were evaluated by two radiologists independently and blinded
to clinical notes and laboratory and radiological reports and they gave their impressions independently according to four COVID-19
reporting systems, then all CTs were interpreted again by the two radiologists for extracting CT features at the same session by
consensus.
Results: Bilateral, lower lobe, peripheral, dorsal and multifocal lung involvements were predominantly seen, and ground-glass
opacities (GGOs) were the most common CT imaging finding in the current study. Reporting systems showed fair to moderate
agreements between senior and junior raters (0.246-0.490, p<0.001). According to the assigned three-category coding system as
similar to that of the Radiological Society of North America (RSNA) Expert Consensus Statement on Reporting and other reporting
systems and guidance, strength of inter-rater agreement values was increased (0.365-0.576, p<0.001) and inter-system agreements
were substantial to almost perfect in both raters.
Conclusion: Radiology reporting including frequently seen CT features and lung parenchyma distributions with systems based on
fewer categories may provide good agreement between observers in patients with suspected COVID-19.