CAD interpretation system is proposed

CHICAGO—A major issue facing radiologists who use computer-aided detection systems for early detection of lung cancer is deciding when a CAD finding truly represents malignancy. An algorithm developed by National Cancer Institute researchers could be the key to getting around this problem.

One of the main goals of the NCI's Lung Imaging Database Consortium initiative was to develop consensus guidelines for the evaluation of CT images of the lungs using CAD.

The LIDC has come up with a two-step process that can help radiologists reach a robust, standardized way to make sense of CAD data, said Samuel G. Armato, MD, associate professor of radiology, University of Chicago. Dr. Armato presented findings of an LIDC-sponsored study at the 2007 RSNA meeting.

Dr. Armato and his colleagues looked into what lung CAD specialists refer to as "ground truth." The study broke the process down into two steps:

First, researchers analyzed variables in the definition of "truth" from combined lung CAD reads by experienced thoracic radiologists.

Next, they analyzed the variability in the performance of other experienced thoracic radiologists based on these definitions of truth.

'True' nodules

Four thoracic radiologists reviewed 25 thoracic CT scans and marked lesions they considered to be nodules 3 mm or larger. They retrospectively agreed upon a set of "true" nodules from different combinations of readings from two of the four radiologists. The nodule-detection performance of the other two radiologists was evaluated based on this two-radiologist assessment of "true" nodules.

The investigators found significant variability across radiologists for lung nodule identification. They also noted, however, that the method provided a solid approach to reaching a straightforward definition of what a true nodule is before the final interpretation is done.

The most important aspect of the study is that it showed that the definition of truth can be highly variable depending upon who provides that truth and what process is used to gather that truth, according to Dr. Armato. CAD researchers need to understand the truth they are using to evaluate their systems and potential limitations. The LIDC two-step process for lung CT scanning with CAD helps make the truth assessment more robust, Dr. Armato said in an interview.

"What we as CAD researchers consider truth is not necessarily 'the' truth, and there are variabilities in that truth. That variability in truth will impact the reported performances of CAD systems. That can be important for research and for FDA approval of commercial systems now and in the future," he said.