Background: Computer-aided diagnosis is increasingly used in radiology and may support not only unexperienced readers but also senior radiologists. It appears promising, especially in the sometimes challenging detection of early ischemic changes in stroke CT.

Purpose: To compare the new post-processing software prototype Frontier_ASPECTS against two senior radiologists in ASPECTS evaluation.

Material and Methods: Retrospectively, pre-interventional CTs of 100 patients, who underwent endovascular revascularization for acute middle cerebral artery ischemia, were blindly re-analyzed with respect to ASPECTS by two neuroradiologists (separately and in consensus) and by use of Frontier_ASPECTS. In addition to a fully automatic Frontier_ASPECTS reading (Frontier_1), Frontier_ASPECTS readings subsequently manually corrected for old cerebral defects (Frontier_2a), the affected hemisphere (known from CT angiography, Frontier_2b), and both (Frontier_3) were assessed. Statistical analysis was performed by intraclass correlation and Bland–Altman analysis.

Results: Median ASPECTS was 10 for Frontier_3 (range = 5–10), 10 for radiologist_1 (range = 4–10), 9 for radiologist_2 (range = 2–10), and 10 for consensus reading (range = 2–10). All Frontier_ASPECTS variants correlated lowly with consensus reading (Frontier_1, r = 0.281; Frontier_2a, r = 0.357; Frontier_2b, r = 0.333; Frontier_3, r = 0.350; always P  < 0.01), while both radiologists and consensus reading correlated highly (radiologist_1, r = 0.817; radiologist_2, r = 0.951; always P  < 0.001). Bland–Altman analysis confirmed a worse agreement between Frontier_3 and consensus reading than between both radiologists and consensus reading.

Conclusion: We found only low agreement between the post-processing software Frontier_ASPECTS and expert consensus reading in ASPECTS evaluation. Notably, performance of Frontier_ASPECTS improved by simple manual corrections but is–at Frontier_ASPECTS’ current development status–inferior to the performance of senior radiologists.