Revisiting the ABCs of Working with AI: A Replication with Radiologists

Daniel Martin · arXiv · 10 Jun 2026

Where this sits in the debate

Will AI actually boost productivity and growth? · its finding: it depends (why studies differ)

Direction of the finding as read from the abstract by an AI model (AI-written).

Abstract

Artificial intelligence (AI) systems increasingly assist human experts, but the consequences of AI assistance on productivity can be heterogeneous. Caplin, Deming, S. Li, Martin, Marx, Weidmann, and Ye (2025b) provide evidence that two characteristics, ability and belief calibration, help to determine the returns to AI assistance. This note shows that their results replicate to a setting where professional radiologists analyze chest X-rays with access to state-of-the-art machine learning predictions. I leverage the public Collab-CXR data repository described by Moehring, Kutwal, Huang, Banerjee, Jacobi, Eber, Mendoza, Chung, Dayan, Gupta, Bui, Truong, Pareek, Langlotz, Lungren, Agarwal, Rajpurkar, and Salz (2025) and first analyzed for human-AI collaboration by Agarwal, Moehring, Rajpurkar, and Salz (2023). To faithfully reproduce the analysis in Caplin, Deming, S. Li, Martin, Marx, Weidmann, and Ye (2025b), I use the radiologist assessments from the repeated-case designs, which include 68 radiologists and 11,420 paired radiologist-patient-pathology observations. The results of this replication support the external validity of their core findings: lower baseline ability and higher calibration predict larger incremental value from AI.

About this publication

Preprint. A preprint — posted publicly before or without journal peer review; findings may still be revised.

Published: 2026 (4 months ago)
Citations: Not recorded for this source

Citation counts come from OpenAlex; none are recorded for this source — absence of a count is not absence of citations.

Revisiting the ABCs of Working with AI: A Replication with Radiologists

Abstract

About this publication

More on this topic