Skip to main content

High-throughput Truthing - Year 2 Report

Wiki Main Page

Summary (4/2019-4/2020):

In 2018, we installed eeDAP on a 14-head microscope and completed a data-collection session, collecting evaluations from 12 pathologists simultaneously in a single visit to Memorial Sloan Kettering (MSK). We loaned the eeDAP system to MSK and they proceeded to conduct a study to compare mitotic figure (MF) counting on a microscope and four different WSI scanners. We did the primary analysis of the results using statistical methods that we developed, accounting for reader and case variability, and this work was presented in 2018 at Pathology Informatics LINK to presentation.

Since our last submission, we have identified other partners and challenges that are still in the development stage, giving us a better opportunity to make an impact on the problem. The first group is led by DIDSR colleagues; they are designing a challenge for the SPIE Medical Imaging community. The second group is The International Immune-Oncology Biomarker Working Group on Breast Cancer. This highly motivated group has more than 140 pathologists, and the chair, Dr. Roberto Salgado, wants to leverage this working group to create an MDDT dataset. He believes that a regulatory-grade FDA-qualified dataset for the community to use in a submission to the FDA will encourage algorithm developers to focus on the detection of lymphocytes to prognosis cancer. We piloted a lymphocyte evaluation study at the annual meeting of the American Society of Clinical Pathology and hope to collect pivotal data in partnership with that and other society conferences that have thousands of pathologists in attendance, enabling high throughput truthing. We also have several new partners that are offering support with hosting data-collection events, digital data infrastructure and technology, and providing/recruiting pathologists.


Presentation at Pathology Informatics Summit 2019:A Collaborative Project to Produce Pathologist Annotations to Evaluate Viewers and Algorithms.
Slide show: 20190508-HTToverviewGallasAtPIsummit-v4.pdf (2 MB, uploaded by Brandon D. Gallas 1 year 5 months ago)

Presentation was given Nov.-Dec. 2018 to FDA/CDRO/OSEL management, the working group, project collaborators, and others. Slide show: 20190402-HTToverviewPublic.pdf (410 KB, uploaded by Brandon D. Gallas 1 year 5 months ago)

Proposal for internal funding Full proposal submitted 10/19/2018: binderFY19_CDRH_CP_Application-GallasHighThroughputTruthingDigiPathPublic.pdf (7 MB, uploaded by Brandon D. Gallas 1 year 5 months ago)Funding awarded in March 2019.

Created on , Last modified on