[HighthroughputTruthingYear1] [[BR]] [HighthroughputTruthingYear2] ===Year 3: High-throughput truthing of microscope slides to validate artificial intelligence algorithms analyzing digital scans of pathology slides: data collection to create the medical device development tool (MDDT)=== In Year 3 of the High Throughput Truthing project, the team will collect data at various collaborating sites and conferences. [[BR]] Check out what was completed in [HighThroughputTruthingYear2 Year 2] and [HighThroughputTruthingYear1 Year 1] [[BR]] '''Pitch''': We are crowdsourcing pathologists to collect data (images + pathologist annotations) that can be qualified by the FDA/CDRH medical device development tool program (MDDT). If successful, the MDDT qualified data along with a statistical software package for data analysis would be available to any algorithm developer to be used to validate their algorithm performance in a submission to the FDA/CDRH. Researchers from the U.S. Food and Drug Administration, alongside academic collaborators, are collecting pathologist annotations as data for AI/ML algorithm validation for tumor infiltrating lymphocyte (TIL) detection and quantitation. We are asking board-certified anatomic pathologists and anatomic pathology residents to score 80 ROIs as part of a research study. We anticipate that this task will take participants a total of 30 minutes. The data are intended to inform the agency’s approach to novel algorithm validation, ensuring high quality commercial products with a faster FDA-pipeline to approval. [[BR]] [[BR]] __Data Collection by Pathologists:__ [[BR]] [[BR]] Recruiting Advertisement: [[File(HTTatUSCAP.pdf)]] [[BR]] [[BR]] You will be presented pre-selected fields of view (FOV) digitally or on a microscope (Figure below, left). For each FOV, you will be prompted for the tumor-associated stromal TIL density, which is a number from 0-100. Once you have determined stromal TIL density for each FOV, you will type or click the value using the software’s guided user interface (Figure below, right). Study training materials on TIL density evaluations can be found in the following section and may be completed ahead of time. [[Image(eeDAPsystem.PNG, 400px)]] [[Image(eeDAPinterfaceHTT.png, 200px)]] [[BR]] (Left: eeDAP system components, labeled. Read more [https://www.spiedigitallibrary.org/journals/Journal-of-Medical-Imaging/volume-1/issue-03/037501/Evaluation-environment-for-digital-and-analog-pathology--a-platform/10.1117/1.JMI.1.3.037501.full?SSO=1 here]. Right: Screenshot of eeDAP user interface for stromal TIL density data collection) __See the Data-Collection Webinar for more information:__ [[BR]] [[BR]] Recording: [[File(20200219_HTTdataCollectionWebinar_Trim.mp4)]] Data-Collection Webinar Agenda: # [HighThroughputTruthingYear3 High Throughput Truthing (HTT) project wiki page] * Brandon D. Gallas, !PhD (15 min) * Slides: [[File(20200219_HTTdataCollectionWebinar-02192020.pdf)]] # Tumor Infiltrating Lymphocytes (TILs) clinical task. * Roberto Salgado, MD (15 min) * Slides: [[File(TutorialWebsiteAdapted29012020.pdf)]] # Instructions for data collection Friday Feb. 28 * Sarah Dudgeon, MPH (15 min) * Slides: [[File(dataCollectionTutorialHTT.pdf)]] * Video demo of !PathPresenter data-collection platform: [https://youtu.be/_XWPoqZsdlk]. # Q&A (15 min) __Training documents:__ [[BR]] '''The Study:''' [[File(dataCollectionTutorialHTT.pdf)]] – This __'''DRAFT'''__ tutorial outlines the project, and provides a demonstration of your tasks for all three data collection platforms. '''The Clinical Task:''' [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6267863/ TILs evaluation manuscript], [https://ncihub.org/groups/eedapstudies/wiki/HighThroughputTruthingYear3/File:TutorialWebsiteAdapted29012020.pdf TILs training slides] – Participating pathologists can review the clinical training slides (required to participate), and read the related manuscript if possible. These materials were created by the [https://www.tilsinbreastcancer.org/ TILs in breast cancer working group]. '''The Hardware Used in the Study:''' [https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4478997/ eeDAP] – Annotations will be collected in digital and microscope modes. For the microscope mode, we will use eeDAP, an Evaluation Environment for Digital and Analog Pathology (eeDAP), a registration system between the microscope and digital whole slide images. __Consent Form:__ [[BR]] [[File(HTT_IRBinformedConsent.pdf)]] __Exit Survey:__ [[BR]] Please complete [[https://docs.google.com/forms/d/e/1FAIpQLSfoh2f4RoxFAgDPPoU430P6Uo26PMbRLmrboLL9wfaF5LA9gQ/viewform this Survey]] when you finish data collection, and help us improve the HTT project. __Get Involved:__ [[BR]] * Supply glass slides and their scanned versions * Help create a Continuing Medical Education course in conjunction with this project * Host an analog (microscope + eeDAP) data collection event at your clinical site * Help spread the word, recruit your colleagues to participate in online data collection __Thank you. For more information contact:__ [[BR]] '''Brandon Gallas''', !PhD, ([mailto:brandon.gallas@fda.hhs.gov brandon.gallas@fda.hhs.gov]) [[BR]] FDA/CDRH/OSEL Division of Imaging, Diagnostics, and Software Reliability [[BR]] on behalf of the High-Throughput Truthing (HTT) Project