Skip to main content
The NCI Community Hub will be retiring in May 2024. For more information please visit the NCIHub Retirement Page:https://ncihub.cancer.gov/groups/ncihubshutdown/overview
close

HTT update 20220531

Wiki Main Page

Previous Updates

We are pleased to announce some recent accomplishments of the High-throughput truthing project (HTT).

A list of all previous updates can be found on this page. Our publications and presentations are organized in this Zotero library. Our data is being shared in this GitHub repository.

The HTT project’s objective is to create a validation dataset fit for a regulatory purpose. Pursuing this objective is expected to inform regulatory frameworks and be instructive to others to develop their own validation datasets. For more detailed information about the project and/or to register as a pathologist-data-collector, please visit https://ncihub.org/groups/eedapstudies/.

The HTT project has been accepted as a project of the Truthing and Validation Workgroup of PIcc.

The HTT project was also accepted as a project of the W.H.O. International Collaboration for Cancer Classification and Research (IC3R).

  • The IC3R project is named PADEA (Pathologist Annotation Datasets for Evaluating Algorithms) to allow the effort to grow to other applications beyond TILs evaluation.
  • Link to IC3R “Projects” page.

To contact us with any questions or join our efforts: Contact Us HERE


Updates

May 2022

  • 5/27: Upcoming Presentation to College of American Pathologists (CAP) AI Committee
    • New CAP Committee chaired by Raj Dash
  • Recruiting: FDA/CDRH/OSEL/Division of Imaging Diagnostics and Software Reliability (DIDSR)
    • DIDSR is recruiting at all levels, Full Time Employees and Fellows
  • Published Manuscript: “Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer”
    • Authors: Victor Garcia, Katherine Elfer, Dieter Peeters, Anna Ehinger, Bruce Werness, Amy Ly, Xiaxion “Bill” Li, Matthew Hanna, Kim Blenman, Roberto Salgado, Brandon Gallas
    • Published 17 May 2022 in Cancers Special Issue “Tumor Infiltrating Lymphocytes (TIL) in Solid Tumors: Emerging Insights.”
    • Open Access Link
  • Presentation: “Development of Pathologist Training Materials using Consensus Driven Annotations of sTIL Assessment in Breast Cancer”
  • Expert Annotations now on DIDSR/HTT repository and other updates
    • Expert Panel annotations (“camic-expert”) now publicly available on DIDSR/HTT
    • R Markdown and PDF Output used for figures in Garcia, et. al Cancers manuscript also available
  • Published Manuscript: “Three-Way Mixed Effect ANOVA to Estimate MRMC Limits of Agreement”

April 2022

  • Poster Presentation: “Tools for collecting pathologist annotations and understanding interobserver variability”
    • Presenter: Katherine Elfer
    • American Association for Cancer Research (AACR) Annual Conference 2022
    • Presented 8 April 2022 in New Orleans
    • See Poster Here

March 2022

  • Submitted Manuscript, “Tools to collect and analyze crowd-source pathologist annotations for validating machine learning algorithms”
    • Elfer K, et. al. submitted to the Journal of Medical Imaging
  • Submitted Manuscript, “Development of Training Materials for Pathologists to Provide Machine Learning Validation Data of Tumor-Infiltrating Lymphocytes in Breast Cancer”
    • Garcia V., et. al. submitted to Cancers Special Issue “Tumor Infiltrating Lymphocytes (TIL) in Solid Tumors: Emerging Insights.”

February 2022

  • Published Manuscript, “FDA fosters innovative approaches in research, resources and collaboration” Zotero Library for Publications and Presentations]

Created on , Last modified on