Skip to main content

Presentation 2022: Pathology Informatics Summit - Training Materials

Back to Presentations and Publications Main Page

Pathology Informatics Summit 2022

Title: Development of Pathologist Training Materials using Consensus Driven Annotations of sTIL Assessment in Breast Cancer

  • David L. Lawrence Convention Center in Pittsburgh, PA
  • 9:45 AM, Wednesday, 11 May 2022
  • In person

Final presentation slides

Garcia_PI_Summit_2022_Slides_-_20220506.pdf (3 MB, uploaded by Victor Garcia 10 months 1 week ago)

Presentation Recording



  • The High Throughput Truthing project aims to develop a dataset of stromal tumor-infiltrating lymphocytes (sTIL) density estimates in hematoxylin and eosin stained invasive breast cancer biopsy specimens fit for a regulatory purpose 1,2. The dataset is derived from a pilot study of 640 unique regions of interest (ROIs) derived from 64 whole slide images. After collecting annotations for our pilot study, we observed greater interobserver variability among sTIL density evaluations than desired. To improve annotation quality, we used the pilot data to create custom training materials.


  • We categorized ROIs based on their mean sTIL density as “10% or less”, “11% to 40%”, “greater than 40%”. We selected 72 unique ROIs from those with the highest and lowest variance. Each ROI was discussed in a group setting by at least three members of our expert panel, one clinical scientist and seven board-certified pathologists who are project collaborators and trained in sTIL scoring. In a series of eight one-hour sessions, our panel reviewed each ROI and provided verbal estimates of stromal percentage and sTIL density, and comments on features that confound the sTIL assessment. We then compiled experts’ scores and comments into a reference document.


  • The expert discussions found a need for additional instructions on sTIL assessment. Challenges were encountered for both stromal percentage and sTIL density assessment. These include evaluating the stromal percentage with respect to the whole ROI, distinguishing fibroblasts and degenerated tumor cell nuclei from lymphocytes, and excluding adipose tissue as tumor-associated stroma.


  • Our approach demonstrates a method by which training materials can be developed using a focus group approach based on pilot study data. From selected ROIs, we will create two data sets: a training set and a proficiency test set. The training set offers participants real-time, descriptive and quantitative feedback based on our compiled expert commentary to solidify their sTIL density knowledge, while the proficiency test set requires participants to demonstrate their competency in sTIL assessment before proceeding with data annotation. These materials will train crowd-sourced pathologists to help create a large algorithm validation dataset, while also improving sTIL evaluation in clinical practice.


  • Victor Garcia
    • Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, White Oak, MD
  • Katherine Elfer
    • Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, White Oak, MD
  • Anna Ehinger
    • Department of Genetics and Pathology, Laboratory Medicine Region Skåne, Lund, Sweden; Department of Clinical Sciences Lund, Division of Oncology, Lund University, Lund, Sweden
  • Dieter Peeters
    • Department of Pathology, Sint-Maarten Hospital, Mechelen, Belgium; Department of Biomedical Sciences, University of Antwerp, Antwerp, Belgium
  • Xiaoxian Li
    • Department of Pathology and Laboratory Medicine, Emory University, Atlanta, GA
  • Amy Ly
    • Department of Pathology, Massachusetts General Hospital, Boston, MA
  • Bruce Werness
    • Inova Health System Department of Pathology, Falls Church, VA; Arrivebio LLC, San Francisco, CA
  • Matthew Hanna
    • Memorial Sloan Kettering Cancer Center, New York
  • Kim Blenman
    • Department of Internal Medicine, Section of Medical Oncology, School of Medicine and Yale Cancer Center, and Department of Computer Science, School of Engineering and Applied Sciences, Yale University, New Haven, CT
  • Roberto Salgado
    • Division of Research, Peter Mac Callum Cancer Centre, Melbourne, Australia; Department of Pathology, GZA-ZNA Hospitals, Antwerp, Belgium
  • Brandon D. Gallas
    • Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA, White Oak, MD

Created on , Last modified on