We did not discuss the reader study protocols as planned in order to have a more important discussion about how this project fits in with the big picture. We will address the protocols at the next meeting. I was also encouraged to be more explicit handing out assignments.
With Philips successful 510(k) deNovo allowing its WSI device to be marketed in the US (and used clinically), why do we need to do studies on the microscope?
The community is not done comparing WSI evaluations to microscope evaluations. The microscope will likely remain the baseline of comparison for image analysis programs as well. Additionally, the community needs to understand the limitations that can inform users on the utility of WSI, and at the same identify areas to improve WSI. There are a number of variables and scanner characteristics that might affect diagnostic performance as spelled out in the technical performance guidance and also outlined in Prathana’s work. eeDAP studies, studies with pathologists performing pathology relevant tasks, can be design to evaluate the impact of differences in the technical performance metrics in an objective way (task-based performance, not opinions).
Another issue posed: Can we get the eeDAP MDDT approved with technical performance info (mainly registration precision) only, no reader studies?
Mischa raised this issue, hoping to get the MDDT approved and available faster. Brandon is not confident the MDDT will be approved without reader studies given the context of use and the FDA’s desire for the value of the MDDT to be clear. Furthermore, the process is likely to be long either way, and there is nothing stopping anyone from using eeDAP to collect data for any regulatory business right now. The motivation to use eeDAP will come once the community believes that it has value to make faster and more precise measurements.
- Brandon has asked the MDDT reviewers if we can get the MDDT approved with technical performance only. Email archived here.
Agenda updated to be minutes:
- Next T-con (Same time Mondays? or Tuesdays?)
-
Tuesdays no conflictsBrandon has a conflict and is proposing Monday May 1 same time (10 am EST)
-
- Updates from each group on downloading and test driving eeDAP in digital mode.
- NOBODY tried. This was supposed to be a necessary but not hugely challenging task.
- Who will be first? I will have a prize of symbolic value.
- Update from James Monaco on the Inspirata team protocol (refer to the wiki page for the current version LINK)
- This link has the potential to go stale. The wiki page should be the most up to date
- I am asking that James update his protocol to address more items identified in the STARD checklist (Standards for Reporting Diagnostic Accuracy Studies).
- I am assigning Philips, CSHL and Darren Treanor to review and provide feedback by email before (and at) our next meeting.
- Update from Brandon Gallas on the FDA team protocol (refer to the wiki page for the current version LINK).
- I am asking ITT, MSKCC, and NCI to review this protocol and provide feedback by email before (and at) our next meeting.
- I am also asking MSKCC and NCI provide a supplement to or otherwise update this protocol to specify the research questions they want to answer (e.g. the devices they want to compare) and the logistics they are arranging for data collection (location and readers they plan to recruit)
- Discuss registration precision: Philips lead? Use a calibration slide.
- I am asking Philips to outline a protocol for evaluating registration precision. We need to figure out if or how to read the proprietary file format. I hope that Philips would work with the open-source community to make their images accessible for the storage and manipulation of biological microscopy data.
- Sizing studies (by simulation, something to start thinking about)
T-con attendees
I am sure I missed some folks. Please add accordingly.
- FDA team: Brandon Gallas, Qi Gong, Weijie Chen, Marios Gavrielides, Ben Berman
- NCI team: Not present
- IIT Madras team: I saw them, but didn’t ask who was on the line.
- MSKCC team: Joe Sirintrapun
- Philips team: Liselotte Kornmann, Prarthana Shrestha, Mischa Nelis
- Inspirata team: James Monaco
- CSHL team: Not present