Resources for conducting challenges

Data

• Phantom data: A variety of resources exist for phantom data that can be utilized in the running of image analysis challenges. Since ground truth is often known in this case, both bias and repeatability can be estimated. Some examples include the
  • Anthropomorphic thoracic phantom (FDA phantom)
  • RIDER PET/CT phantom)
  • DCE Digital Reference Object
• Clincal data: The TCIA hold a variety of collections that are potentially extremely valuable for challenges
  • Public collections
  Shared lists can be made available to easily distribute the data to participants

Image formats and converters

Participants in many of the common medical imaging challenges can be required to submit their results in a variety of image formats. These include:

• Dicom
• Nrrd
• Nifti
• Dicom-Seg
• Dicom-RT
• AIM

A number of tools and libraries facilitate the conversion between the different formats.

• ITK
• GDCM
• DCMTK
• Plastimatch
• Convert3D
• Slicer , including the Reporting module
• Slicer RT

Metrics

A variety of metrics are typically used in segmentation challenges. These include region based measures such as:

• Dice coefficient
• Sensitivity
• Specificity

and surface distance based measures such as

• Hausdorff distance

Many of these metrics are part of standard libraries and tools such as

• ITK
• Slicer
• Convert 3d

or are in open source packages such as

• Covalic
• Insight Journal 707

Statistical Analysis and libraries

The open source R library can be used to generate statistical analysis and visualization of results for the challenges. These include packages:

• Agreement
• Shiny R