Evaluation procedure

Goals
The aims of the evaluation metrics involved in the segmentation contest are twofold:
 measure the degree of accuracy of the left ventricular endocardium and epicardium as the right ventricular endocardium. This will be done through global and local measures of similarity with the reference contours;
 measure the degree of accuracy of the derived clinical indices.
Clinical indice metrics
The clinical metrics are the ones that are the most widely used in cardiac clinical practice. Moreover a set of metrics are computed per structure (i.e. LV cavity, RV cavity and myocardium) to allow the computation of ranking per structure. The following metrics are computed:
 Left ventricular cavity
 Correlation coefficient computed from the set of End Diastolic Volumes (EDV) measurements
 Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
 Correlation coefficient computed from the set of Ejection Fraction (EF) measurements
 Bias computed from the set of EDV measurements
 Bias computed from the set of ESV measurements
 Bias computed from the set of EF measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EDV measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EF measurements
 Right ventricular cavity
 Correlation coefficient computed from the set of End Diastolic Volumes (EDV) measurements
 Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
 Correlation coefficient computed from the set of Ejection Fraction (EF) measurements
 Bias computed from the set of EDV measurements
 Bias computed from the set of ESV measurements
 Bias computed from the set of EF measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EDV measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of EF measurements
 Myocardium
 Correlation coefficient computed from the set of Myocardial Mass (at End Diastolic time instant) measurements
 Correlation coefficient computed from the set of End Systolic Volumes (ESV) measurements
 Bias computed from the set of Myocardial Mass measurements
 Bias computed from the set of ESV measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of Myocardial Mass measurements
 Limits of agreement (LOA = 1.96 times the standard deviation) computed from the set of ESV measurements
Distance error metrics
A set of geometrical metrics are computed per structure (i.e. LV cavity, RV cavity and myocardium) to allow the computation of ranking per structure. The following metrics are computed:
 Left ventricular cavity
 The average Dice value for the left ventricle cavity at ED.
 The average Dice value for the left ventricle cavity at ES.
 The average Hausdorff distance for the endocardial contour of the left ventricle at ED.
 The average Hausdorff distance for the endocardial contour of the left ventricle at ES.
 Right ventricular cavity
 The average Dice value for the LV cavity at ED.
 The average Dice value for the LV cavity at ES.
 The average Hausdorff distance for the endocardial contour of the right ventricle at ED.
 The average Hausdorff distance for the endocardial contour of the right ventricle at ES.
 Myocardium
 The average Dice value for the myocardium region at ED.
 The average Dice value for the myocardium region at ES.
 The average Hausdorff distance for the myocardial contours at ED.
 The average Hausdorff distance for the myocardial contours at ES.
 measure the degree of accuracy of the left ventricular endocardium and epicardium as the right ventricular endocardium. This will be done through global and local measures of similarity with the reference contours;
Please refer to this citation for any use of the ACDC database
O. Bernard, A. Lalande, C. Zotti, F. Cervenansky, et al.
"Deep Learning Techniques for Automatic MRI Cardiac Multistructures Segmentation and
Diagnosis: Is the Problem Solved ?" in IEEE Transactions on Medical Imaging,
vol. 37, no. 11, pp. 25142525, Nov. 2018
doi: 10.1109/TMI.2018.2837502