Algorithm Validation

Brief description of this line of research

The validation of ARIA (automatic retinal image analysis) algorithms is a very important task. Validation indicates the process by which the output of an algorithm can be compared with a reference standard which is usually represented by a ground truth defined by expert performance. Our aim is to contribute towards the objective validation of these algorithms by proposing new measures of evaluation and providing publicly available datasets.



RIM-ONE is an open retinal fundus image database with accurate gold standards of the optic nerve head (ONH) provided by different experts. It includes images from healthy eyes as well as images from eyes with glaucoma at different stages. A variability measurement by zones of the optic disc is also proposed for the purpose of validation. Three hospitals have contributed to the development of this database: Hospital Universitario de Canarias, Hospital Clínico San Carlos and Hospital Universitario Miguel Servet.


DCSeg – An interactive tool for optic disc and cup segmentation of stereo and monocular retinal fundus images

We have developed a new tool to ease the manual segmentation of the optic disc and cup of retinal fundus images, which can be used on monocular and stereo images. By using this tool we have developed the release 3 of RIM-ONE.

Version 0.2b includes a Self-assessment mode, very useful for teaching and learning. This is a beta release that we will continue updating and improving.

The DCSeg tool and instructions can be downloaded from: DCSeg 0.2b

Demo video of optic cup segmentation in stereo mode:

We are glad to hear any comments about it!


RIM-ONE Release 3

The third version of the RIM-ONE database is finally here. This dataset consists of 159 stereo retinal fundus images. The optic disc and optic cup of each image has been segmented by 2 experts in ophthalmology to create the ground truth. The average segmentation is also available to use as the reference segmentation or gold standard.

RIM-ONE third release: RIM-ONE r3


RIM-ONE Release 2

We have published a new version of the database, with more images and better precision on the manual segmentations.

RIM-ONE second relase:  RIM-ONE r2

MATLAB function to convert gold standard files to mask images:  convert_gs_to_image

In this second release, the format of the gold standard files is a little bit different than the first version, it uses more radii to manually segment the optic nerve head and the classification of the images is also different. There is a MATLAB function to convert the gold standard files to mask images. The usage of this function is as follows:
- First unzip the file “RIMONE db”
- Once unzipped, it will contain two folders: “Normal” and “Glaucoma and glaucoma suspicious”
- To convert the gold standard files to images:
convert_gs_to_image("path/to/Normal", "path/to/Output Normal GS images")

convert_gs_to_image(“path/to/Glaucoma and glaucoma suspicious”, “path/to/Output Glaucoma GS images”)


RIM-ONE Release 1

The first release of the RIM-ONE database can be downloaded from here:  RIM-ONE r1

The MATLAB code for the variability measurement procedure:  variability_measurement_procedure

RIM-ONE Variability among experts

RIM-ONE Variability among experts

Related publications:

F. Fumero, J. Sigut, S. Alayon, M. González-Hernández, and M. González de la Rosa, “Interactive Tool and Database for Optic Disc and Cup Segmentation of Stereo and Monocular Retinal Fundus Images,” in Short Papers Proceedings - WSCG 2015, Pilsen, Czech Republic, 2015, pp. 91–97 [Online]. Available:
S. Alayon, M. Gonzalez de la Rosa, F. J. Fumero, J. F. Sigut Saavedra, and J. L. Sanchez, “Variability between experts in defining the edge and area of the optic nerve head,” Archivos de la Sociedad Española de Oftalmología (English Edition), vol. 88, no. 5, pp. 168–173, May 2013 [Online]. Available:
F. Fumero, S. Alayon, J. L. Sanchez, J. Sigut, and M. Gonzalez-Hernandez, “RIM-ONE: An open retinal image database for optic nerve evaluation,” in 2011 24th International Symposium on Computer-Based Medical Systems (CBMS), 2011, pp. 1–6 [Online]. Available:


Useful links:

E. Trucco, A. Ruggeri, T. Karnowski, L. Giancardo, E. Chaum, J. P. Hubschman, B. Al-Diri, C. Y. Cheung, D. Wong, M. Abràmoff, G. Lim, D. Kumar, P. Burlina, N. M. Bressler, H. F. Jelinek, F. Meriaudeau, G. Quellec, T. Macgillivray, and B. Dhillon, “Validating retinal fundus image analysis algorithms: issues and a proposal,” Invest. Ophthalmol. Vis. Sci., vol. 54, no. 5, pp. 3546–3559, May 2013 [Online]. Available:


Leave a Reply