Home | Publications | GSH23

DCSI–An Improved Measure of Cluster Separability Based on Separation and Connectedness

MCML Authors

Fabian Scheipl

PD Dr.

Core PI

Functional Data Analysis

Moritz Herrmann

Dr.

Transfer Coordinator

→ Group Anne-Laure Boulesteix
Biometry in Molecular Medicine

Abstract

Whether class labels in a given data set correspond to meaningful clusters is crucial for the evaluation of clustering algorithms using real-world data sets. This property can be quantified by separability measures. The central aspects of separability for density-based clustering are between-class separation and within-class connectedness, and neither classification-based complexity measures nor cluster validity indices (CVIs) adequately incorporate them. A newly developed measure (density cluster separability index, DCSI) aims to quantify these two characteristics and can also be used as a CVI. Extensive experiments on synthetic data indicate that DCSI correlates strongly with the performance of DBSCAN measured via the adjusted Rand index (ARI) but lacks robustness when it comes to multi-class data sets with overlapping classes that are ill-suited for density-based hard clustering. Detailed evaluation on frequently used real-world data sets shows that DCSI can correctly identify touching or overlapping classes that do not correspond to meaningful density-based clusters.

misc GSH23