MLCvis: A Design Study of a Visual Analysis Tool for Multi-Label Classifier Evaluation and Comparison

Machine learning classifiers are widely used in domain tasks such as audio tagging, document categorization, and image labeling, many of which require multi-label classification. Yet, existing evaluation tools often reduce model performance to aggregate metrics like Accuracy, offering limited insight into classifier behavior at finer-grained levels, such as per-label or per-instance. We present MLCvis, a visual analysis tool developed following a nine-step design study methodology, which supports deeper and more interpretable evaluation and comparison of classifiers, including multi-label ones. Based on the needs of domain experts, MLCvis enables analysis at global, label, and instance levels through linked visualization components. It supports a range of data types (audio, text, image) and addresses common challenges in classifier inspection, including ambiguous labels and threshold sensitivity. We demonstrate MLCvis across three use cases and report findings from two user studies (N = 6 each). Results show high usability and improved task performance compared to traditional tools like confusion matrices. Our work positions MLCvis as a practicable solution for users with basic classifier knowledge who seek a more nuanced understanding of classification behavior in real-world, multi-label settings.