Tempered Balanced Accuracy (tBA)
DOI:
https://doi.org/10.51483/IJAIML.2723Keywords:
balanced accuracy, class imbalance, evaluation metric, weighted recall, cost-sensitive evaluation, confusion matrixAbstract
Class imbalance is present in most practical classification datasets, from intrusion detection to medical diagnosis, and it is well known that standard accuracy rewards a classifier that ignores the minority class. Balanced accuracy (BA) removes this bias by averaging the per-class recalls with equal weight. However, equal weighting is itself a fixed cost model: BA implicitly assumes that the importance of a class is exactly inversely proportional to its prevalence, which over-rewards a rare class that is not critical and under-rewards a rare class that is critical. This paper proposes the Tempered Balanced Accuracy (tBA), a weighted mean of per-class recalls in which the weight of class is its prevalence raised to a tempering exponent . The parameter interpolates continuously between standard accuracy ( ) and BA ( ) and extends beyond BA ( ) for rare-but-critical classes. We show that the chance level of tBA equals for every , that the score of a majority-class predictor has a closed form that decreases monotonically in , and that can be set directly from a desired misclassification cost ratio. Rules for choosing are given for known cost ratios, for the absence of cost information ( as the safe value against class collapse, as the symmetric compromise between accuracy and BA), and as a data-driven estimate from the effective number of classes. Through eight hand-calculated cases worked from labelled samples and nine further cases from confusion matrices, covering binary and multi-class problems, balanced and imbalanced classes at ratios from 4:1 to 999:1, class collapse, minority over-prediction, an absent class, and classifier ranking, we demonstrate that tBA exposes behaviors that both accuracy and BA hide, including a ranking reversal between two classifiers at and a slope of the tBA curve that identifies whether a classifier is weak on its large or its small classes. tBA requires no additional computation beyond the confusion matrix and reduces to existing metrics at its endpoints.
