Why Above-Chance Accuracy Does Not Make a Reliable Classifier
Bridges the perception numbers to the fence. Accuracy figures of 60 to 80% are well above chance and still produce wrong answers at a rate no consequential decision can absorb, and once the base rate in a real population is folded in, the share of positive calls that are correct falls further. The reasoning skill is to convert an accuracy headline into an expected count of errors on a realistic population before believing anything can be built on it.
Questions this Concept answers
- Why is performance above chance a weak basis for a consequential decision?
Sensitivity and specificity
The page defines sensitivity and specificity as the probabilities of a correct call conditioned on the subject's true status, and shows why either one alone can be gamed - a test that always says pos…