P4-metric
The P 4 metric (also known as FS or Symmetric F ) enables performance evaluation of a binary classifier.
≈ 3 min read
Names in other languages
No names in other languages are held for this entry yet.
Article bodies are English and are not machine-translated. A name in another language is not a translation of the article.
The P 4 metric (also known as FS or Symmetric F ) enables performance evaluation of a binary classifier. The P 4 metric is calculated from precision, recall, specificity, and NPV (negative predictive value). The definition of the P 4 metric is similar to that of the F 1 metric, however the P 4 metric definition addresses criticisms leveled against the definition of the F 1 metric.
The definition of the P 4 metric may, therefore, be understood as an extension of the F 1 metric. Like the other known metrics, the P 4 metric is a function of: TP (true positives), TN (true negatives), FP (false positives), FN (false negatives). The main assumption behind this metric is that all the probabilities mentioned above are close to 1 for a properly designed binary classifier.
Indeed, \mathrm{P}_4 = 1 if, and only if, all of the probabilities above are equal to 1. Another important feature is that \mathrm{P}_4 tends to zero any of the above probabilities tend to zero. \mathrm{P}_4 = \frac{4}{\frac{1}{P(+ \mid C{+})} + \frac{1}{P(C{+} \mid +)} + \frac{1}{P(C{-} \mid -)} + \frac{1}{P(- \mid C{-})}} = \frac{4}{\frac{1}{\mathit{precision}} + \frac{1}{\mathit{recall}} + \frac{1}{\mathit{specificity}} + \frac{1}{\mathit{NPV}}}.
In terms of TP,TN,FP,FN it can be calculated as follows: \mathrm{P}_4 = \frac{4\cdot\mathrm{TP}\cdot\mathrm{TN}}{4\cdot\mathrm{TP}\cdot\mathrm{TN} + (\mathrm{TP} + \mathrm{TN}) \cdot (\mathrm{FP} + \mathrm{FN})}. Let us consider a medical test used to detect a rare disease. Suppose a population size of 100000 and 0.05% of the population is infected.
Further suppose the following test performance: 95% of all positive individuals are classified correctly (TPR=0.95) and 95% of all negative individuals are classified correctly (TNR=0.95). In such a case, due to high population imbalance and in spite of having high test accuracy (0.95), the probability that an individual who has been classified as positive is in fact positive is very low: P(+ \mid C{+}) = 0.0095.
We can observe how this low probability is reflected in some of the metrics: \mathrm{P}_4 = 0.0370 , \mathrm{F}_1 = 0.0188 , \mathrm{J} = \mathbf{0.9100} (Informedness / Youden index), \mathrm{MK} = 0.0095 (Markedness). P(C-|-) = 0.01.
Not all metrics are notice this low probability: \mathrm{P}_4 = 0.0388 , \mathrm{F}_1 = \mathbf{0.9478} , \mathrm{J} = 0.0099 (Informedness / Youden index), \mathrm{MK} = \mathbf{0.8183} (Markedness).
Text derived from https://en.wikipedia.org/w/index.php?oldid=1321575995 under CC BY-SA 4.0; modifications have been made.