Continuing the trip into the land of accuracy measures, today we’ll visit the precision recall F-score trio.
In R, assuming that `y`

is a logical vector with true classes and `p`

is a logical vector with predictions, we have:

```
precision<-function(y,p) sum(y&p)/sum(p)
recall<-function(y,p) sum(y&p)/sum(y)
```

While F-score is defined as a harmonic mean of precision and recall, it is usually calculated as such; fortunately we can do some algebra, getting $F:=\frac{2}{\frac{1}{\text{precision}}+\frac{1}{\text{recall}}}=\frac{2\sum y\wedge p}{\sum p+\sum y}.$ Nicer, but there is also $\sum{p\vee y}=\sum p+\sum y-\sum y\wedge p$, so that $F=\frac{2}{1+\frac{\sum p\vee y}{\sum p\wedge y}},$ which translates into this sweet one-liner:

`Fscore<-function(y,p) 2/(1+sum(y|p)/sum(y&p))`

Sometimes, especially in *sparse* cases (when there are few positive objects), we only have a vector of names, indices or ids of both real positives and positive predictions; then, the `%in%`

operator becomes very handy:

```
precisionS<-function(Y,P) mean(P%in%Y)
recallS<-function(Y,P) mean(Y%in%P)
FscoreS<-function(Y,P) mean(c(Y,P)%in%intersect(P,Y))
```

The latter is also a nice interpretation of F-score; one can say it measures to what extent the sets of positives and positive predictions overlap.
To this end, it is obvious that F is invariant to true negatives (i.e., correctly predicted negative objects); one may add as many of them as one whishes, and F will stay the same, while measures like accuracy or AUROC will start to converge to 1.
This may be either good and bad, depending on the use-case.
Generally F is good for *detection* problems, where there are trivial negatives and one looks for an outliers or objects following certain pattern.

All of those three measures critically relay on proper labelling of classes, both in `y`

and `p`

; trashy F value often means this is messed up.
In contrast to AUROC, there is no easy *flipping formula*; it depends on the dataset size and the fraction of positive objects.

You’ll find the code here.

Previously: Augh-ROC, later: Relevance and redundancy.