| perf_wilcox {diffuStats} | R Documentation |
Function perf_wilcox compares all the columns of a matrix
through a wilcox.test.
The columns are assumed to be performance measures (e.g. AUROC)
whereas the rows are instances.
perf_wilcox(perf_mat, adjust = function(p) stats::p.adjust(p, method = "fdr"),
ci = 0.95, digits_ci = 2, digits_p = 3, ...)
perf_mat |
Numeric matrix whose columns contain performance metrics of different methods. |
adjust |
Function to adjust the p-values for multiple testing.
By default, |
ci |
Numeric, confidence interval (defaults to |
digits_ci |
Integer, digits to display in the confidence interval |
digits_p |
Integer, digits to display in the p-value |
... |
further arguments for |
The statistical comparison of the columns is intended to
ease comparisons between methods in a rigorous way.
Methods are compared pairwise and a p-value for difference in
performance.
The function perf_wilcox returns a character matrix so that
(1) the upper triangular matrix contains confidence intervals
on the estimate of the difference between performances, and
(2) the lower triangular matrix contains the two-tailed p-value
that tests difference in performance, with multiple testing correction.
The comparison takes place between row and column in that precise order:
a positive difference favours the row and a negative one, the column.
Character matrix. The upper triangular matrix contains a confidence interval and the estimate of the pairwise difference in performance. The lower triangular matrix shows the associated two-tailed p-value, with multiple testing correction.
# Dummy data frame to test
n <- 100
perf_mat <- cbind(
good = runif(n = n, min = 0.5, max = 1),
so_so = runif(n = n, min = 0.2, max = 0.7),
bad = runif(n = n, min = 0, max = 0.5)
)
wilcox_mat <- perf_wilcox(perf_mat)
# See how the methods in the rows compare to those
# in the columns, confidence interval
# (upper) and p-value (lower)
wilcox_mat