perf_wilcox {diffuStats} | R Documentation

## Compute column-wise statistics in a performance matrix

### Description

Function `perf_wilcox` compares all the columns of a matrix through a `wilcox.test`. The columns are assumed to be performance measures (e.g. AUROC) whereas the rows are instances.

### Usage

    perf_wilcox(perf_mat, adjust = function(p) stats::p.adjust(p, method = "fdr"),
        ci = 0.95, digits_ci = 2, digits_p = 3, ...)

### Arguments

`perf_mat` | Numeric matrix whose columns contain performance metrics of different methods.
`adjust` | Function to adjust the p-values for multiple testing. By default, `p.adjust` with its default parameters is used.
`ci` | Numeric, confidence interval (defaults to `0.95`)
`digits_ci` | Integer, digits to display in the confidence interval
`digits_p` | Integer, digits to display in the p-value
`...` | further arguments for `format`

### Details

The statistical comparison of the columns is intended to ease comparisons between methods in a rigorous way. Methods are compared pairwise and a p-value for difference in performance. The function `perf_wilcox` returns a character matrix so that (1) the upper triangular matrix contains confidence intervals on the estimate of the difference between performances, and (2) the lower triangular matrix contains the two-tailed p-value that tests difference in performance, with multiple testing correction. The comparison takes place between row and column in that precise order: a positive difference favours the row and a negative one, the column.

### Value

Character matrix. The upper triangular matrix contains a confidence interval and the estimate of the pairwise difference in performance. The lower triangular matrix shows the associated two-tailed p-value, with multiple testing correction.

### Examples

    # Dummy data frame to test
    n <- 100
    perf_mat <- cbind(
        good = runif(n = n, min = 0.5, max = 1),
        so_so = runif(n = n, min = 0.2, max = 0.7),
        bad = runif(n = n, min = 0, max = 0.5)
    )
    wilcox_mat <- perf_wilcox(perf_mat)

    # See how the methods in the rows compare to those
    # in the columns, confidence interval
    # (upper) and p-value (lower)
    wilcox_mat

[Package diffuStats version 0.104.0 Index]