buildBench {SummarizedBenchmark} | R Documentation |
Function to evaluate BenchDesign
methods on supplied
data set to generate a SummarizedBenchmark
. In addition
to the results of applying each method on the data, the returned
SummarizedBenchmark
also includes metadata for the methods
in the colData
of the returned object, metadata for the
data in the rowData
, and session information in the
metadata
.
buildBench(bd, data = NULL, truthCols = NULL, ftCols = NULL, sortIDs = FALSE, keepData = FALSE, catchErrors = TRUE, parallel = FALSE, BPPARAM = bpparam())
bd |
|
data |
Data set to be used for benchmarking, will take priority over data set specified to BenchDesign object. Ignored if NULL. (default = NULL) |
truthCols |
Character vector of column names in data set corresponding to
ground truth values for each assay. If specified, column will be added to
the |
ftCols |
Vector of character names of columns in data set that should be included as feature data (row data) in the returned SummarizedBenchmark object. (default = NULL) |
sortIDs |
Whether the output of each method should be merged and sorted using IDs. See Details for more information. (default = FALSE) |
keepData |
Whether to store the data as part of the BenchDesign slot of the returned SummarizedBenchmark object. If FALSE, a MD5 hash of the data will be stored with the BenchDesign slot. (default = FALSE) |
catchErrors |
logical whether errors produced by methods during evaluation should be caught and printed as a message without stopping the entire build process. (default = TRUE) |
parallel |
Whether to use parallelization for evaluating each method.
Parallel execution is performed using |
BPPARAM |
Optional |
Parallelization is performed across methods. Therefore, there is currently no benefit to
specifying more cores than the total number of methods in the BenchDesign
object.
By default, errors thrown by individual methods in the BenchDesign
are caught
during evaluation and handled in a way that allows buildBench
to continue
running with the other methods. The error is printed as a message, and the corresponding
column in the returned SummarizedBenchmark
object is set to NA. Since
many benchmarking experiments can be time and computationally intensive, having to rerun
the entire analysis due to a single failed method can be frustrating. Default error catching
was included to alleviate these frustrations. However, if this behavior is not desired,
setting catchErrors = FALSE
will turn off error handling.
If sortIDs
is TRUE, each method must return a named vector or list. The names will be
used to align the output of each method in the returned SummarizedBenchmark. Missing values
from each method will be set to NA. This can be useful if the different methods return
overlapping, but not identical, results. If truthCols
is also specified, and sorting
by IDs is necessary, rather than specifying 'TRUE', specify the string name of a column in
the data
to use to sort the method output to match the order of truthCols
.
When a method specified in the BenchDesign does not have a postprocessing function specified
to 'post', the trivial base::identity
function is used as the default postprocessing
function.
SummarizedBenchmark
object.
Patrick Kimes
## with toy data.frame df <- data.frame(pval = rnorm(100)) bench <- BenchDesign(data = df) ## add methods bench <- addMethod(bench, label = "bonf", func = p.adjust, params = rlang::quos(p = pval, method = "bonferroni")) bench <- addMethod(bench, label = "BH", func = p.adjust, params = rlang::quos(p = pval, method = "BH")) ## evaluate benchmark experiment sb <- buildBench(bench) ## evaluate benchmark experiment w/ data sepecified sb <- buildBench(bench, data = df)