GenerateLearningsets           package:CMA           R Documentation

_R_e_p_e_a_t_e_d _D_i_v_i_s_i_o_n_s _i_n_t_o _l_e_a_r_n- _a_n_d _t_e_t_s _s_e_t_s

_D_e_s_c_r_i_p_t_i_o_n:

     Due to very small sample sizes, the classical division
     learnset/testset does not give accurate information about the
     classification performance. Therefore, several different divisions
     should be used and aggregated. The implemented methods are
     discussed in Braga-Neto and Dougherty (2003) and Molinaro et al.
     (2005) whose terminology is adopted.

     This function is usually the basis for all deeper analyses.

_U_s_a_g_e:

     GenerateLearningsets(n, y, method = c("LOOCV", "CV", "MCCV", "bootstrap"), fold = NULL, niter = NULL, ntrain = NULL, strat = FALSE)

_A_r_g_u_m_e_n_t_s:

       n: The total number of observations in the available data set.
          May be 'missing' if 'y' is provided instead.

       y: A vector of class labels, either 'numeric' or a 'factor'.
          _Must_ be given if 'strat=TRUE' or 'n' is not specified.

  method: Which kind of scheme should be used to generate divisions
          into learning sets and test sets ? Can be one of the
          following:

          "_L_O_O_C_V" Leaving-One-Out Cross Validation.

          "_C_V" (Ordinary) Cross-Validation. Note that 'fold' must as
               well be specified.

          "_M_C_C_V" Monte-Carlo Cross Validation, i.e. random divisions
               into learning sets with 'ntrain'(s.below) observations
               and tests sets with 'ntrain' observations.

          "_b_o_o_t_s_t_r_a_p" Learning sets are generated by drawing 'ntrain'
               times with replacement from all observations. Those not
               drawn not all form the test set.

    fold: Gives the number of CV-groups. Used only when 'method="CV"'

   niter: Number of iterations (s.'details).'

  ntrain: Number of observations in the learning sets. Used only when
          'method="MCCV"' or 'method="bootstrap"'.

   strat: Logical. Should stratified sampling be performed, i.e. the
          proportion of observations from each class in the learning
          sets be the same as in the whole data set ?

          Does not apply for 'method = "LOOCV"'.

_D_e_t_a_i_l_s:


        *  When 'method="CV"', 'niter' gives the number of times the
           whole CV-procedure is repeated. The output matrix has then
           'fold'x'niter' rows. When 'method="MCCV"' or
           'method="bootstrap"', 'niter' is simply the number of
           considered learning sets.

        *  Note that 'method="CV",fold=n' is equivalent to
           'method="LOOCV"'.

_V_a_l_u_e:

     An object of class 'learningsets'

_A_u_t_h_o_r(_s):

     Martin Slawski martin.slawski@campus.lmu.de

     Anne-Laure Boulesteix <URL: http://www.slcmsr.net/boulesteix>

_R_e_f_e_r_e_n_c_e_s:

     Braga-Neto, U.M., Dougherty, E.R. (2003).

     Is cross-validation valid for small-sample microarray
     classification ?

     _Bioinformatics, 20(3), 374-380_

     Molinaro, A.M., Simon, R., Pfeiffer, R.M. (2005).

     Prediction error estimation: a comparison of resampling methods.

     _Bioinformatics, 21(15), 3301-3307_

_S_e_e _A_l_s_o:

     'learningsets', 'GeneSelection', 'tune', 'classification'

_E_x_a_m_p_l_e_s:

     # LOOCV
     loo <- GenerateLearningsets(n=40, method="LOOCV")
     show(loo)
     # five-fold-CV
     CV5 <- GenerateLearningsets(n=40, method="CV", fold=5)
     show(loo)
     # MCCV
     mccv <- GenerateLearningsets(n=40, method = "MCCV", niter=3, ntrain=30)
     show(mccv)
     # Bootstrap
     boot <- GenerateLearningsets(n=40, method="bootstrap", niter=3)
     # stratified five-fold-CV
     set.seed(113)
     classlabels <- sample(1:3, size = 50, replace = TRUE, prob = c(0.3, 0.5, 0.2))
     CV5strat <- GenerateLearningsets(y = classlabels, method="CV", fold=5, strat = TRUE)
     show(CV5strat)

