batchByIndex {ChemmineR} | R Documentation |
When doing a select were the condition is a large number of ids it is not always possible to include them in a single SQL statement. This function will break the list of ids into chunks and allow the indexProcessor to deal with just a small number of ids.
batchByIndex(allIndices, indexProcessor, batchSize = 1e+05)
allIndices |
A vector of values that will be broken into batches and passed as an argument to the
|
indexProcessor |
A function that takes one batch if indices. It is called once for each batch. The return value from this function is ignored. To accumulate results you can write to a global variable using the "<<-" operator. |
batchSize |
The size of each batch. The last batch may be smaller than this value. |
No value is returned.
Kevin Horan
## Not run: result=NA indices = 1:10000 #run a query on each batch of indexes, appending each result to # "result" as we go. batchByIndex(indices, function(indexBatch){ df = dbGetQuery(dbConnection, generateQuery(indexBatch)) result <<- if(is.na(result)) df else rbind(result,df) },1000) ## End(Not run)