What is Bagging, random forest, out-of-bag error?

Bagging (bootstrap aggregation) averages models fitted to bootstrap samples of the training set. A random forest bags deep trees and, at each split, considers only a random subset of the features, to decorrelate the trees. The out-of-bag error scores each training row with the trees whose bootstrap sample did not contain it.