Dropout

Dropout works by randomly 'dropping out' (setting to zero) a proportion of neurons and their connections during training. This prevents neurons from co-adapting too much and forces the network to learn more robust features. During inference, all neurons are used but their outputs are scaled according to the dropout rate used during training.

Deep neural network training, Preventing overfitting, Ensemble learning approximation