A regularization technique for deep neural networks that generalizes tabu dropout by introducing a tabu tenure - the number of epochs a neuron is prohibited from being dropped after being dropped once - and then selecting that tenure automatically during training. The adaptive variant treats tenure selection as a multi-armed bandit problem, using the negative per-epoch loss as reward and exploration policies such as epsilon-greedy and softmax to converge on the best-performing tenure, improving generalization over standard dropout and basic tabu dropout.
Tabu Dropout is a dropout variant that forbids a neuron from being dropped in two consecutive training epochs, using a one-epoch tabu list to diversify the 'thinned' sub-networks produced each epoch.…