A Simple Stochastic Regularization Technique for Avoiding Overfitting in Low Resource Image Classification

Abstract

Drop type technique, as a method that can effectively regulate the co-adaptations and prediction ability of neural network units, is widely used in model parameter optimization to reduce overfitting problems. However, low resource image classification faces serious overfitting problems, and the data sparsity problem weakens or even disappears the effectiveness of most regularization methods. This paper is inspired by the value iteration strategy and attempts a Drop type method based on Metcalfe's law, named Metcalfe-Drop. The experimental results indicate that using Metcalfe-Drop technique as a basis to determine parameter sharing is more effective than randomly controlling neurons according to a certain probability. Our code is available at https://gitee.com/giteetu/metcalfe-drop.git.