Multi-objective parametric synthesis of loss functions in neural networks with evolutionary algorithms

¹ Reshetnev Siberian State University of Science and Technology 31, Krasnoyarskii rabochii prospekt, Krasnoyarsk, 660037, Russian Federation
² Siberian Federal University 79, Svobodnii ave., Krasnoyarsk, 660041, Russian Federation

^* Corresponding author: morozoveduardmsd@gmail.com

Abstract

The loss function is a fundamental aspect of neural network training and by choosing a suitable one, better results can be achieved. In classification problems, the cross-entropy loss function is almost exclusively used. In this paper the loss function represented by Taylor's series which are optimized with multi-objective evolutionary algorithm. As results show the new loss function can be better than cross-entropy, however application of multi-objective algorithm does not bring an improvement in comparison with single-objective algorithm.