Hybrid pre training algorithm of Deep Neural Networks

St. Petersburg State University, Faculty of applied mathematics and control processes, 199034 St. Petersburg, Russia

^a Corresponding author: ivan.s.drokin@bk.ru

Abstract

This paper proposes a hybrid algorithm of pre training deep networks, using both marked and unmarked data. The algorithm combines and extends the ideas of Self-Taught learning and pre training of neural networks approaches on the one hand, as well as supervised learning and transfer learning on the other. Thus, the algorithm tries to integrate in itself the advantages of each approach. The article gives some examples of applying of the algorithm, as well as its comparison with the classical approach to pre training of neural networks. These examples show the effectiveness of the proposed algorithm.