Skill Oriented Online Master’s Course “Neural Network Modeling of Complex Technical Systems”

In this paper we consider the application of online course “Neural Network modeling of Complex Technical Systems” in the Master’s degree programs in the field of nanotechnology and nanoengineering in Bauman Moscow State Technical University. The course has rather practical than theoretical nature. The aim of this course is skill oriented learning. Nowadays neural network models have become a powerful tool of scientific research for engineers and students. The methods studied during the study of the discipline can be applied to estimation, modeling, classification, clustering, forecasting and more. The neural networks modeling plays a significant role in Master’s education and student’s research work. Neural Networks models are successfully presented in graduation theses. Thanks to online educations students can practice at their own pace and study modern neural networks software products, methods of data preparing, designing and training neural network and then apply these algorithms in practice. According to the steps of neural network modeling algorithm the course consists of three main parts and conclusive one. In this paper course structure and study results are presented.


Introduction
The rapid development of nanotechnology determines the demand for skilled specialists. Besides graduation success depends a lot on confided usage of common methods, algorithms and tools of statistical community special attention in degree programs is given to data analysis and modeling methods. Master's degree programs in the Chair of Electronic Technologies in Mechanical Engineering of Bauman Moscow State Technical University include the course "Neural Network modeling of Complex Technical Systems". Neural networks can be used in technological process control, failure analysis, machine vision, signal identification, etc. [1,2,3].
An artificial neural network is a computing model whose layered structure is similar to the networked structure of neurons in the brain. Layers contain several connected single processing elements, called neurons. The neurons are connected by weights. These weights are tuning in during learning or training until the error becomes minimum. Neural networks are training over many examples. Training data is a specific number of observations, for each of which values of several variables are indicated. Most of these variables are set as inputs and the network will learn to find a match between the values of the input and known output variables. The mathematics of neural networks is well developed and many computer programs are available to create and train neural networks.
The course has rather practical than theoretical nature, it plays a significant role in students research work. For four semesters students of Master's degree studies under the guidance of scientific supervisors a specific scientific problem such as investigation and development of modern technologies, engineering of technical processes and equipment. Students can, depending on their background and interests, choose the research fields such as processing techniques, specialized equipment and key materials of electronics, nanoelectronics and nanoengineering. In their research, they are engaged in the development of technologies, machines and mechanisms, for example, vacuum technics [4,5], vacuum [6,7] and colloidal [8,9] thin films deposition processes, various lithographic and etching processes, etc. The range of tasks that students solve is very wide. The tasks of data analysis, forecasting, regression, clustering and classification are quite common and can be solved using neural network modeling.
Thanks to online education [10,11] students can practice at their own pace and study modern neural networks software products, methods of data preparing, designing and training neural network and then apply neural networks algorithms in practice. Online education places additional demands on teachers and students, but online courses give students an opportunity to plan a course study mode in accordance with a research plan.

Course Structure
Every neural network application is special, but developing the network follows steps: preparing the data, creating the neural network, training the neural network, validating the network's results, tuning the neural network, integrating the neural network into a project. According to the steps of neural network modeling algorithm the course consists of three main parts and conclusive one.
The first part of the course discusses applications of neural networks to practical engineering problems in modern technologies, technological and analytical equipment, sensor and control systems and gives an introduction to basic artificial neural network architectures.
The second part of the course is devoted to accessing and preparing the data and building of neural networks. It provides a clear and detailed survey of fundamental neural network types -multilayer perception, radial basis function network, probability neural networks and Kohonen neural network [12]. The third part of the course discusses training of neural networks, optimisation algorithms and learning rules: backpropagation and several variations of backpropagation, such as the conjugate gradient, Levenberg-Marquardt algorithm, gradient descent and gradient descent with adaptive learning rate [13]. At the end of this part students learn how to validate the network's results and finaltune network parameters.
The fourth part is devoted to the software products: Statistica Neural Networks [14], Neural Network Toolbox for MATLAB (now -Deep Learning Toolbox) [15] and NeuroShell [16]. Figure 1 shows the online course structure. The first three parts end with testing. Upon completion of the course, students do their homework. During the homework students learn to create, tune and train their own neural networks and get full and detailed understanding of research in neural network vision.

A Typical Homework Structure
Typical homework report includes introduction, description part, designing part, results of neural network modeling, discussion and conclusion. Simulated system or process, its input and output parameters and available statistical data are presented and analyzed in the description part. The designing part contains the rationale for choosing inputs and outputs, type of neural network, its parameters, error function and learning algorithm for network training. Description of neural network training, validating and application are presented as results of modeling. New understanding of the problem, interpreting the findings and the prospects for the use of models results are presented in the discussion section. In conclusion section the arguments involved in the body paragraphs are summarized. Thus homework report is a completed research work and therefore results obtained in the process of neural network modeling are widely used as a part of graduation thesis.

Examples of Student Works
Regression models [17] are used in student's research most often. As mentioned above, neural networks are prepared for work, i.e. directly to modeling, through training on examples, which are separate observations for which the values of the input and output parameters of the process are indicated. In accordance with the recommendations [17], the

Neural Network modeling of Complex Technical Systems
Lectures Testing

Home -work
Introduction to neural network modeling The data preparing and creating neural networks Training and testing neural networks Neural networks software number of observations included in the data set and depending on the number of relationships superimposed on the simulated process should be several hundred. When modeling processes associated with the operation of complex equipment, characteristic of technologies implemented in the electronic industry, the use of artificial neural networks in the homework is limited by the insufficient number of experimental results. In this case the results of modeling oriented on average values of the output parameter are obtained. Figure  2 presents the results of modeling the process of wear of a thin-film coating. The output parameter was the number of wear particles. However, understanding the prospects for using neural network algorithms encourages students to conduct large-scale research. And, as a rule, the most interested of them manage to accumulate enough statistics to represent the final variant of their modeling by the end of the master's program.
In some cases, students use data from previously performed in the laboratory experimental studies to do the homework. It was such data that was used to solve the problem of predicting the reliability of the elements of vacuum equipment [3]. Prediction of reliability indicators, in particular, mean time between failures or mean time to failure (MTTF), can be carried out at the design stage (when developing a technical specification, comparing options at the technical proposal stage and when performing a preliminary design) and during operation. When predicting product reliability at the design stage, there is the greatest uncertainty in estimating possible product states. In this case, engineering practice uses methods for predicting reliability indicators based on analysis of data on the reliability of analogous products [18]. The task of predicting or forecasting the reliability of a new product related to a certain group of functional units at the design stage is to determine the set of design parameters and modes that have a significant influence on the reliability of the units and the selection of many N units that are close to the new product in terms of reliability. Thus, to solve the problem, the following initial data were necessary: the design characteristics and the expected modes of operation of the newly developed units; the name and designation of the set of N units -analogues; information about the reliability of unit assemblies, their design characteristics and modes of operation. Sources of a priori information about the reliability of units -analogues were the results of controlled equipment operation. Constructive characteristics and modes of operation were determined directly from the design documentation.
The set of N units-analogues was formed according to the results of the functional analysis of the equipment nomenclature and the decomposition of functional systems. The following parameters of the studied units were used as input factors: the total number of parts, the number of seals, the number of welds, the thickness of the separation elements, the number of through holes, gas permeability, the number of parts with cyclic load, the number of metal seals, the number of adjustment elements, the number of parts subject to wear, the degree of sealing of the shutter, chemical resistance, sealing tightness, the number of elements susceptible to corrosion, the number of preset items, type of drive. The MTTF of the unit (in hours) was chosen as the output parameter as a measure of the reliability of the unit. A block of examples for neural network training was prepared. It included information about 181 units of vacuum process equipment with 16 parameters.
The multilayer perception (MLP) neural network was used in the simulation as the most suitable for the task of predicting reliability indicators. The results of modeling showed a quite acceptable level of validity of MTTF predicting using an artificial neural network. Comparison of the results of calculating the elements of the vacuum process equipment with the results obtained by the cluster analysis method showed a quite acceptable level of validity of predicting reliability indicators using an artificial neural network. Generally the performance of the designed network model was assessed using the root mean square error (RMSE) as a measure of goodness-of-fit. The RMSE of the training, verifying and testing subsets of data were less than 10 %. The ratios between error standard deviation and standard deviation of training data (SD ratio) were 0.05, 0.008 and 0.08 for training, verifying and testing subsets of data, respectively. The correlation coefficients never fell below 0.9.
As a rule, students present the results of neural network modeling not only in reports on homework, but also in their final works. Figure 3 shows the results of one of the studies presented in the homework and subsequently used in the graduation thesis. It illustrates the modeling of the process of colloidal photonic crystal [19] film deposition. Self-assembly [20] processes that occur during vertical deposition have been characterized by a large number of active factors. Therefore, obtaining theoretical dependencies is impossible and the use of neural networks is justified. All training data were determined directly from the results of the series of spectrophotometric measurements during the practical part of the study. The radial basic function (RBF) neural network was used in this work in the simulation as the most precise for the task of predicting nonlinear relation. It describes the relationship between the wavelength and reflectance at photonic band gap (output variable) and colloidal particles size and material, colloidal solution concentration and lifting velocity (input variables). Neural network was used to optimize the effect of the above-mentioned factors. The revealed dependences allowed to choose the optimal conditions for obtaining high photonic crystalline quality of opal film. The results of modeling can be used for various applications of photonic crystal films: nanophotonics, laser technics, plasmonics, etc. Figure 4 shows an example of modeling of metal islands film growth. Island thin films and nanostructures are in high demand in many fields of science: microelectronics and nanoelectronics, optics, photonics, laser technology, solar energy, etc. [6]. Multilayer perception (MLP) with two hidden layers was used in the simulation because of it extrapolation ability. The best way to determine the number of hidden neurons is to train several networks. Preliminary option analysis included two MLP and two RBF neural networks. It was revealed that a few number of hidden neurons result in a higher training error and too many hidden neurons results in a higher validation and testing error.

Results Analysis
The teaching methodology of the course was worked out during the classroom work. As teaching experience shows after completing the study of the discipline "Neural Network modeling of Complex Technical Systems" students understand neural networks algorithms; know modern neural networks software products; are capable to prepare data, design, train, tune and test neural networks; and are capable to put acquired knowledge and skills into practice. Students' independent research, their homework on the subject of their investigation play an important role in achieving these results. Table 1 provides a summary of the topics of homework in 2018, the data and models used in their implementation.  [14]. Several types of networks were used in every research: MLP, RBF, a linear network (Linear). MLP with one and two inner (hidden) layers were considered. The number of neurons in the MLP layers ranged from 2 to 20, and in the second RBF layer it ranged from 5 to 18.
Lecture materials are the main source of information for students. However, when doing homework, they have to independently formulate the goal and objectives of modeling, carry out an independent search for information and study in detail the algorithms that are used in the work. Thus the objective of the course is not only to introduce students to the neural network modeling, but also to develop self-learning abilities.

Conclusion
Nanoscience is a fast-paced field and teaching for future specialists in this field is very closely related with research. Due to the fact that neural network models have become a powerful tool of scientific research, knowledge and skills obtained as a result of mastering the discipline are very popular. The simulation algorithm described in the course "Neural Network modeling of Complex Technical Systems" allows to get useful information for effective student's research. Besides successful research depends on confided usage of common methods and algorithms of statistical community special attention in the course is given to data analysis and modeling methods.
Students study modern neural networks software products, methods of data preparing, designing and training neural network and then apply neural networks algorithms in practice. Neural Networks models are successfully used in students scientific and research projects and presented in Master's graduation theses. The online form allows students to focus on issues relevant to their research project and carry out independent work on the course in accordance with the plan of research.