Optimal Control Methods of Experiment Times in System-of-Systems Combat Computer Simulation

In the process of scheme optimization, in order to eliminate the influence of random factor, it needs to conduct computer simulation of Monte Carlo. Therefore, it is proposed to introduce confidence interval into systemof-systems combat simulation, and confirm whether the Monte Carlo simulation finishes according to data sample generated in simulation process. According to characteristic of data sample, extend correspondingly confidence interval method, and under the condition of obtaining the solution meeting accuracy requirements, reduce simulation experiment times as far as possible. The simulation experiment results show that confidence interval extension method is able to possess self-adaptation control to Monte Carlo simulation.


Introduction
The characteristic of weapon system-of-systems combat computer simulation is: Large battlefield scope, long lasting time, wide varieties of weapon, large quantity, complex combat action and various forms of confrontation effects [1] . Due to the limitation of the complexity of systemic confrontation and restrictions of mathematics methods, the ability to establish mathematical models for problems research and to optimize them is very limited, so it needs to introduce emulation technique. In the process of adopting computer simulation technique to research the problem, the simulation model established usually has many random factors; In order to eliminate the influence of random factors to the evaluation of alternative schemes, it needs to rerun the alternative scheme for several times.
Through bibliographic retrieval, the methods used to confirm experiment times n mainly includes 3 kinds [2] : Expert experience method, simple figure line method, confidence interval method. Currently, the expert experience method could be widely used in simulation experiment [3][4][5] . This method could be simple to use, but not consider the characteristics of model output, therefore, it may cause computing resource waste or inadequate accuracy of solution. In simple figure line method, draw a graph according to accumulative mean value of a series of computational results, and advise to run 10 times at least. The more running times, the flatter the line would be. In this way, the user can select some points which the accumulative mean value tends to be flat in the graph and take it as experiment times. The advantage of such method is easy to understand and execute, just like selecting output interested in decision-making process; the disadvantage is that the user needs to participate in the process, with larger subjectivity.
In confidence interval method, the user is required to give the maximum allowable error in model mean value estimation, which means the accuracy of solution; In process of simulation experiment, adopt running-solving methods for loop iteration until the output result meets solution accuracy. The advantage of such method is to confirm experiment times through statistic inference, and it is a self-adaptive control method; the disadvantage may be that it may lead to the accuracy of the solution is not enough.
In view of the above reasons, the text extends the confidence interval methods to realize the self-adaptive control of experiment times in system simulation optimization.

Confidence interval method 2.1 Method introduction
For experimental scheme A, rerun n times, and assume that the value of some performance index which the i time simulation running is corresponding to i x , then the mean value 1 2 , , of ( ) x n is the unbiased estimation of their expectation. The confidence interval of μ with confidence coefficient of 100(1 )   is [6] : where, n is the time of scheme running; distribution of the student with degree of freedom of n -1, confidence coefficient of 100(1-α); S(n) is the standard deviation of sample x1, x2, L, xn, and the computing method is as follows: It can be known from above analysis that in the process of simulation experiment, the confidence interval can be introduced to determine the repetition times n of experimental scheme.

Finish criteria
In the process of simulation running, the sample data may restrain to some incorrect value earlier, then start to bifurcate. In order to avoid such circumstance, the algorithm needs to set a testing method. When the relative error n d is less than or equal to accuracy requirement r d

Algorithm flow
Step 1. Initialize minimum times min n , reference times ref n , testing times rc n and solution accuracy r d .
Step 2. Set min n n  , rerun the experimental scheme n times.
Step 3. Through running-solving, carry out loop iteration.
Step 3.1 Calculate mean value ( ) x n , with accuracy of ( ) r n   .
Step 3.2 If ( ) rc r n d    , transfer to Step 3.4.
Step 3.3 Run the experimental scheme one time, set 1 n n   , and transfer to Step 3.1.
Step 3.4 Calculate testing times oc n .
Step 3.5.1 Run experimental scheme one time, and set 1 i i   .
Step 3.5.2 Calculate mean value ( 1) x n  , with accuracy of ( ) r n i    . Step 4. Output experimental result ( ) x n ; Operating n times.

Experiment and the analysis of results
In system-of-systems combat simulation, the data simple generated in multiple running of single scheme are mainly subject to two kinds of random distribution: Normal distribution, Bernoulli distribution. Therefore, in order to test the performance of algorithm, use data sample generated in static state (N(μ, σ) means normal distribution; B(1, μ) means Bernoulli distribution) listed in Table 1 to conduct experiment; Set the confidence coefficient of sample mean expectation μ as 0.9, that is set solution accuracy as 0.05; nmin is set as 5. In order to eliminate the influence of data series, in experiment, each static distribution randomly generates 1000 groups of different data series. In the process of algorithm performance testing, the data sample size required may be much larger than 46, but currently available data only gives the data of t distribution in 46 n  ; Therefore, when 46 n  , the algorithm introduces ( ) t n  approximate calculation of solving way given in literature [7] , namely ( ) t n z    .
Though there may be certain error for adopting such approximate value, its relative error shall not exceed 1.3% at most; Therefore, the influence to experiment result is very little. In Table 1 Table 1, when the data sample generated in Monte Carlo simulation process is subject to normal distribution, the confidence interval method is adopted and all solutions can reach to accuracy requirements; But if the data sample generated in Monte Carlo simulation process is subject to Bernoulli distribution, the confidence interval method is adopted, and partial solutions fail to reach to accuracy requirements, which mainly because the value of data sample only has 0 and 1. If the uniformity of data sample generated in the initial phase of Monte Carlo simulation is poor, it is easy to cause earlier finish of simulation. For example, data sample subjects to B(1, 0.9), but the first 15 samples generated by simulation are {1,1,1,1,1,1,1,1,1,1,1,1,1,1,1}, it is easy to obtain that the sample mean value as 1 according to algorithm.

Normal distribution
Through analysis, we could see that: In the process of simulation experiment, if the data sample generated possesses good uniformity, the sample mean value generated by a small amount of data samples should meet solution accuracy. Based on this, the uniformity of data sample is deemed as another criterion to determine whether algorithm for solving has finished in this text.
For normal distribution, there are 3 special values, that is, sample mean value x , sample mean value M x , distribution, transfer to Step5.
Step 4. Adjust min n , rerun experiment scheme min n n  , and set min n n  .
Step 5. Through running-solving, carry out loop iteration.
Step 5.1 Calculate mean value ( ) x n , with accuracy of ( ) r n   .
Step 5.2 If the data sample is subject to Bernoulli distribution, transfer to Step5.4; Otherwise, adopt the method in 3.1.1 to test the data sample, when meet requirements, register 1 node node n n   ; Otherwise, register 0 node n  .
Step 5.3 Calculate testing times oc n  , in case of node oc n n   , transfer to Setp6.
Step 5.4 In case of ( ) r r n d    , transfer to Step 5.6.
Step 5.5 Run the experimental scheme one time, set 1 n n   , and transfer to Step 5.1.
Step 5.6 Calculate testing times oc n .
Step 5.7.1 Run experimental scheme one time, and set 1 i i   .
Step 5.7.2 Calculate mean value ( ) x n i  , with accuracy of ( ) r n i    .
Step 5.7.3 If the data sample is subject to Bernoulli distribution, transfer to Step5.7.5; Otherwise, adopt the method in 3.1.1 to test the data sample, when meet requirements, register 1 node node n n   ; Otherwise, register 0 node n  .
Step 5.7.4 Calculate testing times oc n  , in case of node oc n n   , transfer to Setp6.
Step 5.7.5 In case of ( ) r r n i d     , set n n i   and transfer to Step5.5.
Step 5.7.6 In case of oc i n  , set oc n n n   , and transfer to Step 6; Otherwise, transfer to 5.7.1.
Step 6. Output experimental result ( ) x n ; Operating n times.

Simulation experiment
In order to test the validity of algorithm, conduct experiment to static distribution listed in Table 1 again, and set the solution accuracy as 0.05 and 0.10. The results of the experiment are shown in Table 2.
It can be known through comparison of data in Table  1 and Table 2 that, after algorithm extension, when the data sample generated in the Monte Carlo simulation process is subject to normal distribution, under the circumstance of ensuring the accuracy requirements of solution, the times of simulation experiment can be reduced; When the data sample generated in the Monte Carlo simulation process is subject to Bernoulli distribution, it can reach to accuracy requirements of solution, and when the mean value is less than 0.9, the difference between simulation experiment times and theoretical value is small.

Conclusion
Add the confidence interval extension method into system-of-systems combat simulation as a module of management and control of member, which can selfadaptively control the times of Monte Carlo emulation, so that the solution calculated can meet accuracy requirements and the computing resource will not be wasted. However, since the rerunning times of scheme is directly proportion to , it needs to cautiously give corresponding r d for estimation of  and  before system-of-systems combat simulation to avoid the waste of computing resource.