Blind signal deconvolution based on pulsed neuron model

In this paper, we consider the vector-matrix model of a pulsed neuron, focused on solving problems of digital signal processing. We extend the application domain of the model to the blind signal deconvolution problem. To achieve this goal we propose an unsupervised learning algorithm, which maximizes the absolute value of the normalized kurtosis of the output signal of the deconvolution filter using vector-matrix model of a pulsed neuron. To show the validity of the proposed learning algorithm, some examples of deconvolution of speech signals distorted by reverberation are presented.


Introduction
In recent years, much attention has been paid to pulsed neural networks (PNNs), which are referred to third generation neural networks [1]. Models of such networks are biologically inspired and represent information in the form of pulse sequences (spikes) [2,3]. Analysis of the potential properties of PNNs for processing of signals represented by pulse sequences are quite important for the further development of adaptive systems which processing sensory signals in real time.
Representation of signals in the form of pulse sequences has great potential in areas of critical energy consumption such as sensor networks, autonomous systems, mobile computing, various biomonitoring and health monitoring systems, brain-computer interfaces, and neuro prosthetics [4].
In this paper, we study the vector-matrix model of the multi-input pulsed neuron (MIPN), which is focused on digital signal processing tasks [5]. The supervised learning algorithms for the MIPN were considered in [5][6][7][8]. These algorithms were used to synthesize various filters with arbitrary impulse responses.
The goal of this paper is to develop an unsupervised learning algorithm of the MIPN to solve the task of blind deconvolution -a recover signal distorted by a linear filter with an unknown impulse response. To solve the blind deconvolution task we use the well-known approach based on maximizing the non-Gaussianity of the output signal of a deconvolution filter [9]. Computer simulation of blind deconvolution process of speech signals distorted by the reverberation effect shows that proposed unsupervised learning algorithm allows MIPN to successfully recover the source speech signal. * Corresponding author: vbondarev00@mail.ru

Model of multi-input pulse neuron
Consider the multi-input pulsed neuron model [5]. In accordance with this model (Fig. 1), the input bipolar pulse sequences u i (t) formed by coding neurons are fed to the inputs of linear filters with pulse responses h i (t). These filters simulate the dynamics of synaptic connections. The filter reactions x i (t) are weighted by synaptic weights w i and summed to form the output signal y 0 (t). To find the discrete expression for y 0 (t) we will follow the approach outlined in [5] and introduce the sliding binary vector ) (n i b whose elements are equal to where Δt is the time sampling step, n is the step number, t n is the discrete time, t i j is the time corresponding to the occurrence of the impulse j at the output of presynaptic neuron i, λ i j = ±1 is the sign of the impulse j at input i, k=0,1,...,K-1 and ] . Then the reactions of the each linear filter to the input impulse sequence at a discrete instant of time t n will be written as [5] x where ) T denotes the impulse response vector of the linear filter i. The output signal of the MIPN will be equal to Expressions (2) and (3) represent the digital model of the multi-input pulse neuron and can be successfully used to synthesize various filters capable of processing pulse sequences. Fig. 2 shows a synthesis scheme for various filters using an adaptive modeling approach. A distinctive feature of the scheme is the implementation of the adaptive filter by means of a serial connection of the coding IAF-neuron (Integrate-And-Fire) and the adaptable MIPN. The coding IAF-neuron converts the input signal into a bipolar pulse sequence f(t), which is processed by the MIPN [8]. To get a set of u i (t) input sequences, we use a delay line with I taps.
As a result of adaptation, the MIPN parameters are adjusted so that the mean square of error e(t) is minimized and the specified requirements for the dynamic properties of the adaptive filter are satisfied. These requirements in the scheme are set by the properties of the reference filter having the desired impulse response or the transfer function. Formally, for the synthesis of the desired filter using the adaptive modeling scheme, it is necessary to find the MIPN weighting vector w, which provides a minimum of the objective function where E is the mathematical expectation, e(n)= y 0 (n) − y d (n) is the error that is equal to the difference between the output signal of the MIPN y 0 (n) and the output signal of the reference filter (desired signal) y d (n).
To find the quasi-optimal values of the weighting vector w, we apply the standard descending gradient approach. Taking into account (3), the gradient of the quadratic functional (4) over the weighting vector w will be equal to Having performed the stochastic approximation, we obtain the algorithm that provides the necessary adaptation of the MIPN dynamic properties w(n)= w(n-1)+α(n)e(n)x(n), where α(n) is the learning rate. Algorithm (6), which we derived for MIPN training, in its form corresponds to the standard LMS algorithm for training ordinary neural networks. The difference is only in the vector x (n), which is calculated using (2). The considered scheme of adaptive modeling is based on the supervised learning of the MIPN model and can be used to solve various tasks of signal processing: adaptive restoration of the original source signal from the output pulse sequence of the coding IAFneuron, suppression of additive noise [5], synthesis of filters with a complex shape of the amplitude-frequency response [7].
Since the desired output signal y d (n) is not available when solving the blind deconvolution task, it is impossible to use the supervised learning algorithm (6) to solve this task. Therefore, for this purpose, we need to develop an unsupervised MIPN learning algorithm.

Blind deconvolution based on the MIPN model
In the general case, an adaptive deconvolution filter provides a reconstruction of the source signal s(n) from observations of the output signal u 1 (n) of some distorting filter u 1 (n)=p(n)*s(n), , 0 where p(n) is an unknown impulse response of the distorting filter, * is the symbol of convolution operation. Since the desired signal y d (n) is not available when solving the blind deconvolution task, it is possible to use the well-known self-learning blind deconvolution algorithms based on maximizing the non-Gaussianity of the output signal of the recovery filter (deconvolution filter). In this case, we assume that the distribution of the source signal s(n) differs from the normal distribution [9].
As a measure of non-Gaussianity, the absolute value of the normalized kurtosis k(y 0 ) is usually used [9]. Then we can use the following objective function for minimization where y 0 is the output signal of the deconvolution filter. It is assumed that the signal u 1 (n) is prewhitened.
Using the stochastic gradient descent approach to objective function (8), we derive the following MIPN learning algorithm w(n)= w(n−1)+α(n) β F(y 0 ) x(n). (9) Here F(y 0 ) is the nonlinear activation function where m 2 (y 0 ), m 4 (y 0 ) are the 2nd and 4th order moments; β = −1 when recovering signals with sub-Gaussian distributions and β = 1 when recovering signals with super-Gaussian distributions. Learning algorithm (9) can be considered as a BLMS (Blind Least Mean Square) algorithm resembling the standard LMS learning algorithm (6), where e(n)=γ y 0 3 (n) − y 0 (n), γ= m 2 (y 0 )/m 4 (y 0 ). In the presented form, learning algorithm (9) is similar to the well-known deconvolution algorithms based on the criterion of the maximum absolute value of the normalized kurtosis [9,10]. The difference lies in the way of calculating the vector x(n). Fig.3 shows the adaptive filter training scheme with MIPN when solving the blind deconvolution task. The whitening filter provides the formation of statistically independent samples u(n) arriving at the input of the adaptive filter with MIPN.

Simulation results
The adaptive deconvolution filter with the MIPN model was trained using the algorithm (9). The task was to recover the original speech signal s(n) from the convolution u(n)=p(n)*s(n), where p(n) is the unknown impulse response of the distorting filter. A second order recursive filter with the large feedback delay T = 50 (2.3 ms) was used as a distorting unknown filter, which creates a reverberation effect [10]: where the coefficients were a 1 = −0.7, a 1 = 0.35. The blind deconvolution task is solved if where g is an unknown gain, and d is an unknown delay. In this case, the original source signal s(n) is recovered in the MIPN output with the possible exception of an arbitrary gain g and delay d. Fig. 4 shows the simulation results. The impulse responses of the input filters included in the MIPN and modeling the shape of postsynaptic potentials had the same form of a symmetric exponential function with a time constant τ = 0.0001 s. The values of the MIPN hyperparameters were determined taking into account the sampling frequency and the effective duration of the impulse response of the distorting recursive filter and amounted to: I = 809, K = 129. The distorted signal was prewhitened using a fifth order linear prediction algorithm. Fig. 4a shows a normalized fragment of the source speech signal s(n). Fig. 4b displays the normalized distorted speech signal as a result of the reverberation produced by the second order recursive filter (11). Fig. 4c shows the recovered speech signal which is the output signal of the MIPN y 0 (n). As follows from Fig. 4c the graph of the MIPN output signal y 0 (n) is quite close to the source signal s(n).
To quantify the accuracy of the signal deconvolution, we calculated the correlation coefficient r(s,u 1 ) between the source signal s(n) and the distorted signal u 1 (n), as well as the correlation coefficient r(s,y 0 ) between the source signal s(n) and recovered signal y 0 (n). The indicated correlation coefficients for the signals shown in Fig. 4 are equal to r(s,u 1 )=0.235 and r(s,y 0 ) = 0.962.

Conclusion
The dynamics of MIPN can be described using the vector-matrix model (2)-(3). The MIPN model (2)-(3) allows you to transfer well-known digital signal processing algorithms into the field of pulsed neural networks. Using this opportunity, we developed the MIPN learning algorithm (9) to solve the task of the blind signal deconvolution based on the criterion for maximizing the non-Gaussianity of the recovered signal.
The results of computer simulation show that the algorithm (9) used to train the MIPN model is able to recover speech signals distorted by the reverberation process.