Quantifying active brain areas at spatial hearing process using Electroencephalography (EEG) source localization approach

. The human ability for spatial hearing encourages the development of spatial audio technology to support many human activities. Spatial audio mimics the real-life sound and provides a more immersive hearing experience as if the listener were present in the environment where the sound source was recorded. Recently, research on how spatial hearing processes are encoded in the brain has begun to be developed. As a modality for brain activity measurement that is non-invasive and has a high temporal resolution, Electroencephalography (EEG) is suitable for studying brain responses to moving sound stimuli. This study compared brain activity in hearing stereo and spatial audio stimulus. Through measurements using 21 EEG electrodes on the scalp, the results showed an increase in the mean PSD for all Theta, Alpha, and Beta waves in the spatial audio stimulus compared to stereo audio. The most significant increase in the mean PSD occurred in the Beta wave of 93.8008 µV2/Hz. Furthermore, the source localization approach with Independent Component Analysis (ICA) and Low-resolution Brain Electromagnetic Tomography (LORETA) method was used to estimate and quantify the active brain area of this process. The results showed an activation of the non-auditory cortex when the subject was hearing spatial audio stimulus. By using Paired T-Test of current density for both hearing processes, the results showed there was no significant difference (p > 0.05) in Brodmann area (BA) 41 (Primary Auditory Cortex) and BA 42 (Secondary Auditory Cortex). Meanwhile, for the spatial audio hearing process, there was a significant difference (p < 0.05) in BA 6 (Premotor Cortex), which is related to spatial orientation, and BA 9 (Dorsolateral Prefrontal Cortex), which is associated with executive functions, including working memory and selective attention. This study offers potential insights into spatial hearing research and immersive audio production.


Introduction
Auditory perception provides many benefits and has become essential for living things [1].By nature, most living things have been provided with two ears, not just an auditory sensor.This auditory system with two sensors is known as binaural hearing [2].Binaural hearing analyzes the difference in information between two sensors to build a spatial representation of objects.Binaural hearing in humans enriches the world's perception because it can help to observe more information contained in sound, not only "what" but also "where."In other words, binaural hearing allows humans to measure the spatial depth of a sound source.
The human ability for spatial hearing encourages the development of audio devices with spatial audio technology to support many human activities.Most audio developers know that spatial audio mimics real-life sound.Audio is sent to the listener's ear from a different direction, similar to how listeners process sound in the physical world.This provides a more immersive hearing experience as if the listener were present in the environment where the sound source was recorded [3].
Spatial audio not only considers the sound's volume but also tracks the listener's orientation relative to the sound source.Spatial audio has several advantages when compared to two-dimensional audio: (1) Spatial audio replicates sound processed in real life so that the listener seems to be present in the environment of the sound source, (2) Spatial audio provides a dynamic and immersive experience, and (3) Spatial audio provides increased clarity and accuracy over the live sound.
In recent times, there has been a development of research focused on how the brain encodes spatial hearing processes [4,5].The use of fMRI (functional Magnetic Resonance Imaging) in dynamic brain imaging studies to identify brain regions active during sound localization is common.Unfortunately, although fMRI has a high spatial resolution, it has a low temporal resolution, making it unsuitable for studying highly dynamic stimuli such as spatial audio.Meanwhile, another brain imaging modality that may be used in the study of brain activity in spatial hearing processes is the Electroencephalogram (EEG), a non-invasive method that measures nerve activity by placing electrodes on the scalp.EEG directly measures electric fields resulting from the simultaneous activation of a large group of cortical neurons [6].As EEG has a high temporal resolution, it can capture rapid neural dynamics, making it a suitable instrument to study the location coding of a moving sound source.
EEG studies have a drawback in that they possess a low spatial resolution, which limits the analysis of brain activity to the scalp level.On the other hand, identifying the localization of sources generating electric signals can provide valuable information about various brain conditions.This is accomplished through the reconstruction of the current density distribution of EEG, which is referred to as the EEG inverse problem.One algorithm that has been extensively used for brain source localization over the last two decades is LORETA (Low-Resolution Electromagnetic Tomography).LORETA has proven to be a promising tool in various fields [7,8].
The preprocessing stage is crucial in the EEG source localization process to obtain a clean EEG signal that is free of noise and artifacts.Artifacts may arise from various sources such as muscle activity, electrical line noise, eye movement, and heartbeat.However, removing the affected data segment directly could result in a considerable loss of information.To overcome this issue, Independent Component Analysis (ICA) has been proposed as a powerful approach that identifies the artifactual content of the EEG dataset in a few signals, enabling us to reject them without canceling the entire affected data segment [9].ICA has also been utilized as a valuable tool for extracting artificial signals, and its implementation is expected to enhance the performance of the ongoing source localization [8].
This study compared brain activity on scalp and brain cortical level in subject with two types of audio stimuli, stereo audio and spatial audio, through a pair of headsets attached directly to both ears.Both types of audio originate from the same sound.The stereo audio stimulus is sound sent simultaneously on both headsets, while the spatial audio stimulus is sound that moves from the left to the right headset.The study's findings have the potential to provide insights into research focused on spatial hearing and immersive audio production.

System framework
The study involved recording a raw EEG signal from a 22-year-old male participant with no hearing loss issues.The EEG was captured using 21 scalp electrodes placed based on the 10-20 placement system.During the recording, the participant listened to two different types of audio stimuli (spatial and stereo) delivered via a headset while keeping their eyes closed to minimize the impact of visual stimuli.Each stimulus lasted for 3 seconds.The raw data was initially centered and averaged.A bandpass filter (BPF) was then applied to eliminate unwanted signals and retain only the frequency bands relevant to hearing activity (i.e., Theta, Alpha, and Beta frequency ranges of 4-30 Hz).The noise-free EEG signal was decomposed into 21 independent components (ICs) using the ICA algorithm.All these procedures were carried out using the EEGLab toolbox on Matlab (Mathworks Inc.).
Meanwhile, feature extraction was performed to see if there was a change in the components of the EEG signal between the hearing process with the stereo audio stimulus and the spatial audio stimulus.This aims to provide an overview of the brain's response at the scalp level.The Mean Power Spectral Density (PSD) of each Theta, Alpha, and Beta signal was calculated and compared for both hearing processes.
Next, scalp topography maps were plotted to visualize the estimated power of the filtered signal across the 21 ICs.These ICs were then localized using LORETA, a method that converts scalp electrical potential information into current density distribution.This enabled us to estimate the active brain regions during the hearing process.Fig. 1 displays the system framework utilized in this study.

EEG recording and preprocessing
In this study, a multi-channel clinical EEG device from Nihon Kohden was employed to collect reliable EEG data, despite the time-consuming and movement-constraining nature of conventional EEG measurements.The placement of electrodes in the device adhered to the international 10-20 system.The EEG data was captured using 21 active electrodes (placed on Fp1, Fp2, F7, F3, Fz, F4, F8, T7, C3, Cz, C4, T8, P7, P3, Pz, P4, P8, O1, O2) and two reference electrodes (placed on A1 and A2), resulting in 21 channels.The electrode layout on the scalp is illustrated in Fig. 2.This configuration provided comprehensive coverage of the brain area and adequate data for processing.The EEG sampling rate was set at 500 Hz.
Table 1 illustrates the various frequency bands typically observed in EEG signals, including Delta, Theta, Alpha, Beta, and Gamma.Delta waves are indicative of a deep sleep state, whereas gamma waves are associated with clinical illnesses.However, as the focus of this study does not pertain to such states, only the Theta, Alpha, and Beta frequency bands are relevant.
To obtain Theta, Alpha, and Beta waves from the raw EEG signal, a bandpass filter with a passband frequency of 4-30 Hz is directly applied.This involves using a zero phase Finite Impulse Response (FIR) filter in EEGLab, which utilizes the filtfilt() function in MATLAB.The process includes defining the filter's parameters and then executing it.The critical parameters, hi_cut and low_cut, specify a bandpass filter with a low-frequency (highpass) cutoff of 4 Hz and a high-frequency (lowpass) cutoff of 30 Hz.By doing so, the filter automatically removes extremely low-frequency components caused by movement and breathing (less than 0.4 Hz), noise (more than 60 Hz), and power line frequency (50 Hz).

Independent Component Analysis (ICA) decomposition
The Independent Component Analysis (ICA) is a spatial filtering technique that blindly separates the data matrix using the criterion that the resulting source time courses are maximally independent.It generates an 'unmixing' matrix  that, when multiplied by the original data , produces a matrix  consisting of the independent component (IC) time courses.
Suppose the raw digital EEG signal was recorded from electrodes number (  ) denoted by    () and the filtered EEG signal from all electrodes channels is denoted by where  and  are matrices with the size of   × , and  is   ×   .By applying simple matrix algebra, Equation (1) suggests that, the 'mixing' matrix  −1 , has a size equivalent to the number of independent components.The columns of  −1 contain the relative weights that each component uses to estimate the contribution of each scalp channel, which produces the IC topography plot (scalp map).
The -th IC,   , is formed by multiplying the -th column of  with the -th row of , which is a portion of the original data  [10].
the back-projected ICs   are summed to obtain: The  −1 mixing matrix has columns that represent the estimation weight relative to each electrode for a single component source [10].By mapping these weights to the corresponding electrodes on a color-coded head model, it becomes possible to visualize the scalp projection of each source.

Brain source localization using LORETA
LORETA is a method used to solve the EEG inverse problem, which allows the identification of the relative activity of different brain areas using electrodes on the scalp [8].It achieves this by converting the electrical potential differences on the scalp into current density distributions at a deeper cortical level.As a result, LORETA is a useful tool for investigating the functional connectivity of the brain.
Using the forward head model, which links the brain signal from a specific independent component of the matrix  (denoted as ), the inverse problem is solved with a predefined set of constraints to estimate the current source distribution () [7].The forward problem can be represented as: where  is the transfer matrix of the head volume conductor model.The inverse problem in general can be expressed as, min    (6) with, where  ∈    ×1 represents the vector of electric potential differences measured at   electrodes with respect to a single common reference electrode;  ∈    ×(3  ) is the lead field matrix corresponding to   voxels;  ∈  3  ×1 is the current density;  > 0 is the Tikhonov regularization parameter; and  is the positive definite weight matrix [7].Each voxel is a three-dimensional unit of measurement that represents a volume element of the brain.It is a small cube-shaped space that corresponds to the electrical activity of a group of neurons at a specific location in the brain.It is assumed that the lead field and the EEG measurements are transformed to an average reference.The solution of current density in Equation ( 7) is, where   is the pseudoinverse, given by, the matrix  represents the average reference operator, while the matrix  indicates the squared spatial discrete Laplacian operator.The original LORETA method was published in 1994, and since then, it has undergone significant advancements.One of these advancements is the exact LORETA (eLORETA) technique, which solves the inverse problem with zero localization error.eLORETA introduced a weight matrix that sufficiently takes the deeper sources into account at the cortical level [7].The eLORETA solution is computed by Equation (8), where the weight matrix  is expressed as follows: the matrix   denoted as   ∈  3×3 is the -th diagonal subblock of  and   is matrix column .

Mean Power Spectral Density (PSD) of EEG data
Decomposition of the EEG signal into functionally distinct frequency bands is one of the most widely used methods for EEG data analysis [3].We used simple BPF to decompose EEG data into Theta, Alpha, and Beta frequency band and used Fast Fourier Transform (FFT) to calculate their mean Power Spectral Density (PSD).Table 2 compares the mean PSD of these three frequency bands for each hearing condition.
From Table 2, it can be seen that there was an increase in the mean PSD in all frequency bands when the subject was given a spatial audio stimulus compared to a stereo audio stimulus.The most considerable increase occurred in the Beta wave frequency band of 93.8008 µV 2 /Hz.Beta wave on the EEG signal is associated with a focused condition.A significant increase in this frequency band indicates an increase in the focus level of the subject when hearing a spatial audio stimulus compared to when hearing a stereo audio stimulus.

Brain source localization using LORETA
ICA was utilized to denoise EEG data, and the process was iterated 100 times to obtain stable Independent Components (ICs).In this study, the ICA decomposition was performed with the same number of ICs as the number of input electrode channels.Therefore, the 21 EEG electrode data were decomposed into 21 independent components.However, it should be noted that the number of independent sources in the brain is generally unlimited.
The ICs were ranked and color-coded based on the total EEG power, resulting in a scalp map or topography plot.The colors red and blue represented an increase and decrease in power, respectively.As shown in Fig. 3 and Fig. 4, the 21 ICs scalp maps for hearing activity with stereo audio stimulus and spatial audio stimulus were presented.
Scalp map or topography plot shows the power of each IC on the scalp surface.Because each EEG electrode does not represent the recorded signal immediately below the electrode but rather a superposition of various brain waves in all head areas, the scalp map estimates the brain signal strength of each decomposed IC.Through the scalp map, we also make it possible to remove bad components or components that do not make sense.In Fig. 3, we removed IC 4; in Fig. 4, we removed IC 3.This was done because there can't be any activity in most areas of the brain.The IC residue was used to reconstruct current density distribution information at the scalp level using the EEG source localization approach with LORETA.
Figure 5 and Fig. 6 depict the localization results of 21 ICs filtered EEG of hearing activity with stereo audio stimulus and spatial audio stimulus, respectively.From the two pictures, we can see the difference in the dominant area of the brain for the two types of stimuli.In the first experiment with a stereo audio stimulus, the dominantly active brain area was around the temporal lobe, from the right and then shifted to the left.In neuroscience, the temporal lobe is the brain region associated with human auditory processes [11].Meanwhile, apart from visible activity in the temporal lobe, there was activation around the frontal area for the second experiment with spatial audio stimulus.To quantify the source localization results, the current density distribution was calculated.To represent areas of the brain that showed dominant activity based on these results, several Regions of Interest (ROIs) based on the division of Brodmann areas (BAs) were chosen, as depicted in Fig. 7.The cerebral cortex is involved in various cognitive and behavioral functions [12], and Brodmann areas map the cortex according to its different functions.These areas are numbered sequentially, with a total of 52 areas in the cortex.The different areas are characterized by their microscopic anatomy, including the shapes, types of cells, and their connections.While the precise location of these areas remains a subject of debate, they are still widely used to represent specific brain regions.BA-41 and BA-42 are the primary and secondary auditory cortex, respectively.These areas are the first relay stations of auditory information in the cortex.Then, BA-6 is the premotor cortex, which directs the body and prepares the postural muscles for future movements.It receives crucial for spatial orientation (from the posterior parietal cortex) [13].Lastly, BA-9 is known as the dorsolateral prefrontal cortex, a frontal lobe region generally associated with executive functions, including working memory and selective attention.
After we calculated the average current density of each ROI and each stimulus, we performed a paired T-Test to see the significant difference between the two auditory processes.These results are shown in Table 3.  From Table 3, it can be seen that the difference in BA-6 and BA-9 (left and right) between the stereo audio stimulus and the spatial audio stimulus was significant at a significance level of 0.05 (p < 0.05).Meanwhile, there was no significant difference in BA-41 and BA-42 (p > 0.05).This shows that even though there is an increase in activity, there is no significant change in the temporal area of the brain associated with the auditory process when the subject hears the stereo audio stimulus or the spatial audio stimulus.Meanwhile, when the subject was hearing the spatial audio stimulus, there was a significant increase in activity in the frontal areas of the brain, especially areas associated with selective attention and spatial orientation.From these results, this method is practical and can provide insight into brain activity studies related to the spatial hearing process.

Conclusion
This study compared brain activity on the scalp and brain cortical level in a subject with two types of audio stimuli, stereo audio and spatial audio, through a pair of headsets attached directly to both ears.From the results of EEG feature extraction to analyze brain activity on the scalp, there was an increase in the mean PSD in Theta, Alpha, and Beta waves when the subject was hearing spatial audio stimulus.The most considerable increase occurred in the Beta wave frequency band of 93.8008 µV2/Hz.This indicates an increase in the focus level of the subject when hearing a spatial audio stimulus compared to when hearing a stereo audio stimulus.
From the EEG source localization results using ICA-LORETA, there was a difference in the brain's dominant area for the two stimulus types.In the first experiment with a stereo audio stimulus, the dominantly active brain area was around the temporal lobe, which is the area responsible for hearing activity.Meanwhile, apart from visible activity in the temporal lobe, there was activation around the frontal area for the second experiment with the spatial audio stimulus.
To quantify this process, the mean current density at BA-6, BA-9, BA-41, and BA-42 was calculated and paired T-Tests were used to see the significant difference between both audio stimulus types.The results showed there was no significant difference (p > 0.05) in Brodmann area (BA) 41 (Primary Auditory Cortex) and BA 42 (Secondary Auditory Cortex).Meanwhile, for the spatial audio hearing process, there was a significant difference (p < 0.05) in BA 6 (Premotor Cortex), which is related to spatial orientation, and BA 9 (Dorsolateral Prefrontal Cortex), which is associated with executive functions, including working memory and selective attention.

Fig. 2 .
Fig. 2. The layout of electrodes on scalp

Fig. 4 .
Fig. 4. The 21 ICs scalp map of filtered EEG of hearing activity with spatial audio stimulus.

Fig. 5 .
Fig. 5. Localization results of 21 ICs filtered EEG of hearing activity with stereo audio stimulus.

Fig. 6 .
Fig. 6.Localization results of 21 ICs filtered EEG of hearing activity with spatial audio stimulus.

Table 1 .
Types of EEG Signal

Table 2 .
Mean PSD of EEG data.The 21 ICs scalp map of filtered EEG of hearing activity with stereo audio stimulus.

TABLE 3 .
Mean current density of ROI and T-Test results.