3brain-computer interface
Development of Real-time Wireless Brain Computer Interface for Drowsiness Detection
1Shao-Hang Hung, 1Che-Jui Chang,
1Chih-Feng Chao, 2I-Jan Wang
1Institute of Electrical Control Engineering National Chiao Tung University
Hsinchu, Taiwan
siegfi.iic93g@https://www.wendangku.net/doc/e68670862.html,.tw 1,2 Chin-Teng Lin, Fellow, IEEE, 3Bor-shyh Lin, 2Department of Computer Science
3Institute of Imaging and Biomedical Photonics National Chiao Tung University
Hsinchu, Taiwan
borshyhlin@https://www.wendangku.net/doc/e68670862.html,.tw
Abstract—In this study, a real-time wireless embedded EEG-based brain computer interface (BCI) system was developed for drowsiness detection in a realistic driving task. Accidents caused by driver’s drowsiness behind the steering wheel have a high fatality rate because of the marked decline in the driver’s abilities of perception, recognition, and vehicle control abilities while sleepy. Therefore, real-time drowsiness monitoring is important to avoid traffic accidents. In this study, an embedded EEG-based BCI system which includes a wireless physiological signal acquisition module and an embedded signal processing module was designed, and a real-time drowsiness detection algorithm based on our unsupervised approach was implemented in the embedded signal processing module. EEG signal would be monitored and analyzed by the embedded signal processing module, and the warning tone would be triggered to prevent traffic accidents when the drowsiness condition occurred.
I.I NTRODUCTION
Drivers’ drowsiness has been implicated as a causal factor in many accidents because of the marked decline in drivers’ perception of risk and recognition of danger, and diminished vehicle handling abilities. Therefore, real-time drowsiness monitoring is important to avoid traffic accidents.
Previous studies have proposed a number of methods for detect drowsiness. They can be categorized into two main approaches. The first approach focuses on physical changes during fatigue, such as the inclination of the driver’s head, sagging posture, and decline in gripping force on steering wheel [1], [2]. However, these parameters easily vary in different vehicle types and driving conditions. The second approach focuses on measuring physiological changes of drivers, such as eye activity measures, heart beat rate, skin electric potential, and electroencephalographic (EEG) activities [3]-[9]. These studies indicated that EEG-based method can use a shorter moving-averaged window to track second-to-second fluctuations in the subject performance in a visual compensatory task.
Most of previous studies for EEG-based drowsiness detection are supervised in nature and built up the same detection model for all subjects. However, it is well-known that the individual variability in EEG dynamics relating to drowsiness from alertness is large. In order to alleviate the influence of cross-session variability in EEG dynamics, we proposed an unsupervised subject- and session-independent approach for detection departure from alertness in our previous study [10]. Under the assumption of that EEG power spectrum in an alert state can be reasonably modeled using a multivariate normal distribution, a statistical model of subject’s alert state was generated in every session by using a very limited data obtained at the beginning of the session. The model was validated statistically and then used to assess cognitive state for different subjects effectively.
In this study, we developed a real-time wireless embedded EEG-based brain computer interface (BCI) system for drowsiness detection in a realistic driving task. It includes a wireless physiological signal acquisition module and an embedded signal processing module. EEG signal was collected by the wireless physiological signal acquisition module, and then transmitted to the embedded signal processing module wirelessly via Bluetooth. Our portable and low-power-consumption physiological signal acquisition module with wireless transmission provides the advantages of mobility and long-term measurement. Moreover, a real-time drowsiness detection algorithm based on our unsupervised approach [10] was implemented in the embedded signal processing module. EEG signal would be monitored and analyzed by the embedded signal processing module, and the warning tone would be triggered to prevent traffic accidents when the drowsiness condition occurred.
II.SYSTEM ARCHITECTURE
The basic scheme of the proposed EEG-based BCI system is shown in Fig. 1. The hardware of this system consists mainly of two major parts: a wireless physiological signal acquisition module and an embedded signal processing
module. First, EEG signal was obtained by EEG electrode, and then was amplified and filtered by EEG amplifier and acquisition unit in the physiological acquisition module. Next, EEG signal was pre-processed by microprocessor unit and transmitted to the embedded signal processing module wirelessly via wireless transmission unit. After receiving EEG signal, it would be monitored and analyzed by our drowsiness detection algorithm implemented in embedded signal processing unit. If the drowsiness condition was detected, warning tone device unit would be triggered to alarm the driver.
Embedded signal EEG Electrode
Wireless physiological signal acquisition module Embedded signal processing module
Wireless
Figure 1. Basic schematic of the proposed EEG-based BCI system
A. Wireless Physiological Signal Acquisition Module The wireless physiological signal acquisition module mainly consists of EEG amplifier and acquisition unit, microprocessor unit and wireless transmission unit. Here, EEG amplifier and acquisition unit, which includes a pre-amplifier, a band-pass filter and a 12-bits analog-to-digital converter (ADC) with sampling rate of 512 Hz, was designed to amplify and filter EEG signal. The gain of EEG amplifier and acquisition unit was set to about 5040 times with the frequency band of 0.1 - 100 Hz. Microprocessor unit was used to control the ADC to obtain, pre-process and send EEG data to wireless transmission unit. In this study, Bluetooth module BM0203, which is fully compliant with the Bluetooth v2.0 + EDR specification, was used in wireless transmission unit. The size of the wireless physiological signal acquisition module is 4 cm × 2.5 cm × 0.6 cm, and can be embedded into a headband as a wearable device, as shown in Fig. 2. This module was designed to operate at 31 mA with 3.7-V DC power supply, and can continuously operate over 33 hours with a commercial 11000 mA Li-ion battery.
Figure 2. Our proposed wireless Physiological Signal Acquisition Module
B. Embedded Signal Processing Module
The embedded signal processing module which owns powerful computations and supports various peripheral interfaces was designed as a platform which perform real-time EEG-based drowsiness detection algorithm. The embedded module mainly consists of embedded signal processing unit, wireless transmission unit, and warning tone device unit. After receiving EEG data from the wireless transmission unit, the embedded signal processing unit would real-time process, analyze and display EEG data. If the drowsiness condition was detected, the warning tone device unit would be triggered to alarm the driver.
The block diagram of embedded signal processing unit was shown in Fig. 3. Blackfin embedded processor ADSP-BF533 was used in the embedded signal processing unit. The central-processing-unit (CPU) speed of ADSP-BF533 can be up to 600M Hz. ADSP-BF533 owns two 16-bit multiply-and-accumulate (MAC) to execute 1200 lines addition and multiplication functions, and contains four independent direct-memory-access (DMA) to effectively reduce the processing time of core. A memory-mapped TFT-LCD was used in this module, and shared the same memory bus with SDRAM. In order to reduce the module size, the parallel NOR flash was replaced by SPI flash. Furthermore, this module also owns power management and charging circuits. ADSP-BF533 used UART interface to communicate with wireless transmission unit. The warning tone device unit was designed in an expanded SD card, and can be combined with the embedded signal processing unit via SD/MMC socket. Therefore, SD/MMC socket in this module also provides good interface scalability. The size of the embedded signal processing module is 6.4 cm × 4.4 cm × 1 cm, as shown in Fig. 4. This module was also designed to operate with 3.7-V DC power supply.
In this embedded signal processing module, the modified Universal Boot Loader (U-Boot) was used to perform the initial system configuration and boot the Micro Control Linux (uClinux) kernel. And the drowsiness detection algorithm was implemented as a multi-threaded application on uClinux.
Figure 1. Block diagram of embedded signal processing unit
Figure 2. Our proposed Embedded Signal Processing Module
C. Real-time Drowsiness Detection Algorithm Previous studies mentioned that EEG spectra in theta rhythm (4~7Hz) and alpha rhythm (8~11Hz) usually reflect the changes the cognitive state and memory performance. These findings motivated us to derive the drivers’ alert models and detect their cognitive state from EEG spectra in theta and alpha rhythms. In our previous study [10], we proposed an EEG-based unsupervised approach to detect drowsiness. This approach does not need a labeled training dataset with information on whether the driver is in an alert state or drowsy state at every time instant, and can account for baseline shifts and the variations in EEG spectra due to changes in recording conditions in different driving sessions.
Under the assumption of that the driver should be in an alert state during the first few minutes of driving, the mode of the driver’s alert state can be derived by the first few minutes of EEG recording. In order to build the alert mode, the specific window was selected by Mardia test [11]. If the driver was under alert state, his EEG spectra in Theta and Alpha rhythm would follow a multivariate normal distribution, which can be characterized the alert models. Next, the deviation of the driver’s current state would be assessed continuously from the alert model by using Mahalanobis distance (MD). If the driver remained alert, his EEG spectra in Theta and Alpha rhythm should match the alert model. Otherwise, if the driver became drowsy, then his EEG spectra would deviate from the respective model and hence MD would increase. The flowchart of real-time drowsiness detection algorithm based on the unsupervised approach was shown in Fig. 5.
Extract EEG spectra Extract EEG spectra Build alert model Build alert model Calculate
Calculate Calculate Threshold for Calculate deviation Preprocessing
in Theta rhythm
in Alpha rhythm
of Alpha rhythm
of Theta rhythm
MDA
MDT MDC
drowsiness
Build alert model
from alert model Raw EEG
Output
Figure 5. Flowchart of real-time drowsiness detection algorithm
III. R ESULTS
In this study, a lane-keeping driving experiment was utilized to investigate driving performance under different levels of drowsiness. Here, a virtual reality (VR)-based cruising environment was developed to simulate a car driving at 100 km/hr on a straight four-lane highway at night, as shown in Fig. 6. The driver is asked to maintain the car along the center of the cruising lane. When the subject is alert, his response time to the random drift is short and the deviation of the car from the center of the lane is small. When the subject is drowsy, both the response time and the car’s deviation are high. Therefore, the car’s deviation from the central line was used as a measure of the subject's drowsiness state.
Figure 6. Virtual reality-based cruising environment in lane-keeping driving experiment
Fig. 7 showed the relationship between driving performance (drowsiness state) and Mahalanobis distance for Alpha rhythm (MDA) and for Theta rhythm (MDT). Obviously, it showed that the driver’s drowsiness state was highly correlated to MDA and MDT. Next, the correlation between drowsiness state and MDA, MDT and MDC was investigated. The result was showed in Table I. Here, five subjects were examined. We found that MDC (0.9*MDA+ 0.1*MDT) has higher correlation with drowsiness state. In this investigation, we have demonstrated the feasibility of an unsupervised subject and session independent approach to detect departure from alertness in driver. In future, we plan to identify thresholds on MDA/MDT/MDC which can be used to label the driver’s cognitive state as alert/mild drowsy/deep drowsy. This will require some validation data as well as authentication by experts.
Figure 7. Relationship between driving performance (drowsiness state) and Mahalanobis distance for Alpha rhythm (MDA) and for Theta rhythm (MDT).
TABLE I. THE COMPARISON OF THE CORRELATION BETWEEN DRIVING PERFORMANCE AND MD*AND DRIVING PERFORMANCE FOR CHANNEL OZ
IV.D ISCUSSIONS AND C ONCLUSIONS
In this study, a real-time wireless brain computer interface for drowsiness detection was proposed. The modular approach applied in hardware and software design enables this system to be configurable for different application scenarios. For example, in the future, the EEG acquisition module can be used to connect several optional physiological sensors in addition to the built-in one, and it doesn’t affect the whole system architecture. This system is feasible for further extension. Moreover, our EEG acquisition module is small, light, and wearable, therefore, it is suitable for long-term EEG monitoring in users’ daily life. A novel algorithm for drowsiness detection was also proposed in this study. It can effectively reduce computation complexity, and is suitable to be implemented in the embedded module. This algorithm was validated statistically and then used to assess cognitive state for different subjects effectively. EEG signal would be monitored and analyzed by the embedded signal processing module, and the warning tone would be triggered to prevent traffic accidents when the drowsiness condition occurred.
V.R EFERENCES
[1]T. Pilutti and G. Ulsoy, “Identification of driver state for lane-keeping
tasks,” Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, vol. 29, pp. 486–502, 1999.
[2]J. Qiang, Z. Zhiwei, and P. Lan, “Real-time nonintrusive monitoring
and prediction of driver fatigue,” IEEE Transactions on Vehicular Technology, vol. 53, pp. 1052–068, 2004.
[3]J. A. Stern, D. Boyer, and D. Schroeder, “Blink rate: a possible
measure of fatigue,” Human Factors, vol. 36, no. 2, pp. 285–297, 1994.
[4]K. Van Orden, W. Limbert, S. Makeig, and T.-P. Jung, “Eye activity
correlates of workload during a visual spatial memory task,” Human Factors, vol. 43, no. 1, pp. 111–121, 2001.
[5]H. J. Eoh, M. K. Chung, and S. H. Kim, “Electroencephalographic
study of drowsiness in simulated driving with sleep deprivation,”
International Journal of Industrial Ergonomics, Vol. 35, pp. 307-320, 2005.
[6]S. Makeig and T.-P. Jung, “Changes in alertness are a principal
component of variance in the EEG spectrum,” Neuroreport, vol. 7, no.
1, pp. 213–216, 1995.
[7]T.-P. Jung, S. Makeig, M. Stensmo, and T. J. Sejnowski, “Estimating
alertness from the EEG power spectrum,” IEEE Transactions on Biomedical Engineering, vol. 44, no. 1, pp. 60–69, 1997.
[8] C. T. Lin, R. C. Wu, S. F. Liang, W. H. Chao, Y. J. Chen, and T. P.
Jung, “EEG-based drowsiness estimation for safety driving using independent component analysis,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol.52, no.12, pp. 2726-2738, 2005.
[9]H. Su, and G. Zheng, “A partial least squares regression-based fusion
model for predicting the trend in drowsiness,” IEEE Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, vol.38, no.5, pp.1085-1092, 2008.
[10]N. R. Pal, C. Y. Chuang, L. W. Ko, C. F. Chao, T.P. Jung, S. F. Liang,
and C. T. Lin, “EEG-Based Subject- and Session-independent Drowsiness Detection: An Unsupervised Approach,” EURASIP Journal on Advances in Signal Processing, vol. 2008, Article ID 519480, 11 pages, 2008.
[11]K. V. Mardia, “Mardia’s test of multinormality,” in Encyclopedia of
Statistical Sciences, S. Kotz and N. L. Johnson, Eds., vol.5, pp. 217–221, JohnWiley & Sons, New York, NY, USA, 1985