NeTS: Medium: Collaborative Research: Exploiting Fine-grained WiFi Signals for Wellbeing Monitoring

Sponsored by the U.S. National Science Foundation
Awards: CNS-1826647, CNS-1514238 and CNS-1514224, Duration: 09/01/2015-08/31/2019

   Related image       Image result for fsu images               Image result for stevens institute of technology

https://www.nsf.gov/images/logos/NSF_4-Color_bitmap_Logo.png


Welcome to the website of our research project: "NeTS: Medium: Collaborative Research: Exploiting Fine-grained WiFi Signals for Wellbeing Monitoring". This project is a collaborative effort among three institutions: Rutgers University, Stevens Institute of Technology, and Florida State University. This website is created and maintained to disseminate and share research results and other information related to the project.

Project Description

While proliferating WiFi networks are usually used for wireless Internet connections, they have great potential to capture environment changes and identify human motions of various scales. Examples of such motions range from performing daily activities to breathing and heartbeat during sleep. These various scales of motions can be captured by fine-grained WiFi signals to perform continuous wellbeing monitoring. Wellbeing monitoring leveraging existing WiFi infrastructure is particularly attractive as it requires neither wearing body instrumentation nor active monitoring by the user. Such an approach would facilitate a broad range of healthcare related applications at home environments without frequent hospital visits, such as real-time prediction and prevention of certain health problems (e.g., cardiovascular diseases and sleep apnea). Using existing WiFi infrastructure for wellbeing monitoring not only advances and extends the applications that could be supported by WiFi networks but also enables easy and large-scale deployment in non-clinical settings due to the proliferation of WiFi networks. Additionally, the educational efforts include curriculum development, outreaching to high school students, and engaging both undergraduate and graduate students in research.


This project focuses on building a WiFi enabled continuous wellbeing monitoring framework for fine-grained sleep monitoring and vital signs tracking at home environments. Users do not need to wear any sensors or actively participate in the monitoring process. The proposed framework targets to advance techniques in device-free fine-grained sleep events identification and vital signs tracking during sleep by utilizing existing WiFi signals. The proposed framework develops device-free localization strategies, vital signs tracking methods and statistical learning techniques to depict a comprehensive picture of users' wellbeing. Such wellbeing information is further utilized to assist in real-time disease prediction by leveraging today's ever-growing mobile environments. A hierarchical multivariate logistic regression model is developed to effectively mine through health conditions and identify risk factors of certain diseases. Chances of developing certain health problems, such as cardiovascular diseases, is promptly predicted. The project also provides user-centric access control of archived wellbeing monitoring information to ensure data privacy and coping with distrusted servers.


Personnel

Principal Investigators

http://personal.stevens.edu/~ychen6/fig_fixed_email_large.jpg

Dr. Yingying Chen (Lead PI)
Professor/Adjunct Professor
Department of Electrical and Computer Engineering
Rutgers University/Stevens Institute of Technology
Email: yingche@scarletmail.rutgers.edu
Homepage: http://www.winlab.rutgers.edu/~yychen/

http://www.cs.fsu.edu/~jieyang/Jie%20Yang.jpg

Dr. Jie Yang (PI)
Associate Professor
Department of Computer Sciences
Florida State University
Email: jyang5@fsu.edu
Homepage: http://www.cs.fsu.edu/~jieyang/

 

http://rwjms.rutgers.edu/cvinj/images/0clip_image002_000.jpg

Dr. Jerry Q. Cheng (PI)
Assistant Professor
Rutgers Robert Wood Johnson Medical School
Rutgers University
Email: jcheng1@rwjms.rutgers.edu
 

 

Graduate Students

http://www.cs.fsu.edu/~jieyang/voice_authenticaiton_research_files/download.jpg

Jian Liu
Ph.D. student
Department of Electrical and Computer Engineering
Rutgers University
Email: jianliu@winlab.rutgers.edu
Homepage: http://www.winlab.rutgers.edu/~jianliu/

http://www.cs.fsu.edu/~jieyang/voice_authenticaiton_research_files/download.jpg 


Chen Wang
Ph.D. student
Department of Electrical and Computer Engineering
Rutgers University
Email: chenwang@winlab.rutgers.edu
 

http://www.cs.fsu.edu/~jieyang/voice_authenticaiton_research_files/download.jpg 

Cong Shi
Ph.D. student
Department of Electrical and Computer Engineering
Stevens Institute of Technology
Email: cshi5@stevens.edu
 

http://www.cs.fsu.edu/~jieyang/voice_authenticaiton_research_files/download.jpg 

Sheng Tan
Ph.D. student
Department of Computer Sciences
Florida State University
Email: tan@cs.fsu.edu
Homepage: http://ww2.cs.fsu.edu/~tan/

http://www.cs.fsu.edu/~jieyang/voice_authenticaiton_research_files/download.jpg 


Linghan Zhang
Ph.D. student
Department of Computer Sciences
Florida State University
Email: lzhang@cs.fsu.edu
 


Publications

  1. Non-invasive Fine-grained Sleep Monitoring Leveraging Smartphones  
    Yanzhi Ren, Chen Wang, Yingying Chen, Jie Yang, Hongwei Li
    IEEE Internet of Things Journal (IEEE IoT), 2019.

    Summary: Sleep monitoring has drawn increasing attention as the sleep quality is important to maintain a person’s well-being. For instance, serious health problems such as cardiovascular disease, fatigue or depression are usually associated with inadequate and irregular sleep. Traditional sleep monitoring systems involve wearable sensors with professional installation, and thus are usually limited to clinical usage. Recent work for sleep monitoring can detect several sleep events such as coughing and snoring using smartphone sensors. However, such coarse-grained sleep monitoring is unable to detect the breathing rate which is an important health indicator. In this work, we present a fine-grained sleep monitoring system to detect the breathing rate and sleep events simultaneously by leveraging smartphones. Our system exploits the readily available smartphone earphone placed close to the user to reliably capture the human breathing sound. Given the captured acoustic sound, noise reduction is performed to remove the environmental noise and the breathing rate is then identified based on the signal envelope detection. Our system can further detect some sleep events including snoring, coughing, turning over and getting up based on the features extracted from the acoustic sound. Moreover, we develop a body movement-assisted sleep event detection method to provide higher detection accuracy by further exploiting the user’s body movement patterns captured by the accelerometer embedded on smartphones. Our extensive experiments involving nine subjects over six months’ time period confirm the effectiveness of our proposed system on breathing rate monitoring and sleep events detection under various environments. By combining breathing rate and sleep events, our system can provide noninvasive and continuous fine-grained sleep monitoring for healthcare related applications, such as sleep apnea monitoring as evidenced by our experimental study.

 

  1. Device-free Personalized Fitness Assistant Using WiFi
    Guo, Xiaonan; Liu, Jian; Shi, Cong; Liu, Hongbo; Chen, Yingying; Chuah, Mooi Choo
    PACM on Interactive, Mobile, Wearable, and Ubiquitous Computing, v.2, 2019.

    Summary: There is a growing trend for people to perform regular workouts in home/office environments because work-at-home people or office workers can barely squeeze in time to go to dedicated exercise places (e.g., gym). To provide personalized fitness assistance in home/office environments, traditional solutions, e.g., hiring personal coaches incur extra cost and are not always available, while new trends requiring wearing smart devices around the clock are cumbersome. In order to overcome these limitations, we develop a device-free fitness assistant system in home/office environments using existing WiFi infrastructure. Our system aims to provide personalized fitness assistance by differentiating individuals, automatically recording fine-grained workout statistics, and assessing workout dynamics. In particular, our system performs individual identification via deep learning techniques on top of workout interpretation. It further assesses the workout by analyzing both short and long-term workout quality, and provides workout reviews for users to improve their daily exercises. Additionally, our system adopts a spectrogram-based workout detection algorithm along with a Cumulative Short Time Energy (CSTE)-based workout segmentation method to ensure its robustness. Extensive experiments involving 20 participants demonstrate that our system can achieve a 93% accuracy on workout recognition and a 97% accuracy for individual identification.

 

  1. Signature Verification Using Critical Segments for Securing Mobile Transactions
    Yanzhi Ren, Chen Wang, Yingying Chen, Mooi Choo Chuah, Jie Yang
    IEEE Transactions on Mobile Computing (IEEE TMC), 2019.

    Summary: User signature verification on mobile devices becomes critical to ensure the success deployment of mobile applications such as mobile healthcare and online transactions. Existing approaches mainly focus on user verification targeting the unlocking of mobile devices or performing continuous verification based on a user's behavioral traits. Few studies provide efficient real-time user signature verification. In this work, we propose a critical segment based online signature verification system to secure mobile healthcare and online transactions on multi-touch mobile devices. Our system identifies and exploits the segments which remain invariant within a user's signature to capture the intrinsic signing behavior embedded in each user's signature. Our system extracts features from a user's signature that describe both the geometric layout of the signature as well as behavioral and physiological characteristics in the user's signing process. Given the input signatures for user enrollment, our system further designs a quality score to identify the problematic signature sets to achieve robust user signature profile construction. Our experimental evaluation of 25 subjects over six months’ time period shows that our system is highly accurate in provide signature verification and robust to signature forging attacks.

 

  1. Monitoring Vital Signs and Postures During Sleep Using WiFi Signals
    Jian Liu, Yingying Chen, Yan Wang, Xu Chen, Jerry Cheng, Jie Yang
    IEEE Internet of Things Journal. 5 (3), 2071-2084, 2018.

    Summary: Tracking human sleeping postures and vital signs of breathing and heart rates during sleep is important as it can help to assess the general physical health of a person and provide useful clues for diagnosing possible diseases. Traditional approaches (e.g., polysomnography) are limited to clinic usage. Recent radio frequency-based approaches require specialized devices or dedicated wireless sensors and are only able to track breathing rate. In this work, we propose to track the vital signs of both breathing rate and heart rate during sleep by using off-the-shelf WiFi without any wearable or dedicated devices. The proposed system reuses existing WiFi network and exploits the fine-grained channel information to capture the minute movements caused by breathing and heart beats. It thus has the potential to be widely deployed and perform continuous long-term monitoring. The developed algorithm makes use of the channel information in both time and frequency domain to estimate breathing and heart rates, and it works well when either individual or two persons are in bed. Extensive experiments demonstrate that the proposed system can accurately capture vital signs during sleep under realistic settings, and achieve comparable or even better performance comparing to traditional and existing approaches, which is a strong indication of providing noninvasive, continuous fine-grained vital signs monitoring without any additional cost.

 

  1. Authenticating Users Through Fine- Grained Channel Information
    Hongbo Liu, Yan Wang, Jian Liu, Jie Yang, Yingying Chen and H. Vincent Poor
    IEEE Transactions on Mobile Computing. 17 (2), 251-264, 2018.

    Summary: User authentication is the critical first step in detecting identity-based attacks and preventing subsequent malicious attacks in wireless and mobile healthcare systems. However, the increasingly dynamic mobile environments make it harder to always apply cryptographic-based methods for user authentication due to their infrastructural and key management overhead. Exploiting non-cryptographic based techniques grounded on physical layer properties to perform user authentication appears promising. In this work, the use of channel state information (CSI), which is available from off-the-shelf WiFi devices, to perform fine-grained user authentication is explored. Particularly, a user-authentication framework that can work with both stationary and mobile users is proposed. When the user is stationary, the proposed framework builds a user profile for user authentication that is resilient to the presence of a spoofer. The proposed machine learning based user-authentication techniques can distinguish between two users even when they possess similar signal fingerprints and detect the existence of a spoofer. When the user is mobile, it is proposed to detect the presence of a spoofer by examining the temporal correlation of CSI measurements. Both office building and apartment environments show that the proposed framework can filter out signal outliers and achieve higher authentication accuracy compared with existing approaches using received signal strength (RSS).

 

  1. Hearing Your Voice is Not Enough: An Articulatory Gesture Based Liveness Detection for Voice Authentication
    Linghan Zhang, Sheng Tan, and Jie Yang
    Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS '17).

    Summary: Voice biometrics is drawing increasing attention as it is a promising alternative to legacy passwords for user authentication in wireless and mobile healthcare systems. Recently, a growing body of work shows that voice biometrics is vulnerable to spoofing through replay attacks, where an adversary tries to spoof voice authentication systems by using a pre-recorded voice sample collected from a genuine user. In this work, we propose VoiceGesture, a liveness detection system for replay attack detection on smartphones. It detects a live user by leveraging both the unique articulatory gesture of the user when speaking a passphrase and the mobile audio hardware advances. Specifically, our system re-uses the smartphone as a Doppler radar, which transmits a high frequency acoustic sound from the built-in speaker and listens to the reflections at the microphone when a user speaks a passphrase. The signal reflections due to user’s articulatory gesture result in Doppler shifts, which are then analyzed for live user detection. VoiceGesture is practical as it requires neither cumbersome operations nor additional hardware but a speaker and a microphone that are commonly available on smartphones. The experimental evaluation with 21 participants and different types of phones shows that it achieves over 99% detection accuracy at around 1% Equal Error Rate (EER). Results also show that it is robust to different phone placements and is able to work with different sampling frequencies.

 

  1. Smart User Authentication through Actuation of Daily Activities Leveraging WiFi-enabled IoT
    Cong Shi, Jian Liu, Hongbo Liu, Yingying Chen
    Proceedings of the 18th ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc 2017).

    Summary: User authentication is a critical process in both corporate and home environments due to the ever-growing security and privacy concerns. With the advancement of smart cities and home environments, the concept of user authentication is evolved with a broader implication by not only preventing unauthorized users from accessing confidential information but also providing the opportunities for customized services corresponding to a specific user. Traditional approaches of user authentication either require specialized device installation or inconvenient wearable sensor attachment. This paper supports the extended concept of user authentication with a device-free approach by leveraging the prevalent WiFi signals made available by IoT devices, such as smart refrigerator, smart TV and thermostat, etc. The proposed system utilizes the WiFi signals to capture unique human physiological and behavioral characteristics inherited from their daily activities, including both walking and stationary ones. Particularly, we extract representative features from channel state information (CSI) measurements of WiFi signals, and develop a deep learning based user authentication scheme to accurately identify each individual user. Extensive experiments in two typical indoor environments, a university office and an apartment, are conducted to demonstrate the effectiveness of the proposed authentication system. In particular, our system can achieve over 94% and 91% authentication accuracy with 11 subjects through walking and stationary activities, respectively.

 

  1. FitCoach: Virtual Fitness Coach Empowered by Wearable Mobile Devices
    Xiaonan Guo, Jian Liu, Yingying Chen
    Proceedings of IEEE International Conference on Computer Communications (IEEE INFOCOM 2017).

    Summary: Acknowledging the powerful sensors on wearables and smartphones enabling various applications to improve users’ life styles and qualities (e.g., sleep monitoring and running rhythm tracking), this paper takes one step forward developing FitCoach, a virtual fitness coach leveraging users’ wearable mobile devices (including wrist-worn wearables and arm-mounted smartphones) to assess dynamic postures (movement patterns & positions) in workouts. FitCoach aims to help the user to achieve effective workout and prevent injury by dynamically depicting the short-term and long-term picture of a user’s workout based on various sensors in wearable mobile devices. In particular, FitCoach recognizes different types of exercises and interprets fine-grained fitness data (i.e., motion strength and speed) to an easy-to-understand exercise review score, which provides a comprehensive workout performance evaluation and recommendation. FitCoach has the ability to align the sensor readings from wearable devices to the human coordinate system, ensuring the accuracy and robustness of the system. Extensive experiments with over 5000 repetitions of 12 types of exercises involve 12 participants doing both anaerobic and aerobic exercises in indoors as well as outdoors. Our results demonstrate that FitCoach can provide meaningful review and recommendations to users by accurately measure their workout performance and achieve 93% accuracy for workout analysis.

 

  1. VoiceLive: A Phoneme Localization based Liveness Detection for Voice Authentication on Smartphones
    Linghan Zhang, Sheng Tan, Jie Yang, Yingying Chen
    Proceedings of the 23rd ACM Conference on Computer and Communications Security (CCS 2016).

    Summary: Voice authentication is drawing increasing attention and becomes an attractive alternative to passwords for user authentication in mobile healthcare systems. Recent advances in mobile technology further accelerate the adoption of voice biometrics in an array of diverse mobile applications. However, recent studies show that voice authentication is vulnerable to replay attacks, where an adversary can spoof a voice authentication system using a pre-recorded voice sample collected from the victim. In this paper, we propose VoiceLive, a practical liveness detection system for voice authentication on smartphones. VoiceLive detects a live user by leveraging the user’s unique vocal system and the stereo recording of smartphones. In particular, with the phone closely placed to a user’s mouth, it captures time-difference-of-arrival (TDoA) changes in a sequence of phoneme sounds to the two microphones of the phone, and uses such unique TDoA dynamic which doesn’t exist under replay attacks for liveness detection. VoiceLive is practical as it doesn’t require additional hardware but two-channel stereo recording that is supported by virtually all smartphones. Our experimental evaluation with 12 participants and different types of phones shows that VoiceLive achieves over 99% detection accuracy at around 1% Equal Error Rate (EER). Results also show that VoiceLive is robust to different phone placements and is compatible to different sampling rates and phone models..

 

  1. Privacy Preserving Disease Treatment & Complication Prediction System (PDTCPS)
    Qinghan Xue, Mooi Choo Chuah, Yingying Chen
    Proceedings of the 11th ACM Symposium on Information, Computer and Communications Security (ASIACCS 2016).

    Summary: Affordable cloud computing technologies allow users to efficiently store, and manage their Personal Health Records (PHRs) and share with their caregivers or physicians. This in turn improves the quality of healthcare services, and lower health care cost. However, serious security and privacy concerns emerge because people upload their personal information and PHRs to the public cloud. Data encryption provides privacy protection of medical information but it is challenging to utilize encrypted data. In this paper, we present a privacy-preserving disease treatment, complication prediction scheme (P DT CP S), which allows authorized users to conduct searches for disease diagnosis, personalized treatments, and prediction of potential complications. P DT CP S uses a tree-based structure to boost search efficiency, a wildcard approach to support fuzzy keyword search, and a Bloom-filter to improve search accuracy and storage efficiency. In addition, our design also allows health care providers and the public cloud to collectively generate aggregated training models for disease diagnosis, personalized treatments and complications prediction. Moreover, our design provides query unlinkability and hides both search & access patterns. Finally, our evaluation results using two UCI datasets show that our scheme is more efficient and accurate than two existing schemes.

 

  1. User Verification Leveraging Gait Recognition For Smartphone Enabled Mobile Healthcare Systems
    Yanzhi Ren, Yingying Chen, Mooi Choo Chuah, and Jie Yang
    IEEE Transactions on Mobile Computing (IEEE TMC), Volume 14, Issue 9, Pages 1961-1974, 2015.

    Summary: The rapid deployment of sensing technology in smartphones and the explosion of their usage in people’s daily lives provide users with the ability to collectively sense the world. This leads to a growing trend of mobile healthcare systems utilizing sensing data collected from smartphones with/without additional external sensors to analyze and understand people’s physical and mental states. However, such healthcare systems are vulnerable to user spoofing, in which an adversary distributes his registered device to other users such that data collected from these users can be claimed as his own to obtain more healthcare benefits and undermine the successful operation of mobile healthcare systems. Existing mitigation approaches either only rely on a secret PIN number (which can not deal with colluded attacks) or require an explicit user action for verification. In this paper, we propose a user verification system leveraging unique gait patterns derived from acceleration readings to detect possible user spoofing in mobile healthcare systems. Our framework exploits the readily available accelerometers embedded within smartphones for user verification. Specifically, our user spoofing mitigation framework (which consists of three components, namely Step Cycle Identification, Step Cycle Interpolation, and Similarity Comparison) is used to extract gait patterns from run-time accelerometer measurements to perform robust user verification under various walking speeds. We show that our framework can be implemented in two ways: user-centric and server-centric, and it is robust to not only random but also mimic attacks. Our extensive experiments using over 3,000 smartphone-based traces with mobile phones placed on different body positions confirm the effectiveness of the proposed framework with users walking at various speeds. This strongly indicates the feasibility of using smartphone based low grade accelerometer to conduct gait recognition and facilitate effective user verification without active user cooperation.

 

  1. Tracking Vital Signs During Sleep Leveraging Off-the-shelf WiFi
    Jian
    Liu, Yan Wang, Yingying Chen, Jie Yang, Xu Chen, Jerry Cheng
    roceedings of the 16th ACM Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc 2015).

    Summary: Tracking human vital signs of breathing and heart rates during sleep is important as it can help to assess the general physical health of a person and provide useful clues for diagnosing possible diseases. Traditional approaches (e.g., Polysomnography (PSG)) are limited to clinic usage. In this work, we propose to track the vital signs of both breathing rate and heart rate during sleep by using off-the-shelf WiFi without any wearable or dedicated devices. Our system re-uses existing WiFi network and exploits the fine-grained channel information to capture the minute movements caused by breathing and heart beats. Our system thus has the potential to be widely deployed and perform continuous long-term monitoring. The developed algorithm makes use of the channel information in both time and frequency domain to estimate breathing and heart rates, and it works well when either individual or two persons are in bed. Our extensive experiments demonstrate that our system can accurately capture vital signs during sleep under realistic settings, and achieve comparable or even better performance comparing to traditional and existing approaches, which is a strong indication of providing non-invasive, continuous fine-grained vital signs monitoring without any additional cost.

 

Disclaimer: The papers here are made available for timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders.


 

 

 

© Jie Yang, 2018