oa Wigest: A Ubiquitous Wifi-based Gesture Recognition System

Motivated by freeing the user from specialized devices and leveraging natural and contextually relevant human movements, gesture recognition systems are becoming popular as a fundamental approach for providing HCI alternatives. Indeed, there is a rising trend in the adoption of gesture recognition systems into various consumer electronics and mobile devices. These systems, along with research enhancing them by exploiting the wide range of sensors available on such devices, generally adopt various techniques for recognizing gestures including computer vision, inertial sensors, ultra-sonic, and infrared. While promising, these techniques experience various limitations such as being tailored for specific applications, sensitivity to lighting, high installation and instrumentation overhead, requiring holding the mobile device, and/or requiring additional sensors to be worn or installed. We present WiGest, a ubiquitous WiFi-based hand gesture recognition system for controlling applications running on off-the-shelf WiFi-equipped devices. WiGest does not require additional sensors, is resilient to changes within the environment, and can operate in non-line-of-sight scenarios. The basic idea is to leverage the effect of the in-air hand motion on the wireless signal strength received by the device from an access point to recognize the performed gesture. As shown in Figure 1, WiGest parses combinations of signal primitives along with other parameters, such as the speed and magnitude of each primitive, to detect various gestures, which can then be mapped to distinguishable application actions. There are several challenges we address WiGest including handling noisy RSSI values due to multipath interference and other electromagnetic noise in the wireless medium; handling gesture variations and their attributes for different humans or the same human at different times; handling interference due to the motion of other people within proximity of the user's device; and finally being energy-efficient to suit mobile devices. To address these challenges, WiGest leverages different signal processing techniques that can preserve signal details while filtering out the noise and variations in the signal. We implement WiGest on off-the-shelf laptops and evaluate frequencies on the RSSI, creating a signal composed of three primitives: rising edge, falling edge, and pause. We evaluate its performance with different users in an apartment and engineering building settings. Various realistic scenarios are tested covering more than 1000 primitive actions and gestures each in the presence of interfering users in the same room as well as other people moving in the same floor during their daily life. Our results show that WiGest can detect the basic primitives with an accuracy of 90% using a single AP for distances reaching 26 ft including through-the-wall non-line-of-sight scenarios. This increases to 96% using three overheard APs, a typical case for many WiFi deployments. When evaluating the system using a multimedia player application case study, we achieve a classification accuracy of 96%.