SmokeNav combines an inertial sensor and millimeter-wave radar to enhance situational awareness for first responders in hazardous conditions.
A new system called SmokeNav proposes the combination of an inertial sensor and millimeter-wave radar to improve the effectiveness of environmental awareness for first responders in hazardous situations, keeping these vital workers safe as they risk their lives to protect others.
Between 2014 and 2018, in the U.S. alone, the average number of firefighters killed while on duty was 65 each year, according to the National Fire Protection Association (NFPA). In 2021, this figure rose, with 70 firefighters in the U.S. passing away as the result of line-of-duty deaths. This isn’t a problem exclusive to the U.S., either. In the past four years, a total of 177 firefighters lost their lives in China, according to data collected by the Ministry of Emergency Management of the People’s Republic of China.
One of the most critical roles of first responders is rescuing and evacuating individuals from burning or collapsed structures, so situational awareness is a critical and life-preserving aspect of their operations.
Thus, accurately tracking the location of first responders and providing them with a comprehensive map of an unfamiliar environment enhances the effectiveness of rescue and exit operations by maintaining situational awareness. Such systems can also enable team commanders outside the hazardous vicinity to clearly visualize the situation facing their teammates to assist in developing evacuation strategies.
One problem facing first responders reliant on systems that perform the operations described above is that these platforms commonly use camera-based technology or light detection and ranging (LiDAR) technology.
In environments in which vision is impaired, such as in smoky areas, camera systems also experience degradation and sometimes failure. Likewise, the scattering of light by smoke particles in these types of environments can negatively impact the ability of LiDAR to detect objects and create accurate maps in smoky environments. These potential failures can put first-responder’s lives and the lives of those they rescue on the line.
In a new paper published in the journal Advanced Intelligent Systems, researchers led by Changhao Chen of the College of Intelligence Science and Technology, China, leverage recent developments in single-chip millimeter-wave (mmWave) radar technology to tackle these challenges and develop “SmokeNAV.”
SmokeNAV sees more clearly
The scientists explain that the low-cost, energy-efficient, and lightweight design of mmWave radar technology makes it ideal for equipping first responders who require small, easy-to-carry kits. Additionally, mmWave radar has previously demonstrated reliable performance in challenging environmental conditions such as regions of smoke, fog, and dust.
Beyond this tech, SmokeNAV also integrates an inertial positioning module that considers human motion to generate data that allows the system to calculate positioning accurately.
“SmokeNav enables the reconstruction of an accurate metric map from mmWave radar data through probabilistic occupancy map construction,” the researchers wrote in their paper. “To achieve a semantic understanding of the environment, we introduce a deep-neural network-based semantic segmentation model that incorporates radar reflectivity to enhance performance.”
To test their proposed SmokeNav system, Chen and colleagues conducted extensive experiments in complex real-world situations.
First, the team evaluated the system’s performance in a long corridor, determining its ability to map the environment and localize the user. The team then investigated SmokeNAV’s ability to map and section off an area in a range of diverse smoke-filled environments. The researchers then extended their evaluation of SmokeNAV to encompass more intricate scenarios, introducing obstacles composed of various materials.
“The experimental results demonstrate that SmokeNav achieves precise user localization and generates reliable maps of the environment with semantic segmentation in smoky environments,” the team wrote. “The single-chip mmWave radar was relatively low cost, lightweight, and energy-efficient compared to solid-state LiDAR. Therefore, mmWave radar was more suitable for integration into wearable localization and mapping systems for first responders.”
However, there is still much work to be done with SmokeNAV before it is ready for the field.
The team found that the system, as it currently stands, generated a lower spatial resolution than LiDAR, which resulted in less detailed maps and reduced accuracy in detecting smaller objects and fine features in the environment. They also discovered that the mmWave radar reflects signals off multiple surfaces, creating “clutter” and “ghost objects,” complicating mapping.
The mmWave radar demonstrated its effectiveness in penetrating smoke, fog, and dust, but its range was generally shorter than that of LiDAR. This hindrancemanifested itself in clear conditions, something that currently limits SmokeNAV’s utility in large open spaces.
The researchers have suggested a number of improvements to SmokeNAV that could improve these performance hindrances and make the system more adept at assessing dangers facing first responders.
“Future research can focus on expanding the capabilities of SmokeNav by incorporating additional sensing modalities and advanced machine-learning techniques,” they wrote. “Exploring the integration of other data sources, such as thermal imaging or gas sensors, could further enhance the system’s performance and enable comprehensive perception in highly dynamic and unpredictable environments.
“Additionally, refining the system’s real-time processing capabilities and exploring optimization techniques can contribute to its practical deployment in real-world scenarios, supporting first responders in their critical missions.”
Reference: Zhiqiang Yao et al., SmokeNav: Millimeter-Wave-Radar/Inertial Measurement Unit Integrated Positioning and Semantic Mapping in Visually Degraded Environments for First Responders, Advanced Intelligent Systems (2024). DOI: 10.1002/aisy.202400241
Feature image credit: Joachim Pressl on Unsplash