Millimeter-Wave Bat for Mapping and Quantifying Micromotions in Full Field of View

Millimeter-Wave Bat for Mapping and Quantifying Micromotions in Full Field of View

PDF

Yuyong Xiong¹, Songxu Li¹, Changzhan Gu², Guang Meng¹, Zhike Peng¹

Research. Vol 2021 Article ID 9787484

Less

Research. Vol 2021 Article ID 9787484

• Research Article •

Millimeter-Wave Bat for Mapping and Quantifying Micromotions in Full Field of View

Full

Yuyong Xiong¹, Songxu Li¹, Changzhan Gu², Guang Meng¹, Zhike Peng¹

Affiliations

¹State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

²MoE Key Lab of Design and Electromagnetic Compatibility of High Speed Electronic System, and MoE Key Lab of Artificial Intelligence, AI Institute, Shanghai Jiao Tong University, Shanghai 200240, China

About Author:

Authors’ Contributions

Y.X. and Z.P. conceived the ideas of the work. Y.X. and S.L. designed the experimental validations, conducted the various experiments, and analyzed the data. C.G. and Y.X. designed the prototype of the mmWBat system. G.M. and Z.P. conceived applications, provided technical feedback, and supervised the research. S.L. and Y.X. made experimental validation demos. Y.X., Z.P., and S.L. wrote the manuscript. All authors discussed the results and commented on the manuscript.

Published: 2021-08-20 doi: 10.34133/2021/9787484

Outline

Abstract

Less

Echolocating bats possess remarkable capability of multitarget spatial localization and micromotion sensing in a full field of view (FFOV) even in cluttered environments. Artificial technologies with such capability are highly desirable for various fields. However, current techniques such as visual sensing and laser scanning suffer from numerous fundamental problems. Here, we develop a bioinspired concept of millimeter-wave (mmWave) full-field micromotion sensing, creating a unique mmWave Bat (“mmWBat”), which can map and quantify tiny motions spanning macroscopic to μm length scales of full-field targets simultaneously and accurately. In mmWBat, we show that the micromotions can be measured via the interferometric phase evolution tracking from range-angle joint dimension, integrating with full-field localization and tricky clutter elimination. With our approach, we demonstrate the capacity to solve challenges in three disparate applications: multiperson vital sign monitoring, full-field mechanical vibration measurement, and multiple sound source localization and reconstruction (radiofrequency microphone). Our work could potentially revolutionize full-field micromotion monitoring in a wide spectrum of applications, while may inspiring novel biomimetic wireless sensing systems.

Cite this Article

Yuyong Xiong, Songxu Li, Changzhan Gu, Guang Meng, Zhike Peng. Millimeter-Wave Bat for Mapping and Quantifying Micromotions in Full Field of View[J]. Research, 2021 , 2021 (8) : 9787484 - . DOI: 10.34133/2021/9787484

Full Text

Less

1. Introduction

Less

Bats are arguably the most unusual mammal with a remarkable capability of ultrasonic echolocation, enabling them to perceive the environment and preys in complete darkness [1]. With large and complex-shaped ears and sophisticated neural processing, echolocating bats exhibit accurate three-dimensional localization of multiple targets in the full field of view (FFOV) [2–5]. Furthermore, although the neural processing mechanism remains somewhat unclear, bats are particularly notable for perception of small motions (e.g., insects’ fluttering wings and frogs’ dynamically inflated vocal sac) using short-duration broadband frequency-modulated (FM) calls, which is critical to distinguish and capture preys especially in noisy or cluttered environments [6–9]. It is worth noting that tiny motions are widespread from the natural world to engineering, including heartbeat and bridge vibrations, which carry a wealth of meaningful physical information [10–12]. The accurate and robust perception of tiny motions is significant in a variety of fields, such as healthcare monitoring [13, 14], human-computer interaction [15], Internet of Things [16], and structural health monitoring [17]. Particularly, imaging and quantifying micromotions in a two-/three-dimensional (2D/3D) space is highly desirable for numerous applications from biology to engineering [18–22], and motivating multiple artificial technologies [23–25].

However, current artificial approaches are mainly based on contact sensors (e.g., wearable devices and accelerometers), camera sensing, and laser scanning, each of which suffers from fundamental problems. A network with a mount of accelerometers is generally time consuming and costly and commonly has low accuracy for displacement measurement, while the contact manner has many limitations in practical applications. Camera-based solutions have rich spatial information but suffer low accuracy and high computing load due to sophisticated processing of image stream data. Typically, camera systems also remain challenging with poor lighting conditions and highly dynamic motion visualization. Laser-based approaches, such as the laser Doppler vibrometer, commonly require scanning to achieve planar or spatial micromotion information, which limits measurement synchronization. In addition, they suffer the drawbacks of high cost, large volume, highly specialized setups, and stringent operating environment requirements. Distinct with the common range and velocity detection [26], emerging radar-based radiofrequency (RF) methods can potentially sense and measure micromotions [27–29]. However, they are limited to monitoring only one target or several targets far apart in one-dimensional space, while struggling with tricky clutter interferences [30, 31].

Here, inspired by echolocating bats in terms of micromotion spatial localization and sensing, we have developed a concept of millimeter-wave (mmWave) full-field micromotion sensing (MFMS), a method for noncontact imaging and monitoring tiny motions in FFOV. We refer to this concept as mmWave Bat (mmWBat), which transmits and receives mmWave signals instead of ultrasonic wave signals, allowing artificial system miniaturization and has a large monitoring range. In mmWBat, multiple transmitter and receiver antennas are arranged for measuring azimuth and elevation, mimicking the two powerful ears of bats, which is quite distinct from the existing approaches of radar-based micromotion sensing (the micromotion is commonly measured within the range profile). By emitting and receiving linear-frequency-modulated continuous wave (LFMCW) across multiple sweeps, we establish the fundamental principle of interferometric phase evolution tracking from range-angle joint dimension, enabling the mmWBat to map and quantify micromotions in FFOV simultaneously and accurately. To illustrate the versatility and appealing advantages of the mmWBat, we demonstrate three disparate applications from biology to engineering: multiperson vital sign monitoring, full-field mechanical vibration measurement, and multiple sound source localization and recovery (termed RF microphone). Our work provides a revolutionary approach for full-field micromotion monitoring in various fields, while offering new perspectives for mmWave sensing as well as potentially inspiring novel biomimetic wireless sensing systems based on an understanding of the perception mechanisms used by echolocating mammals.

2. Concept and Implementation of mmWBat

Less

Figure 1 presents a comparison of the echolocating bat (Figures 1(a) and 1(b)) and mmWBat systems (Figures 1(c) and 1(d)) for micromotion sensing in FFOV. The gleaning bat emits a series of ultrasonic calls by its larynx, listens to the echoes by two ears, and processes the contained information of prey and environment via a sophisticated neural system. The bat’s sonar system is generally equipped with two large and complex-shaped ears (pinnae and tragi), enabling excellent multitarget spatial localization [4, 5]. Although the echolocating calls can be diverse among different species and tasks, the short-duration, high-frequency, and broadband calls are validated to have great benefits for accurate target localization and classification [7, 32]. The biosonar system of bats is exceptionally sensitive of preys with motions from large to tiny movement, allowing effectively foraging in cluttered environments. Large motions can provide cues with obvious Doppler frequency modulation; however, some species of bats (e.g., the common big-eared bat, Micronycteris microtis) have been demonstrated with outstanding capability of micromotion perception in clutter related to informative phase variations, suggesting the neural processing mechanism is just beginning to be understood [6, 7, 9, 33].

Figures 1(c) and 1(d) illustrate the schematic of our biologically inspired mmWBat system, which can distinguish and simultaneously monitor tiny motions of multiple targets in a 2D/3D space. To achieve a long detection range and miniaturize the artificial system, our mmWBat system transmits and receives mmWave signals instead of ultrasonic wave signals. The system consists of a mmWave transceiver with an antenna array, an analogue-to-digital converter (ADC), and a processor, which mimic the biological sonar’s emitting and receiving components, neural coding and processing, respectively. A more detailed structure of the mmWBat appears in Supplementary Fig. 1. The antenna array consisting of multiple receiver antennas distributed horizontally and vertically with a certain aperture is configured for mimicking the two large and powerful ears of bats, enabling azimuth and elevation localization. For practical implementation, the multiple-input multiple-output (MIMO) antenna array can be employed to achieve high angular resolution with a small real aperture. Similarly, the mmWBat transmits short-duration, broadband LFMCW mmWave signals with multiple sweeps, which can achieve better range resolution and tracking sensitivity. Note that although bats always shorten their FM calls to avoid overlap of the receiving echoes [34–36], the artificial mmWBat needs to mix the LFMCW broadcasts and the receiving echoes for interference and modulation due to the ultrafast electromagnetic wave propagation speed (i.e., light speed), allowing range detection and sensitive tiny motion perception. Furthermore, the coherence maintenance of the mmWBat system is critical for micromotion perceiving, as the capability of temporal coincidence between sound production and echo reception in the bat’s central auditory system, which can be performed by sharing the clock between the ramp generator and ADC or sampling the baseband signal and ramp signal synchronously.

Here, we demonstrate the proposed fundamental principle of MFMS with the mmWBat system. Figures 2(a) and 2(b) show the schematic of data flow across multiple sweeps with multiple receiver channels and multitarget mapping of the mmWBat. By performing beat frequency estimation of the baseband signal of a certain channel, the mmWBat can commonly achieve the range profile. Although the angular resolution mechanism with digital beamforming depending on an antenna array is well understood, the micromotion quantifying in angle dimension has not been exploited and is considered challenging because of ultrahigh angular resolution requirements, as well as insensitivity to tiny motions. In MFMS, we propose to extract the micromotion information of full-field targets via interferometric phase evolution tracking from the range-angle joint dimension. For ease of illustration, we first consider the single-target scenario. According to the LFMCW ranging principle, the target’s reflection produces a beat signal component and the component corresponding to the

i

th chirp can be simply formulated as

S B i T + t = A t exp j 2 π f b t + φ R + φ i

, where

A t

is the amplitude of the signal and

f b

and

φ R

are the beat frequency and fixed phase shift corresponding to the distance between the target and the antenna, respectively.

φ i

is the modulated phase which is sensitive to tiny motions. Ignoring the amplitude variation along fast time and performing the beat frequency demodulation, we can represent this component with the phasor

A exp j φ R + φ i

(see Supplementary Note 1). Generally, we assume the multiple receiver antennas (or equivalent virtual antenna array) of the mmWave sensing system construct a uniformly spaced linear array. According to the geometric relationship of the propagation paths between different antennas (see Supplementary Fig. 2a), the phasor representation of multichannel downconverted baseband signals for the

i

th chirp can be approximated and simplified (see Supplementary Note 1) as

(1) A exp j φ R exp j φ i exp j 2 π d sin θ λ c + φ i ⋮ exp j 2 π M − 1 d sin θ λ c + φ i,

where

M

is the total number of equivalent receiver antennas,

d

is the distance between adjacent receiver antennas,

θ

is the incident angle (i.e., the azimuth or elevation angle of the target in FFOV), and

λ c

is the wavelength corresponding to the central ramp frequency. It is observed that for the angle-dimensional multichannel phasor signal of each sweep, its frequency is equivalent to

d sin θ / λ c

and its phase is equivalent to

φ R + φ i

. Therefore, we show that the key phase evolution (i.e.,

φ i, i = 1, 2, ⋯

) information corresponding to the micromotion displacement in range dimension can be reserved and equally transferred to angle dimension. As a result, for mmWave micromotion sensing, we can ultimately estimate and extract the phase history across multiple sweeps from the angle dimension.

Extending to common full-field multitarget scenarios, the angle-dimensional phasors regarding multichannel baseband signals can be expressed as

(2) ∑ l = 1 L A l exp j φ R l + φ i l 1 exp j 2 π d sin θ l λ c ⋮ exp j 2 π M − 1 d sin θ l λ c,

where

l = 1, 2, ⋯ L

L

is the total number of detectable targets and

θ l

is the incident angle of the

l

th target. Therefore, we can simultaneously measure and track the phase evolution corresponding to each target from the converted angle dimension. With the aspect of phasor analysis, it is worth noting that our approach is equivalent to separating and isolating the desired components combined with range dimension and angle dimension, offering great benefits of the challenging clutter interference suppression especially for the mutual coupling issues (see Supplementary Note 2) in real scenarios. In addition, for 3D spatial multitarget localization, the range-azimuth and the range-elevation estimations should be performed in succession. However, when extracting the micromotion displacements of full-field targets, only one of the azimuth and elevation angle dimensions is employed to form the range-angle joint dimension.

For practical implementation, Figure 2(c) shows the flow chart of micromotion measurements of full-field targets with the mmWBat, and Figure 2(d) presents the schematic illustrating how to extract the time-domain displacement signals of a certain target via phase variation estimation in the range-angle joint dimension. In Materials and Methods and Supplementary Fig. 2, we show the basic algorithm of full-field phase evolution tracking. Consequently, according to the interferometric measurement principle, the displacement time series of micromotion of each target can be extracted as

x l, i T = λ c φ l, i T − φ l_mean / 4 π / cos ϕ l

, where

φ l, i T

is the extracted variation phase of target

l

for the

i

th sweep,

φ l_mean

is the mean of

φ l, i T, i = 1, 2, ⋯

, and

ϕ l

is the angle between motion direction and the mmWave line-of-sight of target

l

3. Results

Less

3.1. Multi-Person Vital Sign Monitoring with mmWBat

The mmWBat system offers great potential for revolutionizing healthcare solutions in biomedical engineering, which is not wearable, regardless of bad lighting and with complete privacy. Particularly, the example involves noncontact vital sign (i.e., respiration and heartbeat) monitoring is valuable for health disorder evaluation, disease diagnostic, and sleep quality analysis of people, especially for infants and the elderly [37]. Moreover, it is worth noting that the mmWBat can provide a cost-efficient way to simultaneously monitor the vital signs of multiple persons in public places including nursing homes and hospitals. For example, it can perform the health monitoring of a large number of patients in mobile cabin hospitals combating COVID-19, which can save a lot of medical resources, offer long-term comfortable monitoring, and reduce the infection risk of medical workers [38, 39].

The intrinsic principle of vital sign detection with RF sensing is measuring the micromotion of the chest wall caused by cardiopulmonary activity. Although many efforts have been made with microwave radar-based methods, two main challenges remain that prevent the widespread application of this technology. Firstly, since the chest wall movement caused by heartbeat is much smaller than that of respiration, and the movements are averaged with multipoint scattering effect, the current RF-based far-field vital sign monitoring cannot retrieve the heart rate (HR) accurately and reliably in practical scenarios even with careful processing. Secondly, for multiperson (e.g., a couple and their baby in the bed) monitoring scenarios, it requires individuals to be far apart from each other to mitigate mutual interference. The mmWBat system offers an effective approach to address these challenges by full-field micromotion sensing on the multiple body parts of one individual, as well as multiple individuals. For per individual, the mmWBat can automatically locate the chest position of the heartbeat according to the characteristics of heartbeat activity, allowing a great reduction in the average effect of multipoint scattering, which can significantly enhance the desired heartbeat component. For multiple individuals, the mmWBat is attractive to eliminate the tricky mutual interference with sensing in range-angle joint dimension. Note that, since the human body is a flexible continuum, it requires the mmWBat to have relatively high angular resolution for achieving excellent performance.

Figure 3 presents the experimental setup (Figure 3(a)) and results (Figures 3(b)–3(d)) of multiperson vital sign monitoring. To mimic a multiuser scenario in a real-world setting, we recruited three volunteers to sit on a couch shoulder by shoulder in a break room and they stay in a quasistatic situation without large body movement. The mmWBat device sensed and monitored their vital signs simultaneously from a distance of about 2 m, and additional details appear in Materials and Methods. As an example, the extracted chest wall displacement signal of the volunteer on the left is shown in Figure 3(b) and that of all individuals are shown in Supplementary Video 1. It is observed that the desired heartbeat component can be well retrieved with full-field micromotion sensing and localization from different body parts. With sliding window processing, Figure 3(c) presents the comparison results of the tracked respiration rate (RR) and HR time series and the corresponding references, which show a good match. The challenging HR tracking, in particular, can be reliably achieved via heartbeat component enhancement sensing with our proposed MFMS method. Moreover, we evaluate the monitoring accuracy of all three individuals in percentage by defining confidence intervals (see Figure 3(d)), with most measurement errors less than ±3 Beats/Min. In addition, the composition of the success percentage with deviation within 1 b.p.m., 2 b.p.m., and 3 b.p.m. is provided to illustrate the agreement performance. These results demonstrate that mmWBat can monitor multiple individuals’ vital signs (even with zero separation) and micromotions of different body parts, exhibiting a promising technique for contactless vital sign monitoring.

3.2. Full-Field Mechanical Vibration Measurement via mmWBat

The mmWBat is capable of micromotion mapping and quantification of multiple targets or points simultaneously, thereby creating interesting opportunities for full-field mechanical vibration monitoring. Our approach offers appealing advantages as nonintrusive, full-field, large measurement region and vibration scales (μm to m), along with high reliability in harsh environments, which are highly desirable for structural health monitoring, modal analysis, and machinery diagnostics in mechanical, aerospace, and civil engineering. Here, we validate and compare the performance of the mmWBat and the existing radar-based technique by conducting experiments with various scenarios. Figures 4(a) and 4(b) presents the block diagram and a photograph of the experimental setup. As shown in Figure 4(c), we mimic three typical measurement scenarios (similar to gleaning bats), which can comprehensively examine the significant performance improvement of our approach and highlight the key aspect of clutter interference suppression for mmWave sensing. Additional details concerning the experimental setup appear in Materials and Methods.

Figure 4(d) depicts the comparison result of the typical scenario when the two targets are located in adjacent range bins. In conventional radar-based micromotion sensing, the adjacent clutter interference (caused by the reflection of adjacent objects) is common and has an obvious effect on the measurement accuracy of both targets. It is challenging to evaluate and suppress the adjacent clutter due to the mutual coupling between components (see Supplementary Note 2). However, the mmWBat system can effectively eliminate the adjacent clutter interference from the range-angle joint dimension and obtain accurate displacement measurement results (see Figure 4(d)). In addition, Figure 4(g) illustrates the benefit of the MFMS method when the two targets are placed at different azimuth angles. The root mean square error (RMSE) of measurements is adopted to better evaluate the performance. We show that our approach can achieve much better performance in all cases and the measurement error is stably small, offering an effective strategy to eliminate the tricky clutter interference.

For the scenario when the two targets are located in a same range bin, we also provide corresponding comparison results with a similar form (see Figures 4(e) and 4(g)). Obviously, the existing radar-based technique is difficult or impossible to perform phase evolution tracking due to the severe aliasing problems, which results in failure of multitarget vibration measurement and sensitive to the demodulation frequency. In contrast, the MFMS method can accurately extract the vibration displacement of each target via angle-dimensional demodulation, illustrating the effectiveness and significant advantages of our technique (see the good agreement between reference and measured results). Note that due to the heavy coupling in range dimension, the measurement accuracy in this scenario is indeed slightly lower than the previous scenario, which can be solved by increasing the angular resolution of the mmWBat system. Moreover, we further evaluate the performance when the two targets are close in angle dimension (i.e., targets have similar azimuth angles in our example). Experiment results (see Figure 4(f)) show that although the two targets are adjacent in angle profile, the proposed approach can also achieve better measurement accuracy than the conventional technique. As shown in Figure 4(f), since targets A and B are separated far apart, the current technique can achieve a relatively small measurement error due to the weak interference. However, the proposed method can further improve the measurement accuracy even with adjacent angles, which benefits from the isolation and distribution in range-angle joint dimension. Examples of full-field vibration monitoring with the three typical scenarios illustrated are shown in Supplementary Video 2.

Furthermore, Supplementary Video 3 presents the full-field dynamic monitoring of a scaled bridge (

18 cm × 90 cm

) under different load conditions, offering an example of vibration and deformation-based structural health monitoring of large structures. With the mmWBat system, we can accurately and remotely measure full-field multipoint displacements of the scaled bridge, creating an effective approach to solve the problem of dynamic deflection monitoring and torsion monitoring. Note that for the torsion monitoring of a long-span bridge in practical applications, owing to the remote monitoring requirement, the system may need to be equipped with more equivalent receiving antennas, which can better eliminate the coupling interference between two parallel measurement points distributed on both sides of the bridge structure.

3.3. mmWBat as RF Microphone

We also applied the mmWBat to the field of sound source localization and reconstruction, which is significant and useful in a wide range of contexts such as sound separation and enhancement, target recognition, surveillance, and security [40, 41]. Here, we present an innovative approach to locate and reconstruct multiple sound sources using the mmWBat, creating a unique RF microphone. The key enabler underlying our RF microphone is the proposed MFMS method and the fact that sound is produced by micromotions with magnitude of μm length scale and high frequency. With the mmWBat, we detect and monitor the slight surface movements of sound sources via FFOV sensing. Our approach exhibits several exceptional advantages compared to existing techniques such as microphone array. For example, regardless of the characteristics of multiple sound sources and whether they are coherent, we can achieve high directivity and separation with small size and power, while recovering the high-quality audio signal of each source. Moreover, the RF microphone can easily obtain high-performance range and angular positioning without complex and time-consuming estimation calculations.

Figure 5(b) shows the implementation procedure for the RF microphone, which is detailed in Materials and Methods. As a result, we can clearly distinguish the sound sources and obtain their localization information, including their distance and direction, while the extracted micromotion signals can be employed for reconstructing the corresponding sound signals. We validate the feasibility and performance of the RF microphone technique with experiments including three loudspeakers (i.e., sound sources). Figure 5(a) presents the experimental setup, and additional details appear in Materials and Methods. To mimic real scenarios, we put several stationary objects and the three sound source targets in the detection region. As shown in Figure 5(b), we first achieved the range-angle heatmap of all targets and tracked the phase evolution signals corresponding to each possible target based on the MFMS method. Then, we can further process to identify the sound sources (see Materials and Methods and Supplementary Figure 3). Consequently, we were able to extract the range and angle information of all sound source targets accurately (see Figure 5(b), which matches well with the ground truth obtained by rulers), and reconstruct the corresponding audio signal of each source. In the experiment, the three sound sources are different, offering a comprehensive testing scenario. For better evaluation, we show the comparison results of the time-frequency representations of the recovered signal and original audio for each source (see Figure 5(c)–5(e)). It is seen that the RF microphone can effectively separate different sound sources and accurately recover the audio signals. Moreover, as a comparison, the sound signal captured by the traditional microphone of a cellphone is also provided (see Figure 5(f)). Obviously, the conventional microphone cannot separate and isolate different sound sources, which leads to severe sound aliasing. To intuitively evaluate our reconstructed audios with the comparison of the traditional microphone, we recommend listening to Supplementary Video 4.

Note that the RF microphone has a relatively low recovery performance for music sound which generally has many high-frequency components (see Figure 5(e)). This is because the micromotion corresponding to high frequency is ultrasmall, which is susceptible to noise interference. However, the RF microphone can also successfully recover a clear music sound even if high-frequency components are not well preserved (see Supplementary Video 4). Furthermore, for better perception and reconstruction of high-frequency sound, it suggests achieving submicron displacement measurement accuracy which requires high signal-to-noise ratio (SNR). Since the RF microphone has excellent capability for sound source localization, we can further use the beam steering technique to focus on the sound source target of interest, which can well improve the SNR and reduce interference, allowing high-quality broadband sound recovery. Considering that traditional microphones work in response to sound pressure, the RF microphone can be extended to sense and extract the minute motions of other objects stimulated by sound pressure, which is potentially useful for handling scenarios where sound sources are obscured.

4. Discussion

Less

Here, we have demonstrated a bioinspired mmWBat system for mapping and quantifying micromotions in FFOV. Its structure is functionally similar to that of the echolocating bat’s biosonar system. In particular, we develop a fundamental concept of MFMS, which enables us to image and measure tiny motions of full-field targets simultaneously and accurately, providing significant technological advance compared with the current emerging radar-based approaches. We reveal the intrinsic transfer mechanism of the interferometric phase evolution information between range and angle dimension, allowing full-field micromotion sensing and measuring from the range-angle joint dimension. Meanwhile, the MFMS method also provides an effective strategy for tackling the challenge of tricky clutter interference in mmWave sensing. The potential and appealing advantages of our mmWBat concept are highlighted in three disparate applications, namely, multiperson noncontact vital sign monitoring, full-field mechanical vibration measurement, and multiple sound source localization and reconstruction, which provide novel approaches and insights into addressing challenges of micromotion sensing in various fields. We envision that our approach can revolutionize micromotion monitoring technology with a large monitoring range, full-field synchronous measurement, multiscale visualization, high accuracy, robustness, and low cost. Furthermore, our work opens up new perspectives and motivates interesting research of mmWave sensing, while possibly inspiring novel biomimetic wireless sensing systems with an understanding of the perception mechanisms used by echolocating mammals.

5. Materials and Methods

Less

5.1. Algorithm of Full-Field Phase Evolution Tracking

The basic algorithm can be implemented with the following two main steps:

(1)	(1) Full-field target localization and phasor index

As shown in Supplementary Fig. 2b, we first choose a certain (e.g., the first) sweep multichannel baseband signal

H = S 1, ⋯ S m, ⋯ S M

, where

S m m = 1, ⋯ M

denotes the corresponding

m

th channel signal and then perform the calculation with a two-dimensional fast Fourier transform (FFT). Specifically, we first calculate the FFT for each channel baseband signal (i.e., each column of

H

) to obtain the range-dimensional phasor matrix

H f

. Then, we calculate the FFT for each row of

H f

(i.e., phasors of multiple channels) and denote the obtained matrix as

H f f

. Note that to refine the spectrum and reduce the fence effect, we can apply the commonly used zero-padded FFT for practical calculations.

Next, we perform a simple peak detection with

∑ p = 0 P − 1 a b s S k, p

, where

S k, p

is the element in row

k

and column

p

of matrix

H f f

a b s ·

denotes taking the complex magnitude, and

P

is the total column number of

H f f

. Accordingly, we can achieve the located range bins and the peak index

k l l = 1, ⋯

of full-field targets. Similarly, the located angle bins and the corresponding index

p l l = 1, ⋯

can be obtained by directly employing the magnitude peak detection on row

k l

of matrix

H f f

. The obtained phasor corresponding to the desired component of each target by FFT and index search has a certain error with the ground truth, but fortunately, it has no effect on the phase variation tracking (see Supplementary Note 2 for details).

(2)	(2) Full-field phase evolution estimation

When the heatmap and the corresponding phasor indexes of full-field targets are obtained, the phase evolution of each target can be estimated as

(3) φ l, i T = arg ∑ m = 0 M − 1 ∑ n = 0 N − 1 s i n, m e − j 2 π n k l / N z e − j 2 π m p l / M z,

where

φ l, i T

denotes the estimated initial phase corresponding to the

i

th sweep period of the

l

th target,

arg ·

denotes taking the phasor angle,

s i ·

is the multichannel baseband signal of the

i

th sweep period with

N

rows and

M

columns, and

N Z

and

M Z

are the FFT sizes of the first and second FFT calculations in the previous step, respectively. In addition, due to the calculated phase angle that always lies between

± π

, it is essential to perform the unwrap procedure for practical use.

5.2. Experimental Setup and Implementation

5.2.1. Construction of the mmWBat System Prototype

The mmWBat system prototype built consists of a commercial mmWave transceiver, a data capture card, and a laptop. To achieve different angle resolution capacity, we adopted two types of mmWave transceivers: (i) AWR1443, Texas Instruments, which mainly includes microcontroller, single-chip front-end (Tx power: 12 dBm, carrier frequency: 77-79 GHz are used) and onboard antennas (2 Txs and 4 Rxs are used) and (ii) AWR1243P cascade, Texas Instruments, which mainly consists of 4-chip AWR 1243P. The data capture card streams the transceiver raw data over Ethernet to the laptop. The mmWBat system works in LFMCW mode with sawtooth modulation. We use software (mmWave Studio) for transceiver parameter setting, data acquisition, and control. The raw data is saved and processed offline using MATLAB R2017b.

5.2.2. Setup of Multi-person Vital Sign Monitoring Experiments

Three volunteers are asked to sit on a couch shoulder by shoulder and breathe normally. During experiments, they are in a state of daily behavior (i.e., reading books or using their cell phones) in quasistatic situations. A breathing belt and a finger pulse sensor are adopted for each individual, which provide the references of RR and HR time series, respectively. The reference signals are captured by a DAQ device (USB-6210, National Instruments), which is synchronous with the mmWBat system. The second type of mmWBat system prototype is employed in this application, and the main parameters of the mmWBat system are that the transmitted bandwidth is 3 GHz and the sweep cycle is 10 ms.

5.2.3. Setup of Full-Field Mechanical Vibration Measurement Experiments

In these experiments, two corner reflectors are employed as the targets. They are mounted on two linear stages and controlled to achieve different vibration movements (target A: triangular pattern, target B: sinusoidal pattern), respectively. The mimicked vibration device includes two linear stages and a controller with a LabVIEW control interface. To mimic the three typical measurement scenarios, we place target A and the mmWave transceiver on an optical table in different positions. During the experiments, we utilize two laser displacement sensors (LK-G80, Keyence) to provide the ground truths of the two target displacements, respectively. The vibration signals measured by laser sensors are captured by a DAQ device (USB 4431, National Instruments), which is synchronous with mmWBat raw data acquisition. The first type of mmWBat system prototype is employed. With the built system, we employ 2Tx-4Rx antenna configuration with time division multiplexing. The sweep cycle is set to 4 ms, and the transmitting bandwidth is set to 2 GHz (i.e., range resolution is 7.5 cm).

5.2.4. Implementation Procedure for the RF Microphone

First, we achieve the range-angle profile of the monitoring region illuminated by the mmWave beam. Then, we identify the sound source targets depending on the micromotion characteristics of the sound signals. To quickly locate and reduce the amount of calculation, we extract the phase evolution signal of each possible target along slow time according to the range-angle heatmap. To eliminate the interference of low-frequency movement objects and possible undesirable minor phase shift across sweeps due to the temperature drift of hardware, we apply the high-pass filtering (finite impulse response high-pass filter with a cut-off frequency of 40 Hz) to the extracted phase evolution signals. The key consideration for sound localization via mmWBat is to distinguish the sound source targets from other likely static objects. Since the phase history corresponding to a static object is basically broadband thermal noise, there is no obvious frequency component. In contrast, the sound source target usually has rich frequency components which change dynamically. Therefore, we propose to evaluate the time-varying sparseness (TVS) of the amplitude spectrum of each phase history signal with a sliding window. Specifically, we perform an FFT on each window and calculate the ratio of the peak value to the average value of the obtained amplitude spectrum. Then, we distinguish the sound source targets and others with the indicator

TVS = a r t mean + b r t std

, where

a

and

b

are weight coefficients (which are set to 1 and 2, respectively, in our implementation due to the obvious time-varying of sound signal spectrums),

r t mean

and

r t std

are the mean and standard deviation of

r t

(i.e., the calculated ratio time series), respectively. If the indicator is larger than the threshold (empirically set to 10), we consider it a sound signal component.

5.2.5. Setup of RF Microphone Experiments

The mmWBat device employed here is the same as in the full-field mechanical vibration measurement experiments. The difference in parameter settings is that the sweep cycle is set to 0.2 ms (i.e., micromotion displacement sampling frequency is 5 kHz). In the experiments, all targets are placed on an optical table (length: 2 m, width: 1.5 m). In terms of the three sound source targets, source 1 is driven by an arbitrary waveform generator which outputs voltage signals with sinusoidal modulation, source 2 is driven and input by a computer and a power amplifier with a tone signal, and source 3 is input to music signals via Bluetooth. The traditional microphone of a cellphone (P30, Huawei) captured the audio signal via a sound recorder app, and the cellphone is placed next to the mmWave transceiver.

5.2.6. Signal Processing Implementation of Experimental Validations

According to the basic algorithm of full-field phase evolution tracking, when performing 2D FFT for multichannel baseband signals, we used the zero-padded FFT technique with

N z = 2 N

and

M z = 180

. For multiperson vital sign monitoring experiments, after extracting the chest wall displacement time series, we performed the RR and HR tracking by using the sliding window technique (window length: 15 s, step size: 1 s). For each window, the RR is estimated by autocorrelation analysis after band-pass filtering (0.1-0.9 Hz) and the HR is estimated by first-order differential enhancement analysis. For RF microphone experiments, the time-frequency representations of all recovered and original sound signals are achieved by short-time Fourier transform (STFT) with a sliding window size of 1024.

Funding

Less

China Postdoctoral Science Foundation(2019M651488)
National Natural Science Foundation of China(11632011; 51905341)

References

Less

Kunz

T. H.

, and Fenton

M. B.

Bat Ecology, University of Chicago Press, 2005

Finkelstein

, Derdikman

, Rubin

, Foerster

J. N.

, Las

, and Ulanovsky

, “Three-dimensional head-direction coding in the bat brain,” Nature, vol. 517, no. 7533, pp. 159–164, 2015

Aytekin

, Grassi

, Sahota

, and Moss

C. F.

, “The bat head-related transfer function reveals binaural cues for sound localization in azimuth and elevation,” The Journal of the Acoustical Society of America, vol. 116, no. 6, pp. 3594–3605, 2004

Lawrence

B. D.

, and Simmons

J. A.

, “Echolocation in bats: the external ear and perception of the vertical positions of targets,” Science, vol. 218, no. 4571, pp. 481–483, 1982

Wohlgemuth

M. J.

, Luo

, and Moss

C. F.

, “Three-dimensional auditory localization in the echolocating bat,” Current Opinion in Neurobiology, vol. 41, pp. 78–86, 2016

Gomes

D. G. E.

, Page

R. A.

, Geipel

, Taylor

R. C.

, Ryan

M. J.

, and Halfwerk

, “Bats perceptually weight prey cues across sensory systems when hunting in noise,” Science, vol. 353, no. 6305, pp. 1277–1280, 2016

Geipel

, Jung

, and Kalko

E. K. V.

, “Perception of silent and motionless prey on vegetation by echolocation in the gleaning bat Micronycteris microtis,” Proceedings of the Royal Society B, vol. 280, no. 1754, article 20122830, 2013

Corcoran

A. J.

, and Moss

C. F.

, “Sensing in a noisy world: lessons from auditory specialists, echolocating bats,” The Journal of Experimental Biology, vol. 220, no. 24, pp. 4554–4566, 2017

Fenton

M. B.

, Grinnell

A. D.

, Popper

A. N.

, and Fay

R. R.

Bat Bioacoustics, Springer, New York, 2016

Ghaffari

, Aranyosi

A. J.

, and Freeman

D. M.

, “Longitudinally propagating traveling waves of the mammalian tectorial membrane,” Proceedings of the National Academy of Sciences of the United States of America, vol. 104, no. 42, pp. 16510–16515, 2007

Hui

, and Kan

E. C.

, “No-touch measurements of vital signs in small conscious animals,” Science Advances, vol. 5, article eaau0169, 2019

Zhao

, Adib

, and Katabi

, “Emotion recognition using wireless signals,” in Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, New York, 2016, pp. 95–108

Hui

, and Kan

E. C.

, “Monitoring vital signs over multiplexed radio by near-field coherent sensing,” Nature Electronics, vol. 1, no. 1, pp. 74–78, 2018

, Lubecke

V. M.

, Boric-Lubecke

, and Lin

, “A review on recent advances in Doppler radar sensors for noncontact healthcare monitoring,” IEEE Transactions on Microwave Theory and Techniques, vol. 61, no. 5, pp. 2046–2060, 2013

Lien

, Gillian

, Karagozler

M. E.

, Amihood

, Schwesig

, Olson

, Raja

, and Poupyrev

, “Soli: ubiquitous gesture sensing with millimeter wave radar,” ACM Transactions on Graphics, vol. 35, pp. 1–19, 2016

Bhuiyan

M. Z. A.

, Wu

, Wang

, Chen

, and Wang

, “Quality-guaranteed event-sensitive data collection and monitoring in vibration sensor networks,” IEEE Transactions on Industrial Informatics, vol. 13, no. 2, pp. 572–583, 2017

Magalhães

, Cunha

Á.

, and Caetano

, “Vibration based structural health monitoring of an arch bridge: from automated OMA to damage detection,” Mechanical Systems and Signal Processing, vol. 28, pp. 212–228, 2012

Haque

, Milstein

, and Fei-Fei

, “Illuminating the dark spaces of healthcare with ambient intelligence,” Nature, vol. 585, no. 7824, pp. 193–202, 2020

Sellon

J. B.

, Farrahi

, Ghaffari

, and Freeman

D. M.

, “Longitudinal spread of mechanical excitation through tectorial membrane traveling waves,” Proceedings of the National Academy of Sciences of the United States of America, vol. 112, no. 42, pp. 12968–12973, 2015

Seguel

, and Meruane

, “Damage assessment in a sandwich panel based on full-field vibration measurements,” Journal of Sound and Vibration, vol. 417, pp. 1–18, 2018

Adib

, Mao

, Kabelac

, Katabi

, and Miller

R. C.

, “Smart homes that monitor breathing and heart rate,” in Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, New York, 2015, pp. 837–846

Peng

, Munoz-Ferreras

J. M.

, Tang

, Liu

, Gomez-Garcia

, Ran

, and Li

, “A portable FMCW interferometry radar with programmable low-IF architecture for localization, ISAR imaging, and vital sign tracking,” IEEE Transactions on Microwave Theory and Techniques, vol. 65, pp. 1334–1344, 2016

, Shuang

, Ma

, Li

, Zhao

, Wei

, Liu

, Hao

, Qiu

C.-W.

, and Cui

T. J.

, “Intelligent metasurface imager and recognizer,” Light: Science & Applications, vol. 8, 2019

Zaurin

, and Catbas

, “Integration of computer imaging and sensor data for structural health monitoring of bridges,” Smart Materials and Structures, vol. 19, article 015019, 2009

Soh

P. J.

, Vandenbosch

G. A.

, Mercuri

, and Schreurs

D. M.-P.

, “Wearable wireless health monitoring: current developments, challenges, and future trends,” IEEE Microwave Magazine, vol. 16, no. 4, pp. 55–70, 2015

Komissarov

, Kozlov

, Filonov

, and Ginzburg

, “Partially coherent radar unties range resolution from bandwidth limitations,” Nature Communications, vol. 10, article 1423, 2019

Wang

, Munoz-Ferreras

J.-M.

, Gu

, Li

, and Gomez-Garcia

, “Application of linear-frequency-modulated continuous-wave (LFMCW) radars for tracking of vital signs,” IEEE Transactions on Microwave Theory and Techniques, vol. 62, no. 6, pp. 1387–1399, 2014

Xiong

, Peng

, Xing

, Zhang

, and Meng

, “Accurate and robust displacement measurement for FMCW radar vibration monitoring,” IEEE Sensors Journal, vol. 18, pp. 1131–1139, 2017

Xia

, Shandhi

M. M. H.

, Inan

O. T.

, and Zhang

, “Non-contact sensing of seismocardiogram signals using microwave Doppler radar,” IEEE Sensors Journal, vol. 18, no. 14, pp. 5956–5964, 2018

Xiong

, Peng

, Jiang

, He

, Zhang

, and Meng

, “An effective accuracy evaluation method for LFMCW radar displacement monitoring with phasor statistical analysis,” IEEE Sensors Journal, vol. 19, no. 24, pp. 12224–12234, 2019

Yue

, He

, Wang

, Rahul

, and Katabi

, “Extracting multi-person respiration from entangled RF signals,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 2, pp. 1–22, 2018

Schnitzler

, and Kalko

E. K. V.

, “Echolocation by insect-eating bats,” Bioscience, vol. 51, no. 7, pp. 557–569, 2001

Kuc

, and Kuc

, “Bat wing air pressures may deflect prey structures to provide echo cues for detecting prey in clutter,” The Journal of the Acoustical Society of America, vol. 132, no. 3, pp. 1776–1779, 2012

Simmons

J. A.

, Ferragamo

M. J.

, and Moss

C. F.

, “Echo-delay resolution in sonar images of the big brown bat, Eptesicus fuscus,” Proceedings of the National Academy of Sciences of the United States of America, vol. 95, no. 21, pp. 12647–12652, 1998

Finneran

J. J.

, Jones

R. A.

, Mulsow

, Houser

D. S.

, and Moore

, “Jittered echo delay resolution in bats and dolphins,” The Journal of the Acoustical Society of America, vol. 144, no. 3, pp. 1773–1773, 2018

Bates

M. E.

, and Simmons

J. A.

, “Effects of filtering of harmonics from biosonar echoes on delay acuity by big brown bats (Eptesicus fuscus),” The Journal of the Acoustical Society of America, vol. 128, no. 2, pp. 936–946, 2010

Korhonen

, Parkka

, and Van Gils

, “Health monitoring in the home of the future,” IEEE Engineering in Medicine and Biology Magazine, vol. 22, no. 3, pp. 66–73, 2003

Zhang

, Wang

, Zhao

, Guo

, Xu

, Ye

, Ding

, Wang

, Ye

, Pan

, Liu

, Li

, Luo

, Liu

, and Wan

, “The clinical characteristics and prognosis factors of mild-moderate patients with COVID-19 in a mobile cabin hospital: a retrospective, single-center study,” Frontiers in Public Health, vol. 8, p. 264, 2020

World Health Organization Infection prevention during transfer and transport of patients with suspected COVID-19 requiring hospital care, World Health Organization, 2020

McDonald

J. J.

, Teder-SaÈlejaÈrvi

W. A.

, and Hillyard

S. A.

, “Involuntary orienting to sound improves visual perception,” Nature, vol. 407, no. 6806, pp. 906–908, 2000

Wood

S. U.

, Rouat

, Dupont

, and Pironkov

, “Blind speech separation and enhancement with GCC-NMF,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 25, pp. 745–755, 2017

Appendix

Less

Year 2021 volume 2021 Issue 8

PDF

Cite this Article

BibTeX

Article Info

doi: 10.34133/2021/9787484

Receive Date：2021-03-24
Online Date：2026-10-27
Published：2021-08-20

Article Data

Affiliations

History

Received：2021-03-24
Accepted：2021-07-04

Funding

China Postdoctoral Science Foundation(2019M651488)

National Natural Science Foundation of China(11632011; 51905341)

Affiliations

¹State Key Laboratory of Mechanical System and Vibration, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

References

Share

https://castjournals.cast.org.cn/joweb/research/EN/10.34133/2021/9787484

Share to

Scan QR to access full text

Cite this article

BibTeX

Citations

表12种不同金属材料的力学参数

科 Family	属数 Number of genus	种数 Number of species	占总种数比例 Percentage of total species (%)	属 Genus	种数 Number of species	占总种数比例 Percentage of total species (%)
鹅膏菌科Amanitaceae	2	11	5.26	鹅膏菌属 Amanita	10	4.78
小菇科 Mycenaceae	2	12	5.74	丝盖伞属 Inocybe	5	2.39
多孔菌科 Polyporaceae	8	14	6.70	蜡蘑属 Laccaria	5	2.39
红菇科 Russulaceae	3	23	11.00	小皮伞属 Marasmius	6	2.87
				小菇属 Mycena	11	5.26
				光柄菇属 Pluteus	5	2.39
				红菇属 Russula	17	8.13
				栓菌属 Trametes	5	2.39

关闭全屏

BibTeX
EndNote
RefWorks
TxT

Figure 1 Overall comparison of the echolocating bat and the mmWBat. (a) Schematic of the echolocating bat with multitarget tiny motion perception. (b) Schematic of the biosonar system. (c) Schematic of the mmWBat system with micromotion perception in FFOV. (d) Block diagram of the mmWBat system. Txs: transmitters; Rxs: receivers; ADC: analogue-to-digital converter.

Figure 2 Principle and implementation of the mmWBat. (a) Series of multiple chirps (i.e., sweeps) and data flow. The multichannel baseband signals can be arranged and represented to be a three-dimensional matrix. For per channel, fast time corresponds to the sampling baseband signal of each chirp and slow time corresponds to multiple chirps along time. $T$ : sweep period; $B$ : transmitted bandwidth. (b) Schematic of multitarget imaging heat map of the mmWBat. Different colors represent different signal strengths. (c) Flow chart of quantifying micromotions of full-field targets. (d) Schematic of micromotion displacement extraction across multiple sweeps in range-angle joint dimension (one target is shown as an example). dim-1: range-dimension; dim-2: angle-dimension.

Figure 3 Multi-person vital sign monitoring with the mmWBat. (a) Photograph of experiment scenario. (b), Measured result of the chest wall displacement signal corresponding to the individual on the left. The inset shows the obvious heart beat component. (c) RR and HR tracking results of the individual on the left with comparison of the references. Two dotted lines on both sides of the reference are the defined deviation boundaries of ±3 Beats/Min (i.e., confidence intervals). (d) Vital sign monitoring accuracy of all three individuals. The accuracy is calculated as the percentage of the time when the measured rate falls into the confidence interval, which contains the success rates that the RR and HR are different from the corresponding references of 1 b.p.m., 2 b.p.m., and 3 b.p.m., respectively.

Figure 4 Experimental results of full-field mechanical vibration measurement. (a, b) Experimental setup: block diagram (a) and photograph (b). (c) Schematic illustration of three typical measurement scenarios, i.e., adjacent clutter (A), aliasing clutter (B), and adjacent angle (C). (d–f) Comparison results of displacement tracking of targets A and B regarding three typical scenarios. Adjacent clutter scenario (d) and aliasing clutter scenario (e) with $θ 1$ and $θ 2$ about 15° (one angular resolution). Adjacent angle scenario (f) when $θ 1$ and $θ 2$ are similar. (g) Comparison of displacement measurement accuracy (i.e., RMSE) of targets A and B with different azimuth angles (i.e., $θ 1 ≈ θ 2 ≈ 10 °, 15 °, ⋯, 35 °$ ) corresponding to the first two typical scenarios.

Figure 5 RF microphone. (a) Photograph of the experimental scenario. (b) Schematic illustration of procedures and results of sound source localization and recovery. (c–e) Comparison results of sound recovery regarding sound 1 (c), sound 2 (d), and sound 3 (e). Recovered sound signal is with time-domain (A) and time-frequency (B) representation, and the original sound signal is with time-frequency representation (C). (f) Sound recovery results from a cellphone’s microphone.

Articles: Latest Articles; Most Read; Collections

Updates: Events; News; Multimedia

About: About Us

Contact

No. 86 Xueyuan South Road, Haidian District, Beijing

100081

010-62662699(Operation and Technical Support)

qkjq@cast.org.cn

Copyright © 2025 China Association for Science and Technology. All rights reserved. For all open access content, the relevant licensing terms apply.
Sponsored by the Office of the Leading Group for Cybersecurity and Informatization of CAST, and supported by Science and Technology Review Publishing House