Tip / Sign in to post questions, reply, level up, and achieve exciting badges. Know more

cross mob

Machine Learning: PSoC™6 and XENSIV™ Radar Machine Learning-based Human Detection, Identification, and Vital Sign Monitoring

Infineon_Team
Employee
Employee
50 replies posted 25 likes received 25 replies posted

PSoC™6 and XENSIV™ radar solution demonstrations for object detection, identification, health monitoring, and more with privacy protection

human-detection-in-computer-vision-p-800.jpg

Components

Hardware

Name Quantity
XENSIV™ KIT CSK BGT60TR13C 1 Buy Online
CYSBSYSKIT-DEV-01 1 Buy Online

Software

Name Quantity
Infineon ModusToolbox™ Software 1 Download

Description

We showcase demonstrations of human presence detection in arbitrary shaped areas, individual identification, and heartbeat and breath monitoring using PSoC™ 62 module and 60GHz radar on XENSIV™ KIT CSK BGT60TR13C. We deployed Quantum Core’s edge machine learning library “Qore” and their signal processing library on PSoC™ 62 to accomplish these demonstrations. Due to the system’s compactness, and lower installation cost, we expect this system can substitute or cooperate with many camera-based systems in various situations such as construction sites, retail stores, and smart homes. Moreover, this radar-based system can be installed in locations with high privacy requirements, offering a clear advantage over camera-based solutions.

Quantum Core is an associated partner of Infineon, specializing in offering specialized software and solutions in machine learning.

Figure 1: XENSIV™ KIT CSK BGT60TR13C

Infineon_Team_9-1713839395981.png

 

Camera and machine learning-based solutions are widespread in society. Although they are very effective in many use cases, the scale of these system tends to be large, including network connections and servers. The required computational resources are usually demanding for edge computing, and developers must put significant effort into securing privacy.

In this article, we showcase demonstrations of human presence detection in arbitrary areas, individual identification, and heartbeat and breath monitoring using PSoC™ 62 and 60GHz radar of XENSIV™ KIT CSK BGT60TR13C

We deployed Quantum Core’s edge machine learning library “Qore” and their signal processing library on PSoC™ 62 to accomplish these demonstrations. These libraries are optimized to run on MCUs and can operate from model creation to prediction, even with systems having limited computational resources, with a minimum of 64 KB RAM. By collecting reflected radar wave data and correlating the data to each result status through signal processing and supervised learning on PSoC™ 62, we can predict “what is occurring in front of the radar” to achieve the demonstrations described above.

 In the following sections, we firstly explain the concepts of each demonstration. Then we cover the system architecture for both hardware and software. Finally, we describe the details of the demonstrations with videos showing actual deployment.

Demo 1: Human Presence Detection

 

In this demonstration, we draw an arbitrarily shaped closed boundary on the floor and detect whether someone is inside or outside of the boundary. The maximum distance is 6 m. One of the use cases is at construction sites, where we can designate a no-entry area in any shape and send alerts for prohibited entry. The system can be relatively small and easily relocated to adapt to the dynamical site situation. Combining this with human identification also enables entry surveillance up to authority.

Figure 2: Entry detection. The display shows "IN" when a person enters the designated area

Infineon_Team_7-1713839323084.png

 

Demo 2: Human Identification

 

In this demonstration, we identify the individual standing in front of the radar. One of the major differences from camera-based identification is that we can identify the person from any direction, even from the back, by learning data only from the front. The maximum distance is 6 m, and identification up to 5 people is currently possible in this demonstration.

 One of the use cases is in bathrooms, toilets, bedrooms, or other places with high privacy requirements. This demonstration can identify all members of a small family, and radar’s limited resolution is suitable to work as a substitution for cameras in these areas of smart homes. The lower cost compared to cameras enables a larger number of installations to track people in public facilities or retail stores and can be used in cooperation with cameras.

Figure 3: Human identification. Different individuals are identified.

Infineon_Team_6-1713839253294.png

 

Infineon_Team_5-1713839230665.png

 

 

Demo 3: Vital Sign Monitoring

 

In this demonstration, we monitor the heartbeat and breath, not only their rate but also their waveforms. This demo is also capable of simultaneously monitoring multi-people. The maximum distance is 6 m.

One of the use cases is health monitoring of elderly people or people at risk in their homes or care facilities. Additionally, this system can be installed in white goods, such as air-conditioners, at lower cost compared to cameras. It can analyze people’s status to adjust the operations of these appliances in smart homes.

Figure 4: Heartbeat and breath waveform monitoring. HR and BR represent heartbeat and breath rate per minute.

 

Infineon_Team_4-1713839189767.png

 

 

Hardware description: XENSIV™ KIT CSK BGT60TR13C

 

All of the demonstrations described above are realized with XENSIV™ KIT CSK BGT60TR13C without any modifications. This is a ready-to-go kit for multiple applications, with PSoC™ 62 MCU, BGT60TR13C radar, Wi-Fi/Bluetooth module, and DPS368 pressure sensor installed in a thumb-sized small form factor. In these demonstrations, we used only PSoC™ 62 and radar on this kit, but combining with other modules will open up more possibilities.

 BGT60TR13C radar sensor chip is the core of our demonstrations. This chip can operate as a 60 GHz radar transmitter and receiver by itself with 1 RX and 3 TX integrated antennas. The resolution is down to ~ 3cm, which is sufficient to realize not only detection of human presence but also individual identification. The power consumption is as low as 5 mW (duty cycle), which is helpful for edge IoT implementation. These characteristics make our radar solutions more reasonable substitutes to camera solutions in some use cases.

PSoC™ 62 plays crucial roles in our demonstrations. In this kit, PSoC™ 62 is implemented on CYSBSYSKIT-DEV-01. This module offers 1MB SRAM and 2MB application flash, which are sufficient to run applications with complex calculations, including machine learning applications using “Qore” library. In the current demo set-up, Infineon and Quantum Core engineers have worked together to program the “Qore” software library having the machine learning algorithms onto the PSoC™ 62 module in the c (any customer interested in leveraging this in their designs can contact Infineon or Quantum Core contacts). This module is also supported by ModusToolbox™, which enabled fast prototyping of the demo developments. Although not used in the demonstrations, Wi-Fi and Bluetooth support of the module and Arm Cortex-M4 and M0+ dual core feature of PSoC™ 62 ensure extensibility to create more complex system deployed in the real world.

Figure 5: XENSIV™ 60GHz radar sensor.   

Infineon_Team_3-1713839146445.png

Figure 6: CYSBSYSKIT-DEV-01 module. PSoC™ 62 is implemented on CYSBSYS-RP01 module.

Infineon_Team_2-1713839096706.png

 

Software description: Machine learning library “Qore”

 
Although most of the interface part were made with ModusToolbox™, we used machine learning library “Qore” for the most crucial point of the software.

“Qore” - uses a technology called reservoir computing. In this technology, signals are input to a non-linear, dynamical system called a reservoir (or an equivalent system reproduced on software) and mapped to a higher-dimensional space. The mapped signals are then input to a simple machine learning algorithm, such as Ridge regression, to obtain predictions. It is known that with this scheme, machine learning problems with time series data can be solved with great accuracy at a much lower cost compared to methods using deep learning because the only part requiring learning is the simple machine learning part. In “Qore”, Quantum Core’s original signal processing method, optimized for this reservoir computing, is also implemented before inputting signals to reservoir to enhance the accuracy of learning and predictions. With these technologies, “Qore” can operate from model creation to prediction, even with 64 KB RAM systems, and has shown various achievements in edge machine learning for sounds, mechanical vibrations, electric signals, radar signals, and more.

Figure 7: The concept of reservoir computing in “Qore”. Multi-dimentional time series data is input to the signal processing part and then input to the reservoir, implemented as an echo state network, and mapped to higher dimensions. Then, a simple machine learning algorithm, such as Ridge regression, is applied to the output to obtain prediction data.

Infineon_Team_1-1713839030020.png

 

DemoNSTRATION Details

 

Demo 1: Human Presence Detection


We first collect reflected waveform data of situations where someone is inside and outside the boundary. A person must walk along the boundary while staying barely inside the boundary for 30 seconds, then walk barely outside for 30 seconds. This data is then input into “Qore” library and a model is created within a few minutes. After the model is created, the kit can detect whether someone is inside or outside the boundary on a real-time basis.

Figure 8: Demonstration video of entry detection outside. When using this Kit in actual operation, it is necessary to comply with the radio-related laws and certification systems of the country or region where this Kit is being used.

Source: YouTube: Entry Detection Outside(SUB-Eng). Publisher: QuantumCoreSource: YouTube: Entry Detection Outside(SUB-Eng). Publisher: QuantumCore

 

Demo 2: Human Identification

 

We first collect reflected radar waveforms of people standing in front of the radar for 10 seconds per person. This data is then input into “Qore” library, and a model is created within a few minutes. After the model is created, the kit can identify who is standing in front of the radar on a real-time basis.

Figure 9: Demonstration movie of individual identification inside. When using this Kit in actual operation, it is necessary to comply with the radio-related laws and certification systems of the country or region where this Kit is being used.

Source: YouTube: Human Identification Demo powered by Infinion's 60GHz Radar - QuantumCore. Publisher: QuantumCoreSource: YouTube: Human Identification Demo powered by Infinion's 60GHz Radar - QuantumCore. Publishe...

 

Demo 3: Vital Sign Monitoring

 

We achieve this demo through two approaches. Our signal processing algorithm enables heartbeat waveform monitoring up to 4 m, and breath information up to 0.8 m. To monitor at a further distance, we use “Qore” library to predict heartbeat and breath waveforms. First, we collect radar waveforms and the heartbeat and breath waveforms obtained by the signal processing algorithm at a close distance. Then, we then input these results into “Qore” library to create a model. With this model, we can monitor heartbeat and breath waveforms at a maximum distance of 6 m. This demonstration is also capable of simultaneous heartbeat monitoring for up to 4 people.

Figure 10: Demonstration movie of heartbeat and breath waveform monitoring outside. HR and BR represent heartbeat and breath rate per minute. When using this Kit in actual operation, it is necessary to comply with the radio-related laws and certification systems of the country or region where this Kit is being used.

Source: YouTube. Vital Sign Monitoring Outside(SUB-Eng). Publisher: QuantumCoreSource: YouTube. Vital Sign Monitoring Outside(SUB-Eng). Publisher: QuantumCore

Figure 11: Monitor screen of heartbeat and breath waveform. HR and BR represent heartbeat and breath rate per minute.

Infineon_Team_0-1713838882935.png

Contributors

Community Manager

490 Views
Project Analytics
  • Views: 490
  • Comments: 0
  • Likes: 3