gesture recognition device

1. Question requirements
Title: Gesture recognition device
1.1. Task:
       Design and produce a gesture recognition device based on TI's sensor chip FDC2214 to realize the judgment of guessing games and punch games. The device can also directly use the FDC2214EVM board, and it is required that no more than 2 FDC2214 chips or EVM boards be used. The device has two working modes: training and judgment. In the judgment mode, the experimental device can judge the guessing game and the punching game for designated persons. Here, the judgment of the guessing game refers to the judgment of the hand gestures "rock", "scissors" and "paper", and the judgment of the boxing game refers to the judgment of the hand gestures "1", "2", "3", "4" and "5". "judgment. In the training mode, any person can be trained on the gestures of the guessing game and the punching game. After a limited number of trainings, the correct gesture judgment of the guessing game and the punching game can be made.
1.2. Requirements
(1) The device works in the judgment mode, and can judge the guessing game for the designated person by the contestant, and give accurate judgments of the gestures "rock", "scissors" and "paper". It is required that the time of each judgment should not be more than 1 Second. (18 points)
(2) When the device works in the judgment mode, it can judge the punching of the designated person by the contestant, and give accurate judgments of the gestures "1", "2", "3", "4" and "5". Each decision is required to take no more than 1 second. (28 points)
(3) The device works in the training mode and performs guessing gesture training on any tester. The number of training times for each action is no more than 3 times, and the total training time is no more than 1 minute; then switch the working mode to the judgment mode. , the trained personnel are required to make guessing judgments, and each judgment time is required to be no more than 1 second. (21 points)
(4) The device works in the training mode, and performs fisting gesture training on any tester. The number of training times for each action is no more than 3 times, and the total training time is no more than 2 minutes; then switch the working mode to the judgment mode. , the trained personnel are required to make punch judgments, and each judgment time is required to be no more than 1 second. (29 points)
(5) Others. (4 points)
(6) Design report. (20 points)
Table 1 Design Plan Requirements
1.3, Explanation
(1) The "designated person" in the question is the person designated by the students of the participating team, and the "arbitrary tester" is the person temporarily selected by the review teacher.
(2) FDC2214 is a capacitance detection sensor based on the principle of LC resonant circuit. The basic principle is shown in Figure 1. An inductor and capacitor are connected to the input end of each detection channel of the chip to form an LC circuit. The measured capacitor sensing end (the gray marked part in Figure 1 is the measured capacitor) and the LC circuit When connected, an oscillation frequency will be generated, and the measured capacitance value can be calculated based on this frequency value.
Figure 1 Basic principle of FDC2214 sensor.
       The working principle of FDC2214 can be used to realize the function of gesture proximity and recognition. As shown in Figure 2, the yellow part is called the "sensing plane of FDC2214". This plane is made of conductive material. When the human hand approaches the conductor, it transmits When sensing the plane, the capacitance of the sensing end changes, which will lead to changes in the oscillation frequency of the LC circuit, thus reflecting the proximity of the gesture and the determination of the gesture.
       The following characteristics exist in experiments based on FDC2214 to achieve gesture proximity and judgment: the larger the area of ​​the sensing plane, the smaller the distance between the gesture and the sensing plane, the greater the change in the frequency of induction, the more sensitive the system will be, but at the same time, it may also be introduced The more noise there is. Therefore, when designing the sensing plane,
comprehensive considerations must be made based on the actual situation. For further design documents, please refer to the following link: http://www.ti.com/lit/an/snoa940a/snoa940a.pdf
Figure 2 Gesture sensing diagram
In order to facilitate training and judgment testing, it is recommended that student works can specify the test area. As shown in Figure 3. During testing or training, the tester's gestures are required to be close to the test board. It is recommended that the distance between the tester's gestures and the FDC2214 sensor of the work is not less than 1 cm.
Figure 3 Schematic diagram of testing the gesture recognition device
(3) The specific definitions of guessing game and punching game gestures should be in line with public perception. When conducting gesture training for any tester, the tester must follow the guidance of the student to conduct the training; after the training is completed, the gesture judgment test must be conducted under the guidance of the student.
 
 
2. Problem analysis
       The design and implementation of this gesture recognition device is mainly based on the basic principle of parallel plate capacitors, with the STM32 microcontroller as the core, and the copper plate capacitance value is read through the FDC2214 capacitive sensing chip. The data is analyzed and processed by the microcontroller, and the current gesture is judged through the initial threshold value. The device menu and analysis results are displayed on the TFTLCD screen, and the recognized gesture is announced by the JQ8400-FN voice playback module. At the same time, there is a training interface that can conduct gesture training for any person. After training, the gesture judgment can be made correctly to realize gesture recognition of different people and different hand types. On this basis, a man-machine battle system has been added, and any person can play a guessing game with the system after training. The time and number of times required for training and the time required for judgment of this device all meet the design requirements, and it has the advantages of simple operation and high accuracy.
        With the STM32 microcontroller as the core, the data of the copper plate capacitance value is read through the FDC2214 capacitive sensing chip. The data is analyzed and processed by the microcontroller and displayed through TFTLCD. The overall structural framework of the system is shown in Figure 4.
Figure 4 Overall structural framework of the system Figure
2.1. Processor selection
option 1: Select AT89C52 as the core component of the system to realize control and processing functions. AT89C52 is an 8-bit high-performance microcontroller. However, its programming is more complicated in implementing complex communication methods such as I2C. At the same time, it is unable to cope with the collection and processing of large amounts of data, so it is not suitable as the main control chip for this topic.
Option 2: Use the STM32F103 microcontroller as the microprocessor. Compared with other chips, it is more cost-effective. This microcontroller uses the mainstream Cortex-M3 core, with a working frequency of up to 72MHz, fast processing speed, rich peripherals, reasonable power consumption, and reasonable Price cost. In addition, this microcontroller has a high degree of integration, and it is relatively easy to develop software through library functions.
Option 3: Use FPGA as the device controller. FPGA can quickly collect data and perform signal processing, and can accurately perform real-time control, but its production cost is high and the production time is long.
After comprehensively considering many factors such as sensor communication method, computing processing speed, and production cost, it was decided to use the STM32F103 microcontroller as the control core of the microprocessor module to make full use of the resource advantages of the microcontroller and choose option two.
 
2.2. Power module selection
option 1: Use model aircraft batteries, which can be recharged repeatedly. The advantage of this method is that the gesture recognition device does not need to be connected to an external power supply, is more convenient to install, and has a long battery life. However, the voltage will fluctuate and it cannot work stably for a long time.
Solution 2: Use an AC-DC power adapter to output 5V voltage to power the microcontroller. Although the system installation is more complicated and requires external wiring, it has high power, high device integration, small size and stable voltage.
Option 3: Using QGP3200-3S power rechargeable lithium battery, no external wiring is required, and it is easy to install and does not require a step-down module.
Based on the above three options, we choose option three from the perspective of economy, convenience, stability and display.
 
2.3. Display module selection
Option 1: Use OLED screen. OLED has the advantages of small size and simple control, but this system has developed a menu interface and image display. There is a lot of content to be displayed, and the OLED display screen is too small to meet the usage requirements.
Option 2: Use TFTLCD screen. TFTLCD can display screen information at high speed, high brightness and high contrast. It has good usage characteristics: low-voltage application, low driving voltage, high safety and reliability in solid-state use; low power consumption, its energy consumption is only 1/10 of a CRT monitor. The monitor is beautiful and fully functional, and can meet the display requirements of this system.
Based on the above two options, considering the beauty and display of the work, choose option two.
 
2.4. Capacitor plate connection method
scheme 1: Use 4 copper plates to connect independently, and each plate leads to a wire to connect the 4 channels.
Option 2: Use a copper plate to read the signal from a single channel. The advantage is that there are no restrictions on the pressing and placing posture of the hand, but there are too similar data, which may cause misjudgment.
Solution 3: Use independent connection channels for the hand-shaped copper plate, the copper plate at the little finger position and the copper plate at the ring finger position. Since the contact area of ​​the little finger is smaller, gestures can be recognized more accurately.
After comprehensively considering the above three options, considering stability and accuracy, choose option two.
 
 
3. Schematic design description
3.1. The core
       microprocessor module of the microcontroller uses STM32F103 as the main controller. It uses the Corte-M3ARM core with a main frequency of up to 72MHz, up to 8 timers, rich interfaces, and up to 112 fast enhanced I/O port, has 9 communication channels, 2 I2C interfaces, 3 USART interfaces, 2 SPI interfaces, fast processing speed, rich peripherals, reasonable power consumption, and reasonable price and cost. In addition, this microcontroller has a high degree of integration, and it is relatively easy to develop software through library functions. Its core schematic diagram is shown in Figure 5.
 
Figure 5 STM32F103 core schematic diagram
 
3.2. Display circuit principle
      TFT refers to thin film transistor, that is, each liquid crystal pixel is driven by a thin film transistor integrated behind the pixel, so that high-speed, high-brightness, and high-contrast display can be achieved Screen information is one of the best LCD color display devices currently. Its circuit schematic is shown in Figure 6:
Figure 6 TFTLCD circuit schematic
 
3.3. FDC2214 capacitive sensor principle
        FDC2214 is an anti-noise and anti-noise sensor for capacitive sensing solutions. EMI, high-resolution, high-speed, multi-channel capacitive-to-digital converters. The device features an innovative narrowband-based architecture that provides high noise and interference rejection while delivering high resolution at high speeds. And it supports a wide excitation frequency range, which can bring flexibility to system design. The schematic diagram of the capacitive sensor composed of FDC2214 is shown in Figure 7.
Figure 7 Basic principles of FDC2214 sensor
3.4, JQ8400-FN voice chip principle
        JQ8400-FN voice chip supports MP3WAV hardware decoding, supports FAT file system, 24-bit DAC output internally uses DSP hardware decoding, non-PWM output, dynamic range supports 90dB, signal ratio 85dB, with multiple control modes, two-wire serial port mode, and one-wire serial port control. Supports SPIFLASH to be simulated as a USB flash drive, and directly updates the voice in SPIFLASH just like operating a USB flash drive. The schematic diagram of the voice playback module composed of JQ8400-FN chip is shown in Figure 8.
Figure 8 Basic principles of JQ8400-FN voice playback
4. PCB design instructions
PCB layout principles:
1. Arrange the position of each functional circuit unit according to the circuit flow, so that the layout facilitates signal circulation and keeps the signal in the same direction as possible.
2. Place the layout around the core components of each functional unit as the center. The components are arranged evenly, integrally and compactly on the PCB, minimizing and shortening the leads and connections between the components.
3. Under the premise of ensuring the electrical performance, the components are placed on the grid and arranged parallel or vertically to each other in order to be neat and beautiful. Under normal circumstances, components are not allowed to overlap; the components are arranged compactly and the components are evenly distributed on the entire layout. Consistent density.
4. The wiring design rules are: line width 20mil, spacing 6mil, hole outer diameter 24mil, hole inner diameter 12mil.
The circuit schematic diagram is shown in Figure 9, and the FDC2214 and surrounding circuit PCB are shown in Figures 10 and 11:
Figure 9 FDC2214 circuit schematic
 
Figure 10 FDC2214 surrounding circuit PCB (front)
 
 Figure 11 FDC2214 surrounding circuit PCB (reverse)
5. Software description
      Every time the system is started, the capacitance affected by the system's environment will be read in the initialization module, and the judgment threshold will be reset based on this value to eliminate the impact of the environment on the system and improve the system's anti-interference capability. The overall program design of this system is shown in Figure 12.
Figure 12 Overall programming flow
code block:
#include "led.h"
#include "delay.h"
#include "key.h"
#include "sys.h"
#include "lcd.h"
#include "usart.h "
#include "exti.h"
#include "bsp.h"
#include "jq8400.h"
 
#define TRAIN_NUM 5
#define AVG_NUM 20
#define TEST_NUM TRAIN_NUM
 
u8 con=1;//Working status flag 1 menu, 2 judgment, 3 training, 4 post-training judgment, 5 duel
u8 work=0;//Confirm button mark
u8 change=0;//Training selection switch
u8 change2=0;//Judgment selection switch
u8 change3=0;//Identification selection switch
int data0[5]; //Recognition threshold
u8 flag1=0;//Guessing gesture
u8 flag2=0;//Punching gesture
u8 value=0;//Computer punching gesture
void judge(float a, float b, float c ,float d,float e);
float temp0=0;
void Sample_Fun(float a);
void huaquan(void);
int workmode=0;
void huaquan(void);
int A0=0,A1=0,A2=0,A3=0;
int main(void)
{
delay_init();
NVIC_PriorityGroupConfig(NVIC_PriorityGroup_2);
USART2_Config();
uart_init(115200);
KEY_Init();
EXTIX_Init();
LCD_Init ();
IIC_Init();
InitMultiFDC2214();
voiceplus();
huanjing();
while(1)
{
huaquan(); //FDC2214 detection and identification
LCD_DISPLAY(); //TFTLCD display
}
}
void huaquan()
{
A0= FDC2X14ReadCH(1)/1000; //FDC2214 capacitance reading
A1= FDC2X14ReadCH(2)/1000;
A2= FDC2X14ReadCH(3)/1000;
A3= FDC2X14ReadCH(4)/1000;
if(A2>=data0[1]&&A2 <=data0[0])
{
flag1=1;
if(A1<=data0[2])
{
flag1=2;
if(A0<=data0[3])
flag1=3;
}
else if(A0<=data0[ 3])
flag1=2;
}
else if(A2<=data0[1])
{
flag1=2;
if(A1<=data0[2])
{
flag1=3;
if(A0<=data0[3])
flag1 =4;
}
else if(A0<=data0[3])
flag1=3;
 
if(A3<=data0[4])
flag1=5;
}
else if(A1<=data0[2])
{
flag1=1;
if(A0<=data0[3])
flag1=2;
}
else if(A0<=data0[3])
{
flag1=1;
}
else if(A3<=data0[4])
;
else
{
flag1=0;
sound=1;
}
 
 
if(A2>=data0[1]&&A2<=data0[0])
{
if(A1<=data0[2])
{
flag2=1;
}
else if(A0<=data0[3])
flag2=1;
}
else if(A2<=data0[1])
{
flag2=1;
if(A1<=data0[2])
{
flag2=3;
if(A0<=data0[3])
flag2=3;
}
else if(A0<=data0[3])
flag2=3;
}
else if(A3<=data0[4])
{
flag2=2;
}
else
{
flag2=0;
sound=1;
}
}
void huanjing()
{
A0= FDC2X14ReadCH(1)/1000;
A1= FDC2X14ReadCH(2)/1000;
A2= FDC2X14ReadCH(3)/1000;
A3= FDC2X14ReadCH(4)/1000;
data0[0]=A2-1500;
data0[1] =A2-6500;
data0[2]=A1-1500;
data0[3]=A0-1500;
data0[4]=A3-5000;
}
 
 
Physical display description
Figure 13 System block diagram
          is shown in Figure 13. This device It can be divided into 4 parts in total:






Area 1 is the copper plate part, which is directly connected to the four channels of the FDC2214 capacitive sensor module. In order to facilitate data collection and processing, we cut the copper plate into six pieces to accurately collect gestures and avoid recognition errors.
Area 2 is the FDC2214 capacitive sensor module, which communicates with STM32 through IIC, feeds back the identified capacitance value to the processor, and performs subsequent operations such as threshold comparison.
Area 3 is the voice broadcast module. The control core sends voice broadcast instructions to JQ8400-FN through the serial port, and the voice prompts the gesture status and the winning or losing relationship in the guessing mode.
Area 4 is the STM32F103 microcontroller, which serves as the control core of the microprocessor module. And the TFTLCD screen is used to display the system menu and judgment results to enhance the degree of visualization.






 
6. Precautions
When designing, you should pay attention to the impact of environmental changes on the capacitance reading value. In the initialization module of this design, the capacitance affected by the system's environment will be read, and the judgment threshold will be reset based on this value. Eliminate the impact of the environment on the system and improve the system's anti-interference ability.
 
7. Demo video and related code
Gesture recognition device based on STM32 and FDC2214_bilibili_bilibili
https://pan.baidu.com/s/1Ds68Q_7TBd-86sYmqVSudQ?pwd=6666