2021 D Topic - Internet-based camera measurement system + yyds team

Contents1
. Introduction2
. Team Introduction3
. Project Analysis4
. System Solution5 . Theoretical Analysis and
Calculation6
. Circuit Program and Design7
. Physical Display8
. Description of Competition Topic9
. SummaryIntroductionThis   design is based on the principle of image recognition and designs an Internet-based camera measurement system. The system uses Raspberry Pi as the core processing terminal, and realizes a star-shaped interconnection network through a 100M Ethernet switch and two camera nodes to complete the measurement and real-time display of the swing angle of the fishing line and the length of the fishing line. The system first uses the CMOS sensor module to realize the camera function; then the k210 chip is used to complete the recognition and positioning of the laser pen; then the pendulum-related formula is used to calculate the length of the fishing line and the offset angle of the laser pen, and finally the LCD is used to display the measurement results and sound and light alarm. After testing, it is proved that the absolute error of the system for measuring the length of the fishing line is less than 1cm, the absolute error of the laser swing angle measurement is less than 5°, and the measurement time is less than two seconds.  Keywords: Internet; angle measurement; length measurement; Raspberry Pi; k210 Team introduction The team members are Xiao Xiao Wu, Zhang Haohao, and Xiao Tu Tu. They are seniors in electronic information engineering and are good at image processing. They are proficient in using stm32, k210, openmv, Raspberry Pi, etc. They have done a number of scientific research projects and college student innovation and entrepreneurship projects during their time at school and have a number of software copyrights. Award-winning experience: (1) National Second Prize of the 2021 National College Student Electronic Design Competition (2) Provincial Third Prize of the 2020 Hubei Provincial Electronic Design Competition (3) Successful Participation Award of the 2019 National College Student Electronic Design Competition (4) Third Prize of the 13th China College Student Computer Design Competition Central and Southern Region Competition (5) Second Prize of the Fourth National College Student FPGA Innovation Design Competition Finals (6) National Second Prize of the 2021 National Electronic Design Competition "Internet-based Camera Measurement System" (7) National Third Prize of the 2021 Computer Design Competition (8) National Second Prize of the 2020 Blue Bridge Cup National Software and Information Technology Professional Talent Competition (9) 2021 National College Student Innovation and Entrepreneurship Project "Anti-epidemic Disinfection Drone Based on STM32" (10) Project Analysis of the 2021 National College Student Innovation and Entrepreneurship Annual Conference I. Topic Requirements 1. Basic Requirements (1) Design and make two independent camera nodes, each consisting of a camera and corresponding circuits. Both camera nodes can capture and display the motion video of the laser pen. (2) Design and make terminal nodes. The real-time videos captured by the two camera nodes can be displayed separately and simultaneously on the terminal display. The laser pen can be identified in the video, and the outline of the laser pen is framed in real time with a red box in the video. (3) The measurement system sets a one-button start at the terminal node. Starting from the swing of the laser pen, the measurement system automatically measures the length l by processing the video signal of the laser pen's periodic swing, 50cm≤ l ≤150cm, and the angle θ is self-determined. When the measurement is completed, the terminal will sound and light prompts and display the length l. The absolute value of the measurement error is required to be less than 2cm, and the measurement time is less than 30 seconds. 2. Development part (1) After one-button start, the measurement system works together through the network of two independent camera nodes. When θ=0° and θ=90°, it can automatically measure the length l, 50cm≤ l ≤150cm. The absolute value of the measurement error is required to be less than 2cm, and the measurement time is less than 30 seconds. (2) After one-button start, θ can be measured, 0°≤ θ ≤90°. The absolute value of the measurement error is required to be less than 5°. The measurement time is less than 30 seconds. (3) Others. Initial analysis and design ideas of the topic:        First of all, our team's initial idea is to need two cameras, a laser, a switch, and then a terminal display. Use 3D printing to print a shell for the laser machine to facilitate the addition of counterweight and fixation. Use a tripod to fix the two camera sensors. In terms of terminal display, we began to plan to use stm32f4 and w5500 to communicate with the terminal through spi or serial port to network port and display. The camera uses k210 or openmv, or raspberry pie. The terminal and the camera system communicate through tcp or udp through the switch. The sound alarm uses a speaker or buzzer, and the light alarm uses an LED light. This is the initial idea. System solution 1. Solution description        This system is a hardware platform built with raspberry pie as the terminal and K210 camera module as the camera node. It combines target monitoring, Ethernet transmission, and network collaboration to build a camera measurement system. The external display of the raspberry pie can display the real-time image processing video shot by two K210 separately and simultaneously. The Raspberry Pi sets a one-key start command, and the two K210s work together through the network to feed back the measured data to the Raspberry Pi. The Raspberry Pi processes the data to obtain the measured data, and the display shows the length and angle values. The system architecture diagram is shown in Figure 1.1. Figure 1 System Organization Diagram II. Comparison and Selection of Schemes 1. Camera Module Selection Scheme 1: The MaixBit development board uses K210 as the core unit, which is very powerful. The chip has a built-in 64-bit dual-core processor, 8M of SRAM, and built-in multiple hardware acceleration units (KPU, FPU, FFT, etc.), with a total computing power of up to 1TOPS. Scheme 2: OpenMV is based on the STM32F767CPU, integrates the OV7725 camera chip, and efficiently implements the core machine vision algorithm in C language, providing a Python programming interface. After comprehensive consideration, this design requires K210 to be able to detect targets and has a computing power much higher than OpenMv. Combined with the cost performance, Scheme 1 is selected. 2. Terminal selection Option 1: The STM32F103ZET6 development board is an ARM-based 32-bit microcontroller with high performance, low cost, low power consumption, innovative core and peripherals, 144 pins, 512K bytes of flash memory, and reads network port information and displays it on the screen. Option 2: Raspberry Pi is an ARM-based microcomputer motherboard with SD/MicroSD card as memory hard disk. It can connect keyboard, mouse and network cable, and has a TV output interface for video analog signal and HDMI high-definition video output interface. After comprehensive consideration, Raspberry Pi's performance in image data processing far exceeds that of stm32. The team members used Raspberry Pi 4B in the early stage and had a project foundation related to TCP transmission. Raspberry Pi has more development options, so they chose Option 2. 3. Bracket fixation option Option 1: Use a tripod to fix the two cameras for easy lifting and moving. Option 2: Use aluminum profiles to fix the camera position. After comprehensive consideration, since question D requires extremely high measurement accuracy, one or two pixels may result in a large deviation, so we chose option 2. We used aluminum profiles to fix the relative spatial position of the camera and laser, and used 3D printed brackets to fix the k210 to the aluminum profile. 4. Option 1 for network port module: Use the w5500 network port module with spi communication.
 
 
 


 

 
 



















                                           

 

 


                                                                                                               
 
 















Solution 2: Use USR-K7, a new and small serial-to-Ethernet module.
After comprehensive consideration, I plan to adopt Solution 2. USR-K7 is a device used to transmit UDP data packets and UART interfaces through data transmission. Equipped with ARM's Cortex-M4 processor, it has low power consumption, fast speed and high stability.
 
 
Theoretical analysis and calculation
 
1. Shooting method analysis
       Under natural lighting conditions, K210 is used to collect and process information. The Raspberry Pi sends synchronization instructions to K210, uses the camera module, configures the camera and captures the image, including initializing parameters, shooting images, determining whether the specified object is captured and reading the pixel position of the object, etc. The measurement results are transmitted from the serial port to the network port through a byte stream, and then transmitted to the Raspberry Pi through a switch.
 
2. Terminal display method analysis
       Raspberry Pi 4B receives the transmitted byte stream through the network port, parses it through the jpg parsing method, and finally displays the real-time picture of the camera on the host computer and selects the display with a red frame. The real-time video shot by the two camera nodes can be turned on and off on the display.
 
3. Network collaborative working principle
(1) The terminal issues a command: The terminal issues a one-button start command, and sends the command to the switch through the network port communication and UDP protocol. The switch sends the command to the two USR-K7s through the network port communication and TCP protocol. The USR-K7 transmits the command to the two camera nodes through the serial port communication, and the k210 starts working.
(2) The terminal receives data: The two K210s process the data transmitted by the camera and return it to their respective connected USR-K7s through the serial port. After receiving the data, the USR-K7 returns it to the switch through the network port communication and TCP protocol. The switch returns the data transmitted by the two camera nodes to the terminal through the network port communication and TCP protocol.
4. Analysis of the laser swing method
      We first stabilize the laser, keep it still, and do not spin. Use a thin tube to blow the middle of the laser. This can ensure that the blowing force is balanced and the laser motion trajectory can be close to a straight line instead of a circle. (This is a critical step)
 
III. Measurement method and parameter calculation
1. Transparent thin line length calculation
K210 uses model matching and uses a red box to frame and display in real time on the LCD screen to obtain the coordinates of the center point of the red box. When the laser pen is in a stationary state or a moving state, the vertical coordinate of the center point obtained at the lowest point has a linear relationship with the length of the thin line:
                              
where is the actual height range, is the maximum value of the vertical coordinate, and y2 is the minimum value of the vertical coordinate.
2. Angle calculation
During the periodic motion of the laser pen, camera A can obtain the range of change of the coordinates of the center point recognized by the laser pen. Similarly, camera B can obtain the range of change. The angle satisfies:
 
Circuit program and design
I. Hardware framework
This system uses k210 as the camera node for image processing. The image data and measurement data are connected to the switch through the serial port to network port module. The Raspberry Pi processes the data of the two camera nodes through the switch network port to complete the motion display and trajectory measurement of the laser pen. As shown in Figure 1.
                                                                                                    Figure 1 System structure diagram
 
Main module description:
Raspberry Pi 4B: We are equipped with Ubuntu 18.04, and a host computer is written through pycharm. Please see the attachment for the pin diagram of Raspberry Pi 4B. We use the above GPIO10 to turn on and off the LED and the buzzer alarm, and call the pygame library for voice broadcast. Finally, the host computer program is run by setting the sh script to start the computer.
K210: See the attachment for the schematic diagram of k210. We use maixpy ​​to write in python language. Because k210 is compatible with the graphics library of openmv, we call it for image processing. Although k210 can be written in c language through the exploration ide, because we are more proficient in using micropython development, I chose the latter to develop k210.
Serial port to network port: Because we are very familiar with the use of serial ports, there have been various projects that use serial ports, as well as network communication projects, so we chose the serial port to network port module instead of the w5500 module based on spi communication. We will try it in the future if we have the opportunity, after all, spi is faster.
3D printing is convenient for fixing k210 and preventing the laser from spinning. The 3D printing file will also be placed in the attachment.
2. Program design
 
       The two camera nodes perform real-time target detection, real-time detection and identification of laser pens, and continuous analysis of jpg format image data. The parsed data is output to the serial port, and connected to the network port of the Raspberry Pi through the switch via the serial port to the network port. The Raspberry Pi configures the local IP address and the serial port to network port module to be in the same network segment. The Raspberry Pi connects the IP addresses of the two network port modules to see if a connection can be established. After the Raspberry Pi establishes a connection with the two network port modules, it opens two threads to process the data transmitted by the two network port TCPs respectively, and verifies the transmitted frame header and frame tail data to obtain the measurement data and image data. The Raspberry Pi uses pygame for UI design and image display to realize the separate and simultaneous display of real-time videos shot by the two camera nodes. The interface has a one-button start function. The Raspberry Pi sends measurement instructions to the two network port modules through UDP at the same time. The network port modules convert the content in the UDP receiving buffer area into serial port data. After receiving the serial port data, K210 performs verification and comparison. If it meets the receiving frame format, it sends the measurement data to the Raspberry Pi for data processing and display.
                                                                                                                        
Figure 1 Program flow chart
 
 
Physical display
 
 
                                                                                                                        
 
 
                                                                                                                        
 
 
                                                                                                                         
 
 
                                                                                                                       
                                                                                                                          
 
Competition work description
(1) It is recommended to use a single strand of transparent fishing line with a diameter less than 0.2mm for the flexible transparent thin wire of the pendulum. Do not use general twisted sewing thread to prevent the laser pendulum from rotating after being lifted. Considering the difference between the actual pendulum and the ideal pendulum and the difference in gravity acceleration in different places, the system should have a calibration processing function.
(2) The system must obtain the information of the pendulum from the video information captured by the camera node. No other sensors or additional devices shall be installed on the pendulum and its vicinity. The calibration of the θ angle can be achieved by using a protractor to measure the angle between the laser indicator light spot trajectory and the OA edge.
(3) The viewing range of the two camera nodes is limited to the content of the laser pendulum swing interval, and cannot include the content of the entire flexible thin wire and the content of the ground laser light spot trajectory. During the measurement of l and θ, if the video contains the above content, this part of the content must be covered with paper. Otherwise, no test will be performed.
(4) The shooting background is a general laboratory scene, and the background objects can be stationary. No additional processing is required.
(5) Desktop computers and laptops must not be used for the three nodes.
Our work uses aluminum profiles to fix the relative positions of the three nodes. The two placement heights of the k210 board need to ensure that the fishing line 50cm to 150 can see the image and frame it on the k210 display.
Because the video is too large, greater than 50m, and may still be under review, the video link is: https://www.bilibili.com/video/BV1Gi4y197xy/
 

Summary
My teammates and I participated in the electronic competition for three years. Except for the summer vacation during the epidemic in my sophomore year, I spent every summer vacation at school to prepare for the electronic competition. We experienced short sleeves to down jackets, down jackets to short sleeves. We participated in three electronic competitions. From being ignorant at that time to being able to carry the banner now, after three years of electronic competition, we have gained a lot and exercised our own mentality and team writing ability. A successful team is inseparable from the hard work of every member of the team. From the beginning of my freshman year, I couldn't write stm32 code well, and I would be constantly complained about soldering boards. Now I have the choice to choose development Board, writing algorithms, our team is growing step by step. The fact that we were able to get the second place in the national electronic competition this year is also inseparable from the teacher who always took us to do projects, the experience accumulated in the early stage, continuous trial and error, and continuous improvement of our abilities. The team members are also getting better and better. I still remember the three of us staying up late to work hard in the electronic competition, the frustration of not being able to adjust the code, the joy of realizing the function, and the debate on improvement plans. Each of them is something worth recollecting in the future. Now the three of us are almost finished with our senior year, and each of us will work hard in our own direction. In the future, we will get better and better, compete on a higher platform, and hone ourselves.