Question d, Internet-based camera measurement system

### 1. Project Introduction

This project is based on Question D of the 2021 College Student Electronic Design Competition to implement camera measurement of relevant parameters of the pendulum model. The hardware part of this question consists of a camera, Raspberry Pi, Gigabit Ethernet switch and terminal TX2. The camera and Raspberry Pi form two camera nodes. TX2 is the terminal display node. The camera node and the terminal node form a LAN through the switch. Implement network transmission. The terminal node obtains the video information of the camera node through the LAN, and implements corresponding functions through signal processing and detection.

- - -

### 2. Project requirements

**Internet-based camera measurement system (Question D)**

**[Undergraduate Group]**

The first task is

to design and produce an Internet-based camera measurement system. The system structure is shown in Figure 1. In the figure, the three vertices of a square area with a side length of 1 meter are A, B and O respectively. The system has two independent camera nodes, placed at A and B respectively. The two camera nodes should shoot as orthogonally as possible along the AO and BO directions, and realize network interconnection through a 100M/Gigabit Ethernet switch and a terminal node connected to the switch. The switch must be a universal switch for the Internet, and the network port used can be specified arbitrarily. A laser pointer suspended from a flexible transparent thin line is suspended above point O. The length of the transparent thin line is l. The laser pointer is always on and indicates downward, and the indicator light point coincides with the O point when it is still and drooping. Pull the laser pointer away from the resting point less than 10cm. When released, the laser pointer swings freely. It should be ensured that the trajectory of the laser pointer's indicated light point moves back and forth in a straight line through point O. The angle between the trajectory and the OA side is θ. Use this system to measure the length l and angle θ.
![image.png]
Second requirements

1. Basic requirements
(1) Design and produce two independent camera nodes, each node consists of a camera and corresponding circuit. Both camera nodes can capture the movement video of the laser pointer and display it.
(2) Design and make terminal nodes. The real-time video captured by the two camera nodes can be displayed separately and simultaneously on the terminal display. The laser pointer can be identified in the video, and its outline is framed in real time by a red box in the video.
(3) The measurement system can be started with one click at the terminal node. Starting from the swing of the laser pointer, the measurement system automatically measures the length l by processing the periodic swing video signal of the laser pointer, 50cm ≤ l ≤ 150cm, and the θ angle is self-determined. When the measurement is completed, the terminal will sound and light prompts and displays the length l. The absolute value of the measurement error is required to be less than 2cm and the measurement time is less than 30 seconds.
2. Function part
(1) After one-click startup, the measurement system works together through the network of two independent camera nodes. When θ=0° and θ=90°, it can automatically measure the length l, 50cm ≤ l ≤150cm. The absolute value of the measurement error is required to be less than 2cm and the measurement time is less than 30 seconds.
(2) After one-click startup, θ can be measured, 0°≤ θ ≤90°. The absolute value of the measurement error is required to be less than 5°. The measurement time is less than 30 seconds.
(3) Others.

Three instructions

(1) It is recommended to use single-strand transparent fishing line for the flexible transparent thin line of the pendulum, with a diameter of less than 0.2mm. Do not use ordinary twisted sewing thread to prevent the laser pointer from rotating after being lifted. Considering the difference between the actual pendulum and the ideal pendulum and the differences in gravity acceleration in different places, the system should have the function of calibration processing.
(2) The information the system obtains about the pendulum must come from the video information captured by the camera node, and other sensors and additional devices are not allowed to be installed on or near the pendulum. The calibration of the θ angle can be achieved by using a protractor to measure the angle between the laser pointer trajectory and the OA side.
(3) The viewing range captured by the two camera nodes is limited to the swing range of the laser pointer, and cannot include the content of all flexible thin lines and the content of the laser point trajectory on the ground. During the measurement of l and θ, if the video contains the above content, use a piece of paper to cover this part. Otherwise the test is not performed.
(4) The shooting background is a general laboratory scene, and the background objects can be stationary, and no additional processing is required.
(5) Desktop computers and laptops are not allowed for the three nodes.

- - -

### 3. Project function introduction

* Network video data is transmitted in RTSP streaming format, without special emphasis on time synchronization, and has high real-time performance;
* Calculate relevant parameters based on the pendulum physical model, and use the frame difference method for moving target detection. Reduce the amount of calculations and improve program operation efficiency, and the calculation results are highly accurate;
* This project can quickly and accurately calculate the rope length and angle of the laser pointer when it is pendulum moving;
* It has a sound and light prompt function, and the operation interface has a measurement status mark, and the measurement After completion, the system will issue a sound prompt;
* It has the real-time tracking function of the laser pointer, and the laser pointer can be circled in a red frame in real time;
* Both the camera node and the terminal node can display the video in real time, and the terminal display module can selectively display the video taken by different camera nodes. ;
* The project has strong adaptability and can realize laser pointer detection and measurement under complex backgrounds.

- - -

### 4. Project plan

1. Calculation of length and angle.
Use the period formula T= 2π√(l/g) of the ideal pendulum motion model. The rope length l can be measured by measuring the swing period. The swing period passes Time domain zero-crossing detection.
Angle detection is derived using the tangent formula.
The measurement process is as follows:
![image.png] 2. The laser pointer detection
is implemented using the frame difference method. The video sequence collected by the camera is continuous. If there are no moving targets in the scene, the changes in consecutive frames will be very weak. If there are moving targets, there will be obvious changes between consecutive frames. Based on the above idea, since the target in the scene is moving, the position of the target image in different image frames is different. The frame difference method performs a difference operation on two or three consecutive frames of images in time, and the pixel points corresponding to different frames are subtracted. , determine the absolute value of the grayscale difference. When the absolute value exceeds a certain threshold, it can be judged as a moving target, thereby realizing the target detection function. The schematic diagram of the frame difference method is as follows:
![image.png] 3. Video network transmission
Video network transmission adopts RTSP stream format. The process of RTSP streaming is as follows:
![image.png]
This project uses FFMPEG on the Raspberry Pi for RTSP streaming, and accesses the link in the main control module to obtain video data.

- - -

### 5. System hardware

1. Overall system block diagram
The overall system block diagram is shown below, which mainly includes the main control module Raspberry Pi 4B, USB camera, Ethernet switch and display.
![image.png] 2. Terminal node and camera node
The terminal node uses JetSon TX2 as the main control, receives the video stream information of the camera node on the Ethernet physical medium, and automatically measures the length by processing the periodic swing video signal of the laser pointer. l and angle θ.
The camera node is controlled through Raspberry Pi 4B. The USB camera is connected to Raspberry Pi 4B. The monitor is connected to Raspberry Pi 4B to display real-time video. At the same time, Raspberry Pi 4B performs RTSP streaming to realize video data transmission.

- - -

### 6. Project properties

* [x] Functional test
* [x] Program test

- - -

### 7.

See attachment for software.

- - -

### 8.

The link to the demonstration video is as follows:
[https://www.bilibili.com/video/BV1HU4y1K7c5/](https://www.bilibili.com/video/BV1HU4y1K7c5/)

- - -

# ## 9. Module

1. Main control module NVIDIA® Jetson™ TX2+ display [https://www.nvidia.cn/autonomous-machines/embedded-systems/jetson-tx2/](https://www.nvidia. cn/autonomous-machines/embedded-systems/jetson-tx2/) (I used this as a course design device and got it directly. If you don’t have this, you can use a Raspberry Pi instead, but the frame rate may not reach 30 frames)
2. Camera point Raspberry Pi 4B + display + 640*480 USB camera [https://m.tb.cn/h.finmOUn?sm=5657d5](https://m.tb.cn/h.finmOUn ?sm=5657d5)
3. Gigabit switch*1
4. Several network cables + USB cables

### 10. Notes

1. Affected by physical factors such as the rotation of the laser pointer, the laser pointer should be as close to the pendulum as possible when releasing it Movement, the light spot basically remains straight.
2. Note that the IP address of the Raspberry Pi and the master node must correspond to each other~~~~