# Fifth Lichuang Electronic Design Competition# Use MQTT to realize 4G shooting and DTU transparent transmission function modules for P2P intranet penetration

Note: * is required.

[Please fill in during the registration stage↓]   1. I am an electronic engineer and have been engaged in embedded development for 12 years, mainly for stm series and ESP series development and C# host computer development.   2. The main functions of this project are: 4G communication and 5 million pixels Photography, TF card storage, RS485 serial bus, MQTT communication, P2P communication. With the help of Alibaba Cloud IoT MQTT platform, connect the intranet host and 4G shooting module, and use the public network cloud to implement UDP-NAT port mapping, thereby realizing P2P hole-making intranet transparent transmission, and then the two can bypass the MQTT platform for image processing Direct transparent transmission of large files or DTU data streams. The core hardware is mainly built by GD32F103CB, CPLD, SDRAM, OV5640 image sensor and SIM7600CE full Netcom 4G module. The software is mainly: microcontroller embedded C, CPLD's VerilogHDL, and the host computer is written in C#. It is divided into a main control panel and a hole-punching program. The hole-punching program is deployed on the rented Tencent Cloud.

[Please fill in during the competition stage↓] * 1. Describe the project details (1) The MQTT protocol and its platform are currently the mainstream in the field of IoT applications and are used in many IoT products. However, it is usually only used in communication applications with small packets and short reports, and is not suitable for large files or data flows. On the other hand, most current image and video applications use Linux or Android platform solutions, such as HiSilicon or Raspberry Pi, which are high-cost and power-consuming, and have a certain entry barrier. At the same time, for the transmission of video images, the best thing is that the two parties of data exchange can be directly connected without the need for an intermediary server. In this project, I want to implement the MQTT+ image stream pass-through function in a low-cost, low-power, low-threshold way. (2) This project can be divided into MQTT communication and P2P communication according to the communication type. The first is MQTT communication. At present, MQTT-Broker service providers and many Internet companies can provide access services, and the cost is very low. I chose Alibaba Cloud's IoT platform as the service provider. Information and examples of running the MQTT protocol on a microcontroller can be found online. Since the microcontroller C program I developed used FreeRTOS as the skeleton, I also found good MQTT code from the FreeRTOS official website, and it was easy to transplant it to my microcontroller. MQTT is based on the TCP protocol. The 4G module SIM7600CE I used has a TCP/IP stack. Through a few AT commands on the serial port, you can easily and quickly establish a TCP connection and connect to Alibaba Cloud. As for the control interface program written in C# on the PC host, there is also a mature MQTT.NET.DLL library on the Internet for easy calling. So it took me about a week to set up the MQTT communication channel between the two. Note here that since I want to use a custom command format, I need to set the forwarding rules of the Alibaba Cloud console. When I want to forward, I convert the forwarded binary data into a string through base64 on the microcontroller or PC program, then encapsulate it in json format and send it to the MQTT server. Only then will the server parse it correctly and forward it for me. Next is P2P communication. P2P works on UDP, and its hole punching principle is based on routing cone NAT mapping. Therefore, I need a public network server as a mapping intermediary. The specific method is: write an echo program in C# and deploy it on the cloud with public IP. The echo program will always listen to the UDP port. If anyone queries the IP port number of the external NAT to which it belongs, the echo program will return it to it. The process of drilling holes is: when the internal network host learns that the 4G module has been connected to MQTT, it first queries the echo program for its external NATip port, and then sends it to the 4G module through MQTT. The 4G module uses this IP port to try to send a UDP packet to Host, if the gateway where the host is located is a cone NAT type, it will receive the packet from the 4G module; if it is a symmetric NAT type, it will not be received. At this time, the roles need to be swapped, so that 4G can first obtain its own NATIP port, and then pass MQTT is sent to the host, allowing the host to try to make a hole. In this way, as long as either of the two parties is a cone NAT, the hole can be successfully drilled. My own PC is a two-layer TPLink intranet. After testing, China Unicom's 4G card and China Mobile Home Broadband are cone NAT, and China Mobile's 4G card is symmetrical NAT. Others have not been tested. Once the hole is successfully drilled, heartbeats must be sent at regular intervals and the router's NAT list must be valid to keep the P2P channel open. The SIM7600CE also comes with a UDP protocol stack, which is convenient to use. (3) In summary, my terminal module and host program run two communication protocols respectively, among which MQTT is used for control messages and P2P is used for large file data transmission. (4) In terms of image shooting, since the OV5640 is a chip with a JPG engine, I only need to set its output format and receive and cache it in time. Since the output bus bandwidth of OV5640 is high, I additionally use the CPLD+SDRAM method to specifically cache JPG data, and then when the microcontroller needs image data, it reads the SDRAM cached data through CPLD sub-packaging. Therefore, CPLD plays the role of a transfer station, and its code is written in VerilogHDL. (5) From the video I provided, it can be seen that the image transmission rate under the P2P channel is about 40~50KB. A 5 million pixel JPG can usually be transferred in 10 seconds, and the logic of packet loss and retransmission is also added here. If you build an MJPG video stream in the future without considering packet loss and retransmission, it is estimated that it can reach 70KB. Then this speed should also be 7 frames/s for transmitting MJPG with a resolution of 320 and 240. Moreover, SIM7600 has its own GPS function. In the future, it can be used as an unmanned remote control car. It can be remotely controlled through map + real-time video, and can go anywhere as long as there is 4G.   2. Describe the challenges faced by the project and the problems solved. Since this project involves many technologies, even if some technologies have resource references, it will take a lot of time to integrate and debug. And it is a cross-platform and cross-network project, so it is difficult to develop, but also a lot of fun. For example, P2P transmission works on top of UDP, so the logic required to ensure data integrity when transmitting images must be defined and written by myself. Here I refer to TCP's window technology and packet loss retransmission technology, and use them in P2P transmission after being slightly simplified.   3. Describe the key points involved in the hardware and software parts of the project (1) In this project, I have customized two sets of protocols. One is a control message protocol encapsulated in the MQTT package, such as hole drilling and IO peripherals. control, serial port packet forwarding, shooting parameter settings, etc.; the other is a large file or streaming protocol specifically on P2P. (2) In terms of hardware, the CPLD+SDRAM image acquisition solution I developed myself is relatively original, and it can also be extended to other projects to solve the pain point of low-end microcontrollers being unable to acquire images. (3) The main energy and time of this project are spent on microcontroller code design and programming, and the FreeRTOS operating system I use is a very powerful tool and is open source and free.   4. Project material list display shooting module

4G module

* 5. Upload project pictures 5.1. Print the competition logo picture on the PCB, if not, it will be deemed as giving up the competition           5.2. Other pictures of the project PC console interface 4G module schematic diagram

Shooting module schematic diagram * 6. Demonstrate your project and record it as a video for upload 6.1. Official website of the video upload competition 6.2. Video title and link of Station B https://www.bilibili.com/video/BV1Jz4y1Z7BU/ The 5th Lichuang Electronics Design Competition - "4G Photography and DTU Transparent Transmission Function Module for P2P Penetration Using MQTT" - Functional Demonstration

< span class="colour" style="color:rgb(170, 170, 170)">< span class="size" style="font-size:12px">Notes: ①: Video requirements: Please horizontal screen Shoot, the resolution is no less than 1280×720, the format is Mp4/Mov, the size of a single video is limited to 100M; ②: Video upload: please upload to the official website of the competition and B station ( www.bilibili.com ) simultaneously, B station’s top 10 most popular projects You will receive a cash reward of 1,000-5,000 yuan, and other uploaded projects will receive a 100-yuan Lichuang Mall no-threshold coupon; ③: Video title: The 5th Lichuang Electronic Design Competition: {Project Name}-{Video Module Name}; such as the fifth 2018 Lichuang Electronic Design Competition: "Autonomous Driving" Project - Team Introduction. < /span>< /span>< br> < br>   7. It is the first public release. The first public release   8. Open source document schematic diagram, BOM * 9. References FreeRTOS official website CSDN official website.NET official website github official website GigaDevice official website " TCP/IP Principles" "MCU Principles" "Proficient in C#"

Click the link for more details: https://www.szlcsc.com/go/dsxq