AI+IOT dormitory-free renovation based on ESP32

Winning works of Guangzhou Six School League

Team: Qiu Biao and Zhen Pair

Title: See attachment

Awards: Enterprise Special Award

Overview:

① High-power desktop lighting that can be controlled by face recognition, APP control, front desk control, and can independently adjust color temperature and brightness.

② The computer screen can be mapped in real time at a high frame rate to a colorful light strip with a strong background and adjustable saturation and brightness. Since it is transmitted through the campus network, real-time mapping of the screens of computers on other campuses can be achieved.

③The dormitory lights can be controlled with a cheap, high-stability solution without affecting the appearance, violating the safety of electricity in the dormitory, or affecting the original switch function.

④ All controls are through the campus network, allowing all three campuses and five campuses to efficiently control the system in the dormitory.

⑤Cheap but high-performance sound system, together with colorful light strips, creates a dormitory viewing mode.

⑥Vivid front-end interface and automatic time adjustment.

Foreword:

I've always had the idea of ​​renovating my dormitory. Currently, there are many videos of others renovating dormitories on bilibili, but most of them cause some degree of damage to the appearance of the dormitory or have other problems. For example, in many videos of changing ordinary door locks in dormitories to fingerprint or face recognition locks, the author usually sticks a development board directly on the door, and coupled with the messy wiring, it really affects the beauty of the dormitory. Taking advantage of the Guangzhou Six-School League sponsored by Jialichuang, I realized some of my ideas and made a non-sensory dormitory smart home control system. The entire system is basically invisible in normal dormitory life, which means it will not affect the beauty of the dormitory. Currently, this system is still under development, and all current parts of the program will be open sourced first. In the future, ESP32 face recognition will be added to open the dormitory door (there is already a perfect solution that does not affect the appearance, but I have never had the time), desktop projection and projection touch as the central control (all devices can be basically invisible at normal angles). When you arrive, you have no sense, you can only see the shadow cast). During the summer vacation, the entire project will be upgraded, overhauled, and sorted out (currently the procedures for each part are very confusing, because it is just for verification). After all is completed, all the data will be open source. Cloning is not recommended at the moment.

The following is the original text of the design report (slightly lazy):

2. Topic selection

B Electronic design of smart home control system.

3. Abstract

The full name of the system designed by this group is " Intelligent Networked Home Desktop Control System Based on Cheap ESP32 ". The design principles of this system are low cost, high quality, non-inductive, silky smooth, low power consumption and high reliability throughout the entire design process. At present, the system has been initially completed and has been put into use for two days, adding a lot of luster to dormitory life during the epidemic. The system's keen face detection capabilities and robust networking capabilities jointly create an excellent user experience.

4. Detailed description of design plan

  The main hardware of this system is as follows:  

The layout of the background of this system (including light strips, control panels, power supplies, and speakers) is all hidden, that is, the existence of the system cannot be felt when the power is off (it does not occupy the desktop and does not reveal its flaws). ), that is, the “no feeling” in the design purpose . Part of the hidden layout is as shown below:

The background control panel is as follows, using the board previously designed for DIY projectors

The front desk is a movable screen, which can be placed on the desktop to check the time ( NTP automatic time adjustment) and control the device. The views from all directions are as follows (the shell will be 3D printed later):

The actual overall effect is as follows (only the camera and front-end central control can be seen, and the camera will be switched to the front-end later):

( 1) The hardware selection and design process was slightly bumpy, and the records are as follows:

① Main control: Due to well-known reasons, the prices of imported processors such as STM32 have been artificially high recently. There are many domestic SOCs with low prices and strong performance (such as the Allwinner F1C series). However, since the SDK is not public, if you want to obtain it, you need to place a large order and sign an SDK confidentiality agreement, resulting in very little information. In order to balance cost performance and ease of use, I finally chose the ESP32 series chip as the front-end (central control screen) main control. The price of this series of bare chips is only about 10 yuan, and its module with minimal circuit and SPI Flash only costs 10 yuan. It costs about 15 yuan. I chose the ESP32-S3 series chip as the back-end main control. This series of modules only costs about 30 yuan and comes with hardware tensor operation acceleration, so it is suitable for AI model reasoning. I have also achieved extremely effective results for this purpose. Best face detection lights up. The above two main control chips both support full-protocol Bluetooth and WIFI, and are developed through Espressif's official ESP-IDF tool chain.

②Desk lamp driver: To realize a desk lighting with adjustable color temperature, I use a yellow light strip and a white light strip. Through PWM, it is expected that the color temperature and brightness can be adjusted by adjusting the duty cycle of the two. Since the scientific community’s experiments have shown that the lighting frequency requirement is greater than 3KHz (only at this time can we ensure that stroboscopic light is basically harmless to the eyes), I used 9KHz PWM to modulate the light strip. In addition, the power of the light strips I chose is 13W for a single strip and 26W for two strips in total, so I need to find a driving circuit that meets high power and high frequency. Originally, I wanted to solve the problem by purchasing ready-made driver modules, but this would increase the cost and make it inconvenient to compress the volume. Finally, through experiments, I found that each light strip only needs to be driven by one MOS tube model SI2302, and it can be driven perfectly without generating any heat. Considering that the MOS tube is prone to breakdown, it was originally planned to use optocoupler isolation. However, in order to reduce costs and take into account the size, a diode was connected in series between the main control IO port and the MOS tube gate for simple isolation to prevent current backflow. After long-term lighting experiments, I found that this solution does not bring any instability.

③: Networking solution

Since the official ESP-IDF development kit is currently not adapted to enterprise-level 802.1x certification and cannot be directly connected to the campus network, I provide a network for this system through a router connected to the campus network.

④: Dormitory light control scheme

Under the premise of not violating the safety of electricity in the dormitory, we want to control the dormitory lights by pressing the switch with a stepper motor. However, this will cause the original switch to be occupied and cannot be controlled manually. In addition, it will damage the beauty of the dormitory. I learned that there is a button cover that can be placed directly on ordinary keys. It can control the switch remotely or manually control the switch by touching its surface. However, this type of switch is expensive, with the cheapest one costing about 100. At the same time, I cannot obtain its communication protocol, so I cannot control it with the cheap system I built. Finally, I found ROOME, a brand that had gone out of business. The smart switch sets they produced needed the software they provided to control them, but the server was down and therefore unusable. Therefore, they only sold for 12 yuan on Taobao. At the same time, netizens contacted former employees of this company, obtained the protocol for its BLE control switch and shared it in the Taobao comment area, so it became my best choice. It is beautiful, stable and low-priced. Its installation effect in the dormitory is as follows:

⑤：Audio

In order to create a dormitory movie viewing mode, in addition to light strips, speakers are also needed. I chose Baofengyingyin TV speakers to make a small sound system. Since Baofeng Video has gone bankrupt, many high-quality brand-new speakers have been sold at low prices. I spent 18 yuan to buy a pair of speakers with a total power of 20W. I hid them in the cracks of my desk and paired them with a 10-yuan transformer-coupled amplifier board to achieve a set of High quality sound system. Currently it can only be connected via Bluetooth, and will be added to the background main control chip in the future.

(2) Software design

  The development environment of this system involves: ESP-IDF, VS2017, QT5, E4A

① Desk lamp brightness adjustment:
In line with the purpose of silky smoothness, I used a new Freertos thread to achieve that the lamp strip always maintains silky smoothness (gradient) during the process of turning on, off, adjusting brightness, and adjusting color temperature, that is, no matter what The brightness and color temperature of the light strip are not likely to change suddenly due to violent adjustments or any adverse conditions. (See the demonstration or attached video for details) There is a detail here. Since the color temperature is adjusted by adjusting the brightness relationship between the yellow and white lights, when the color temperature is the lowest or the highest, only one of the two lights should be on and the other off, that is. At this time, the brightest brightness can only reach the brightness of a light strip. When the color temperature is the middle value, the two lights should light up with the same brightness. At this time, the maximum overall brightness can reach twice that of the lowest or highest color temperature. That is, for different color temperatures, there is a brightness maximum value. Therefore, the maximum value is calculated in real time in our software and fed back to the front end. Therefore, in the picture in the [Interface] section below, the background of the brightness adjustment slider bar of the desk lamp is shallow at one end, that is, the inaccessible area. In addition, I also added the face recognition function to automatically turn on and off the lights (see video or demonstration for details)

②Interface:
The front-end of this system uses the LVGL open source interface library to implement the interface. Each element on each screen has 2~3 animation details, and even the background will move, making the entire interface lively and interesting. (See the demonstration or attached video for details) In addition, I designed a digital display animation library through my own method, which can achieve a smooth and interesting switching process between any two numbers from 0 to 9. In order to speed up the development, I first used the ported version of LVGL on the computer VS2017 to design and debug the interface. After the final interface was completed, I imported it to ESP32 for running and adaptation. Although the final actual frame rate is not high, it is still worth it considering the cost of the front-end main control chip is less than 15 yuan.

Some screenshots of the interface are as follows:

The actual display effect is as follows:

③Ambilight:
Ambilight here refers to mapping the real-time picture of the computer screen onto the WS2812 color light strip to achieve a visual effect similar to the computer picture overflowing the screen, which is suitable for use when watching movies and playing 3D games. At first, I used QT's built-in capture API to capture the entire screen. The frame rate could reach 35, but the CPU usage reached 10%, so I only took screenshots around the screen each time and divided them into 4 screenshots, which indeed reduced the CPU usage to 0.6 %, but the frame rate is only 10 frames. Finally, after checking the information, I learned that Microsoft has released a new screenshot method after Windows 8: DXGI, which can accelerate screenshots through the GPU, so I used it as the final solution, easily increasing the frame rate to 90 frames, and basically not occupying the CPU. The windows system evaluates the power consumption of my host computer as extremely low. In addition, considering the problem of low color saturation in computer display interfaces most of the time, I subtracted the minimum color value from each color channel and then normalized it to achieve adjustable color saturation and adjustable brightness of the colored light strips.

④Network: The front desk and back desk are connected to each other through WIFI and connected to the router, and at the same time, they are connected to the campus network and public network. The front desk connects to the public network to obtain timing services, and the back desk connects to the public network to access Xiaoai classmates, thereby realizing the voice command of the mobile phone to turn on and off the dormitory lights. Since the IP address obtained by the router connected through the network cable on the campus network is fixed, you only need to set up the router to bind the front-end and back-end IPs to the MAC, and set up port forwarding, so that anyone on the three campuses or five campuses can connect to The campus network provides high-quality control of the system in my dormitory. In order to improve real-time performance, UDP communication is used in the front-end and back-end, mobile phone terminal, and computer screen capture and sending host computer, and the actual measurement results are good. To this end, I designed an Android app for judges located on other campuses to remotely control the device during assessment. Please ensure that your mobile phone is connected to the campus network before taking control.

5. Testing and Debugging

  The functions of this system have been gradually improved during the development process this month, and the developed functions have been used every day since the prototype was prototyped half a month ago, as well as being used as a test. Until now ( 23:00 on May 13th) there has been no problem.

Since the performance of this system is mainly reflected in the feeling it gives people, it is impossible to obtain a series of values ​​​​to test its perfection. Therefore, it may be mainly through demonstration or video recording to let the judges feel its convenience.

We selected the following criteria for testing desktop systems: five points in total: security, reliability, practicality, aesthetics and response speed .

1. Security testing

Half a month of trouble-free operation

2. Reliability

After the final results came out, the team took the same operations as normal use and tested the reliability of the system multiple times in a short period of time. It quickly turned on and off the system within 1 minute and repaired the system.

Various experiments such as quick adjustment of the brightness and saturation of the light strip and quick switching of app pages were conducted to verify the reliability of the system in daily life. Ultimately, no matter how quickly and violently the system was tuned, it failed to destabilize it.

1. Practicality

Due to the discreet nature of the system's layout, I often forget it exists until it lights up when I'm about to sit down, or turns off slowly after I leave my seat. The experience is wonderful.

4. Beautiful

Test data team recorded in the form of video