Chip selection

Why choose esp32? Because I think it has powerful performance, dual-core architecture, 240MHZ main frequency, strong computing power, and supports me for offline speech recognition.

This project does not use additional speech recognition modules. It only collects sounds with a microphone, and esp32 is used to parse and analyze the audio data.

The pit of the board

This is my first time to draw a board that was produced as a sample in the factory. I am really grateful to Lichuang EDA for providing such a good platform. I learned that strong electricity cannot be deposited with copper, etc. It's really because I'm not very skilled. I originally soldered on 5 lights, but two of them turned on. I took apart the remaining three. Maybe I broke them, so I changed the positions of the three lights. It still wouldn't light up, so I finally dismantled the last three lamps and will replace them later.

Hardware design

1.Main IC

1.ESP32-WROOM is a 32-bit chip developed by Espressif, with built-in WIFI, Bluetooth, 240HZ main frequency, etc.

2.MSM261S4030H0R A microphone with I2S interface for voice input

3.CP2102N-A01-GQFN28R a USB download chip

4.WS2812E 5 LED lamp beads (mainly considering that esp32 has a built-in RMT transmitter, and the waveform sent just meets the timing of WS2812)

5.AMS1117-3.3_C347222 5V to 3.3V

6.HK4100F-DC5V-SHG relay

........

There are many more that I won’t list one by one.

2. Overall framework of hardware

The main controller of the entire board is ESP32.

GPIO18 generates RMT waveform and sends it to WS2812, thereby lighting up the LED light.

GPIO32 33 25 is the I2S interface of esp32, which is connected to the pin corresponding to the microphone.

GPIO4 16 is connected to the relay to drive strong current

GPIO0 EN pin is connected to two external keyboards

The rest are some download circuits and power supply parts

software design

1. Development environment

Ubuntu18.0.4

vscode

esp-idf 3.2

2. Setting up the environment

Since the development environment is compiled under the Linux platform, I chose Ubuntu. You can see the installation steps and build it step by step from Espressif's official documentation.

Specific URL -->: https://docs.espressif.com/projects/esp-idf/zh_CN/v3.3.2/get-started/linux-setup.html

**The first step:** is to obtain the tool chain. The architecture of eps32 is xtensa architecture, so you need to download the cross-compilation compiler, connector, etc. After obtaining the tool chain, add it to the environment variable.

**The second step:** is to obtain esp-idf. Due to the domestic network environment, it is relatively slow to obtain the code from github. You can download it in a reasonable way. Pay attention to adding the --recursive parameter when cloning the warehouse, such as :

git clone -b v3.3.2 --recursive https://github.com/espressif/esp-idf.git

This parameter is required for all clones, otherwise the cloned warehouse will be incomplete.

Next, add the path of esp-idf to the environment variable.

run

python -m pip install --user -r $IDF_PATH/requirements.txt

Install the python library, because some idf drivers are scripts written in python.

Step 3: Get the Alibaba Cloud SDK, but I didn’t go to Alibaba Cloud to get it. I found it in Espressif’s warehouse. They have completed the library for connecting the underlying hardware interface to the Alibaba Cloud Feiyan platform. Run:

git clone https://github.com/espressif/esp-ali-smartliving.git

Compile ali-smartliving-device-sdk-c library

Execute in the esp-ali-smartliving directory:

cd ali-smartliving-device-sdk-c
make reconfig (选择SDK平台，选择esp32)
make menuconfig (选择相关功能配置,默认不需要修改,该步骤可以省略)
make (生成相关头文件和库文件)

Espressif also provided a sample project, and I made modifications during the project.

Step 4: Install Vscode. Vscode is free and has powerful plug-in functions. Vscode also releases a Linux version. I feel it is the best choice for developing esp32.

After these few steps, everything will be free and open source, so there is no need to worry about copyright reasons.

Generally speaking, it is more troublesome to set up the environment under Linux. Unlike IDEs such as Keil, you can set up the compilation tool chain step by step by yourself. The feeling is not felt in IDEs such as Keil.

3. The overall architecture of the software

1. Alibaba Cloud provides SDK for the Life IoT platform. The overall architecture of this software is based on Alibaba Cloud's SDK and Espressif's esp-idf.

2. esp32 is based on the freeRTOS operating system, which facilitates concurrent execution in the form of tasks.

The picture below is the general process I drew. There are two tasks. The main task in the mid-term is distributed with some small tasks.

The firmware source of speech recognition and Espressif Systems audio development framework esp-adf can perform offline speech recognition and support voice wake-up.

Espressif's speech recognition has been proposed by Espressif. It has compiled libraries. These libraries are all libraries trained by Espressif. CNN convolutional neural networks are trained in large quantities. Thank you for the libraries provided by Espressif.

Say: Hi Lexin, when you enter the wake-up mode, the LED light will flash in the form of a breathing light, indicating that you have entered the voice monitoring mode. Say the command to control the development board. If there is no response within 5s, restore the previous state and re-enter voice monitoring. model.

The picture is ugly

Alibaba Cloud monitors data without drawing pictures. It roughly parses Json data and executes corresponding commands. Alibaba Cloud's SDK has a dedicated API to provide calls.

Peripheral driver design:

Externally equipped with WS2812 microphone and relay

I have been working on this driver for WS2812 for a long time. I put all the relevant RMT parts under the ws2812.c file and encapsulated it into a structure to provide calls.

struct led_strip_s {

    esp_err_t (*get_hsv)(led_strip_t *strip, uint32_t *h, uint32_t *s, uint32_t *v);

    esp_err_t (*get_rgb)(led_strip_t *strip, uint32_t *red, uint32_t *green, uint32_t *blue);

    esp_err_t (*set_hsv)(led_strip_t *strip, uint32_t h, uint32_t s, uint32_t v);

    esp_err_t (*set_rgb)(led_strip_t *strip, uint32_t red, uint32_t green, uint32_t blue);

    esp_err_t (*set_pixel)(led_strip_t *strip, uint32_t index, uint32_t red, uint32_t green, uint32_t blue);

    esp_err_t (*refresh)(led_strip_t *strip, uint32_t timeout_ms);

    esp_err_t (*clear)(led_strip_t *strip, uint32_t timeout_ms);

    esp_err_t (*del)(led_strip_t *strip);

};
// 通过调用这个函数得到一个结构体
led_strip_t *led_strip_new_rmt_ws2812(const led_strip_config_t *config);

下面举个简单的例子
/******example*******/
led_strip_t *g_leds = led_strip_new_rmt_ws2812(&strip_config);  // 得到led的结构体
g_leds->g_leds->set_hsv(g_leds, h, s, v);   // 调用这个函数设置 灯的hsv属性

By designing the lamp driver in this way, the operation is simpler, making it easier to add and delete functions in the future, and the flexibility of the code is greatly improved.

Microphone, there is an I2S library under the esp-idf framework, which can be called directly. The audio part still needs to be carried out, and part of the filtering is done. For the specific code, I refer to the one in esp-adf.

static void i2s_init(void)

{

    i2s_config_t i2s_config = {

        .mode = I2S_MODE_MASTER | I2S_MODE_RX,           // the mode must be set according to DSP configuration

        .sample_rate = 16000,                            // must be the same as DSP configuration

        .channel_format = I2S_CHANNEL_FMT_RIGHT_LEFT,    // must be the same as DSP configuration

        .bits_per_sample = 32,                           // must be the same as DSP configuration

        .communication_format = I2S_COMM_FORMAT_I2S,

        .dma_buf_count = 3,

        .dma_buf_len = 300,

        .intr_alloc_flags = ESP_INTR_FLAG_LEVEL2,

    };

    i2s_pin_config_t pin_config = {

        .bck_io_num = BOARD_DMIC_I2S_SCK,  // IIS_SCLK

        .ws_io_num = BOARD_DMIC_I2S_WS,    // IIS_LCLK

        .data_out_num = -1,                // IIS_DSIN

        .data_in_num = BOARD_DMIC_I2S_SDO  // IIS_DOUT

    };

    i2s_driver_install(1, &i2s_config, 0, NULL);

    i2s_set_pin(1, &pin_config);

    i2s_zero_dma_buffer(1);

}

The relay only uses the output function of GPIO. There is also a driver in esp-idf: include #include "driver/gpio.h"

 gpio_config_t io_conf;

    io_conf.intr_type = GPIO_PIN_INTR_DISABLE;
    //set as output mode
    io_conf.mode = GPIO_MODE_OUTPUT;
    //bit mask of the pins that you want to set,e.g
    io_conf.pin_bit_mask = GPIO_OUTPUT_PIN_SEL;
    //disable pull-down mode
    io_conf.pull_down_en = 0;
    //disable pull-up mode
    io_conf.pull_up_en = 0;
    //configure GPIO with the given settings
    gpio_config(&io_conf);
通过调用 
   gpio_set_level(GPIO_OUTPUT_IO_16, leavel); 就可以进行很简单的GPIO控制

Partitioning of ESP32 partition table

esp32 divides the FLASH space. For a detailed introduction to partition tables, please refer to: https://docs.espressif.com/projects/esp-idf/zh_CN/v3.3.2/api-guides/partition-tables.html

Detailed introduction is provided in the documentation. We burn the quadruple information into the specified location of flash.

The module I use is a 4M flash. Because it needs to perform speech recognition, the nvs need to be divided into larger parts. The next one stores wifi information. I also reserved two spaces for OTA upgrades, but this space is too large. It is too small to be used. The last one is to store the four-tuple information of the Alibaba Cloud Life IoT Platform.

The code below is my partition table. The first partition is very large because speech recognition requires a lot of space. The last partition stores four-tuple information.

factory,  app,  factory, ,        3840K
nvs,      data, nvs,     ,        0x4000,
otadata,  data, ota,     ,        0x2000,
phy_init, data, phy,     ,        0x1000,
ota_0,    app,  ota_0,   ,        0x1000,
ota_1,    app,  ota_1,   ,        0x1000,
fctry,    data, nvs,     ,        0x4000

Burn quadruple to flash

mass_mfg 目录中有一参考配置：single_mfg_config.csv，请拷贝成自己的配置文件，如 my_single_mfg_config.csv。

cp single_mfg_config.csv my_single_mfg_config.csv

使用自己的 ProductKey、ProductSecret、DeviceName、DeviceSecret 对 my_single_mfg_config.csv 进行修改：

key,type,encoding,value
aliyun-key,namespace,,
DeviceName,data,string,config
DeviceSecret,data,string,dsj3RuY74pgCBJ3zczKz1LaLK7RGApqh
ProductKey,data,string,a10BnLLzGv4
ProductSecret,data,string,pVfLpS1u3A9JM0go

将 config，dsj3RuY74pgCBJ3zczKz1LaLK7RGApqh，a10BnLLzGv4，pVfLpS1u3A9JM0go 修改为你对应的值。

修改完成后，使用如下命令生成对应的 NVS 分区：

$IDF_PATH/components/nvs_flash/nvs_partition_generator/nvs_partition_gen.py --input my_single_mfg_config.csv --output my_single_mfg.bin --size 0x4000

下载
针对 esp32：

$IDF_PATH/components/esptool_py/esptool/esptool.py write_flash 0x003f1000 my_single_mfg.bin
注：0x003f1000 这个地址，是存放四元组的起始地址，一定要算对，要跟分区表一致，否则会造成配网失败

Mobile phone distribution network

Creating products on the Alibaba Cloud Life IoT platform specifically refers to the set of processes. One thing to note here is that when using this tool to configure the network, you must choose one-click network distribution.

Then configure the network by scanning the QR code. The entrance to the QR code is here. Scan this QR code to configure network with one click

After the network configuration is completed, you can see this on your mobile phone and you can control it with your mobile phone.

Practicality

I think that later on, I can buy a lampshade to cover the board and put it on the table. The relay can be connected externally to control a fan or something.

This does not completely rely on the network. When the network is disconnected, it can be controlled offline, which is convenient for users. I will add OTA upgrade in the future to achieve remote upgrade.

8.26 I found that running an offline neural network takes up three-quarters of the flash size, leaving only 1M of space, which is not enough to store OTA upgraded firmware. Therefore, if you want to share OTA upgrade and offline identification, you need to increase the flash capacity. , maybe there is some way to crop it, I'll take a look.