TinyML and Efficient Deep Learning Computing | MIT 6.S965 Fall 2022

Have you found it difficult to deploy neural networks on mobile devices and IoT devices? Have you ever found it too slow to train neural networks? This course is a deep dive into efficient machine learning techniques that enable powerful deep learning applications on resource-constrained devices. Topics cover efficient inference techniques, including model compression, pruning, quantization, neural architecture search, and distillation; and efficient training techniques, including gradient compression and on-device transfer learning; followed by application-specific model optimization techniques for videos, point cloud, and NLP; and efficient quantum machine learning. Students will get hands-on experience implementing deep learning applications on microcontrollers, mobile phones, and quantum machines with an open-ended design project related to mobile AI.

• Chapter 1：Basics of Neural Networks (1 hours and 1 minutes and 31 seconds)

• Chapter 2：Training Deep Nets with PyTorch (18 minutes and 54 seconds)

• Chapter 3：Pruning and Sparsity (1 hours and 7 minutes and 6 seconds)

• Chapter 4：Pruning and Sparsity (1 hours and 7 minutes and 45 seconds)

• Chapter 5：Pruning and Sparsity (1 hours and 7 minutes and 39 seconds)

• Chapter 6：Quantization (1 hours and 11 minutes and 45 seconds)

• Chapter 7：Quantization (1 hours and 11 minutes and 42 seconds)

• Chapter 8：Quantization (1 hours and 11 minutes and 15 seconds)

• Chapter 9：Quantization (1 hours and 10 minutes and 53 seconds)

• Chapter 10：Neural Architecture Search (1 hours and 4 minutes and 0 seconds)

• Chapter 11：Neural Architecture Search (1 hours and 12 minutes and 54 seconds)

• Chapter 12：Neural Architecture Search (1 hours and 12 minutes and 50 seconds)

• Chapter 13：Neural Architecture Search (1 hours and 6 minutes and 57 seconds)

• Chapter 14：Knowledge Distillation (1 hours and 7 minutes and 25 seconds)

• Chapter 15：Knowledge Distillation (1 hours and 7 minutes and 21 seconds)

• Chapter 16：MCUNet- Tiny Neural Network Design for Microcontrollers (1 hours and 6 minutes and 41 seconds)

• Chapter 17：MCUNet- Tiny Neural Network Design for Microcontrollers (1 hours and 6 minutes and 38 seconds)

• Chapter 18：Distributed Training and Gradient Compression (1 hours and 1 minutes and 19 seconds)

• Chapter 19：Distributed Training and Gradient Compression (1 hours and 1 minutes and 5 seconds)

• Chapter 20：Distributed Training and Gradient Compression (57 minutes and 32 seconds)

• Chapter 21：Distributed Training and Gradient Compression (57 minutes and 28 seconds)

• Chapter 22：On-Device Training and Transfer Learning (1 hours and 15 minutes and 57 seconds)

• Chapter 23：On-Device Training and Transfer Learning (1 hours and 15 minutes and 55 seconds)

• Chapter 24：On-Device Training and Transfer Learning (1 hours and 6 minutes and 16 seconds)

• Chapter 25：On-Device Training and Transfer Learning (1 hours and 6 minutes and 29 seconds)

• Chapter 26：TinyEngine - Efficient Training and Inference on Microcontrollers (1 hours and 15 minutes and 6 seconds)

• Chapter 27：TinyEngine - Efficient Training and Inference on Microcontrollers (1 hours and 15 minutes and 1 seconds)

• Chapter 28：Efficient Point Cloud Recognition (1 hours and 14 minutes and 53 seconds)

• Chapter 29：Efficient Point Cloud Recognition (1 hours and 14 minutes and 14 seconds)

• Chapter 30：Efficient Video Understanding and Generative Models (1 hours and 37 minutes and 46 seconds)

• Chapter 31：Efficient Video Understanding and Generative Models (1 hours and 37 minutes and 40 seconds)

• Chapter 32：Efficient Transformers (1 hours and 18 minutes and 8 seconds)

• Chapter 33：Efficient Transformers (1 hours and 18 minutes and 4 seconds)

• Chapter 34：Basics of Quantum Computing (38 minutes and 38 seconds)

• Chapter 35：Basics of Quantum Computing (38 minutes and 38 seconds)

• Chapter 36：Quantum Machine Learning (1 hours and 9 minutes and 1 seconds)

• Chapter 37：Noise-Robust Quantum Machine Learning (1 hours and 8 minutes and 57 seconds)

• Chapter 38：Course Summary (12 minutes and 47 seconds)

• Chapter 39：Course Summary (12 minutes and 49 seconds)

• Chapter 40：AI Model EfficiencyToolkit (45 minutes and 15 seconds)

• Chapter 41：AI Model EfficiencyToolkit (45 minutes and 12 seconds)