The esp32-ai project is an open-source initiative that brings a language model to run entirely on the ESP32-S3 microcontroller, without the need for a GPU, internet, or API. This model boasts 28.9 million parameters and can operate with 512KB SRAM, 8MB PSRAM, and 16MB Flash.
The key to its efficiency lies in keeping the main computation within the fast memory, while the embedding table of approximately 25 million parameters is stored in Flash, with the system only reading the necessary parts when generating tokens. The model currently supports writing short stories for children but is not capable of answering questions or writing code.
However, this project demonstrates the expanding boundaries of AI on ultra-small devices. The model's specifications include 28.9 million parameters, a 4-bit model size of about 14.9MB, and a speed of around 9.5 tokens per second.
It can display text directly on a small screen. For more information, visit the @@N8NLINK0@@.
References
These external sources were used to verify the article and provide deeper context.
Source Images
Conclusion
The esp32-ai project showcases the potential of running complex AI models on minimal hardware, paving the way for innovative applications of AI in constrained environments.
