TuyaOpen Release v1.6.0

TuyaOpen 1.6.0 Release Notes

TuyaOpen 1.6.0 is officially released! This update brings a major upgrade to Tuya AI APIs to V2.1, with comprehensive enhancements to core capabilities: new OPUS encoding support and Raspberry Pi 4/5 device adaptation, along with deep architectural refactoring that significantly optimizes audio/video processing and multimodal interaction performance, covering three core modules: multimodal, device MCP, and multimedia.

Additionally, AI-related APIs have been refactored and upgraded to create a more lightweight edge-side AI interaction capability library, lowering the integration barrier for developers. The supporting development toolchain has been improved, with enriched demo examples to help developers get started quickly and develop efficiently.

AI APIs Update

• Support for OPUS audio encoding format reporting, significantly improving transmission efficiency and reducing AI interaction latency
• Support for video, image, and file reporting, enabling multimodal AI-Agent model interaction
• Optimized voice interruption response speed for a more natural conversation system
• Refactored AI application component implementation framework, organizing the functionality of each module

New Features

• Agent-triggered music/story playback functionality
• Support for device MCP with four built-in tools, allowing agents to actively trigger devices (query device information/switch modes/adjust volume/take photos)
• Simple device MCP registration API, enabling developers to quickly extend hardware MCP capabilities
• Button switching between multiple chat modes (long-press conversation mode, button conversation mode, wake-up conversation mode, free conversation mode)
• Support for displaying camera-captured images on devices and sending photos to AI for multimodal analysis
• Support for custom mode registration, allowing flexible customization of voice conversation strategies
• Support for custom UI style registration, enabling developers to quickly customize their AI interaction UI/UX

Component Structure

ai_main (Main Controller)
    ├── ai_agent (Cloud Communication)
    ├── ai_mode (Conversation Mode Management)
    ├── ai_audio (Audio Processing)
    ├── ai_ui (Interface Display)
    ├── ai_video (Video Processing)
    ├── ai_skills (Skill Processing)
    ├── ai_mcp (MCP Protocol)
    ├── assets (Resource Files)
    └── utility (Utility Classes)

New audio_player Audio Playback Service Component

Multiple Audio Stream Data Sources

• Memory buffer (MEM)
• Local files (FILE)
• Network streams (URL/HTTP)

Multiple Audio Decoding Formats

• MP3, WAV, Speex
• Opus, OGG Opus

Dual Audio Stream Playback (AI Conversation/Multimedia)

• Foreground Stream: For voice prompts, warnings, etc.
• Background Stream: For background music, etc.
• Support for preemption mode (foreground interrupts background) and mixer mode (simultaneous playback)

Audio Processing Components

• Decoder: Decodes compressed audio to PCM
• Resampling: Supports sample rate conversion
• Mixer: Supports multi-channel audio mixing
• Volume Controls: Volume control

Playback Management

• Create/add/delete playback items
• Previous/next track switching
• Loop playback (single track/playlist)
• Auto-play
• Playback controls: play/pause/resume/stop

Display Management Driver - New Features

New Frame Buffer Management

• New graphics drawing function interfaces (point drawing, matrix fill, full-screen fill)
• New image rotation interfaces (90°, 180°, 270°)

Format Conversion

• Pixel format conversion (RGB565 ↔ RGB888 ↔ RGB666, etc.)
• YUV422 to frame buffer conversion
• Monochrome conversion (supports multiple dithering algorithms)

New Graphics Accelerator (tal_dma2d) Driver

• Hardware-accelerated image format conversion, e.g., YUV422 -> RGB565/RGB888
• Hardware-accelerated 2D memory copy for image transmission and replication

Resource Management

• Multi-context support (can create multiple handles)
• Thread-safe (mutex protection)
• Hardware resource reuse (single hardware instance, multiple logical handles)

lvgl_v9 Multi-Display Device Support

• Support for dynamically adding display devices
• Support for multiple display devices working simultaneously
• Automatic association of input devices and display devices by device name

libhttp Component

• New HTTP session management functionality, supporting persistent connections and streaming reads

New Demo Examples

• New!! Development environment support for Raspberry Pi 4/5 and other SBC series, can run applications like your_chat_bot on Raspberry Pi
  • Requires external audio sound card (1-2 channel microphone ADC + 1 channel speaker DAC)
  • New Raspberry Pi SoC environment cross-compilation toolchain, can compile projects in (Raspberry Pi native environment/Win/Linux) environments
  • Support for ASR KWS voice wake-up and 3A voice algorithms
• Dual-eye emotion application duo_eye_mood UI supports separate display and management of dual-eye animations (dual-screen independent display)
  • Two SPI screens can render different content
• New switching between LVGL and Camera streams
  • Provides example samples
  • Completed rendering, acceleration, and scheduling optimizations on MCU

Other Fixes

• Fixed occasional Bluetooth network configuration failure issue

Development Improvements

• Support for exporting compile_commands.json to improve code navigation capabilities in editors like VSCode and Cursor

---

TuyaOpen 1.6.0 Release Notes

TuyaOpen 1.6.0 正式发布！本次更新重磅升级涂鸦 AI 基座至 V2.1 版本，核心能力迎来全面焕新：新增 OPUS 编码支持与树莓派4/5 设备适配，同步完成架构深度重构，大幅优化音视频处理及多模态交互性能，覆盖多模态、设备 MCP、多媒体三大核心模块。

此外，AI 相关 API 完成重构升级，打造更轻量化的端侧 AI 交互能力库，降低开发者集成门槛；配套开发工具链同步完善，丰富 Demo 示例，助力开发者快速上手、高效开发。

AI 基座更新

• 支持 OPUS 音频编码格式上报，大幅提升传输效率降低 AI 交互延时
• 支持视频、图片、文件上报，实现多模态 AI-Agent 模型交互
• 优化语音打断的响应速度，更加自然的对话系统
• 重构了 AI 应用组件的实现框架，梳理了各个模块的功能

新增功能

• 智能体触发音乐/故事播放的功能
• 支持设备 MCP，内置了四种工具，智能体可以主动触发设备（查询设备信息/切换模式/调节音量/拍照）
• 简易设备 MCP 注册 API，开发者可以快速拓展硬件 MCP 能力
• 按键切换多种聊天模式（长按对话模式、按键对话模式、唤醒对话模式、随意对话模式）
• 支持设备显示摄像头拍摄的画面，并拍照并发送给 AI 多模态分析
• 支持自定义的模式注册，语音对话策略可灵活定制
• 支持自定义的 UI 风格注册，开发者可快速定制自己 AI 交互 UI/UX

组件关系

ai_main (主控制器)
    ├── ai_agent (云端通信)
    ├── ai_mode (对话模式管理)
    ├── ai_audio (音频处理)
    ├── ai_ui (界面显示)
    ├── ai_video (视频处理)
    ├── ai_skills (技能处理)
    ├── ai_mcp (MCP协议)
    ├── assets (资源文件)
    └── utility (工具类)

新增 audio_player 音频播放服务组件

支持多音频流数据源

• 内存缓存区（MEM）
• 本地文件（FILE）
• 网络流（URL/HTTP）

支持多种音频解码格式

• MP3、WAV、Speex
• Opus、OGG Opus

双音频流播放（AI对话/多媒体）

• 前台音频流 (Foreground-Stream)：用于语音提示、警告等
• 后台音频流 (Background-Stream)：用于背景音乐等
• 支持抢占模式（前台打断后台）和混音 Mixer 模式（同时播放）

音频处理组件

• Decoder 解码：将压缩音频解码为 PCM
• Resampling 重采样：支持采样率转换
• Mixer 混音：支持多路音频混合
• Volume Controls：音量控制

播放管理

• 创建/添加/删除播放项
• 上一首/下一首切换
• 循环播放（单曲/列表）
• 自动播放
• 播放/暂停/恢复/停止等播放控制

Display 显示管理驱动新增多种功能

新增帧缓冲管理

• 新增图形绘制功能接口(点绘制,矩阵填充,全屏填充)
• 新增图像旋转接口(90°、180°、270°)

格式转换

• 像素格式转换（RGB565 ↔ RGB888 ↔ RGB666 等）
• YUV422 到帧缓冲转换
• 单色转换（支持多种抖动算法）

新增图形加速器 (tal_dma2d) 驱动

• 用硬件加速进行图像格式转换，如 YUV422 -> RGB565/RGB888

• 硬件加速的 2D 内存拷贝，用于图像传输与复制

  资源管理

  • 多上下文支持（可创建多个句柄）
  • 线程安全（互斥锁保护）
  • 硬件资源复用（单硬件实例，多逻辑句柄）

实现 lvgl_v9 的多显示设备的功能

• 支持动态添加显示设备
• 支持多个显示设备同时工作
• 通过设备名称自动关联输入设备和显示设备

libhttp 组件

• 新增 HTTP 会话管理功能，支持持久连接和流式读取

Demo 示例新增

• New!! 开发环境支持树莓派4/5 等 SBC 系列，可在树莓派上运行 your_chat_bot 等应用
  • 需要搭配外置音频声卡（1-2路麦克风 ADC + 1路喇叭 DAC）
  • 新增树莓派 SoC 环境交叉编译工具链，可在（树莓派自身环境/Win/Linux）环境下编译工程
  • 支持 ASR KWS 语音唤醒和 3A 语音算法
• 双眼情绪应用 duo_eye_mood 的 UI 支持双眼动画分开显示和管理（双屏异显）
  • 两个 SPI 屏幕可以渲染不同内容
• 新增 LVGL 和 Camera 流之间来回切换
  • 提供 example 示例
  • 完成了 MCU 上面的渲染、加速和调度优化

其他修复

• 修复蓝牙配网偶现失败的问题

开发改善

• 支持导出 compile_commands.json，便于提升在 VSCode、Cursor 等编辑环境中的代码跳转能力

TuyaOpen Release v1.5.1

1. The T5-Pocket supports connecting to expansion devices such as RFID and thermal printers, and has added new features such as Snake, Dino Run, a level gauge, and an e-book reader.
2. A new UART plain text AI dialogue application example has been added.
3. A new BMI270 six-axis attitude sensor routine has been added.
4. Support for 4g-l511c has been added.
5. TuyaOpen now supports AI pixel screens.

---

1、T5-Pocket 支持连接 RFID、热敏打印机等扩展设备，新增贪吃蛇，小恐龙跑酷、水平仪、电子书等功能
2、新增 UART 纯文本 AI 对话应用示例
3、新增 bmi270 六轴姿态传感器例程
4、支持了 4g-l511c
5、TuyaOpen 支持 AI 像素屏

TuyaOpen Release v1.5.0

1. Optimize T5AI to support SMP (Symmetric Multi-Processing) architecture
2. Add LVGL V8, allowing free selection between V9/V8 versions
3. Add multiple development boards including TUYA_T5AI_Pocket, TUYA_T5A_Core, WAVESHARE_T5AI_TOUCH_AMOLED_1_75, and others
4. Add tuya_t5_pocket gaming console application
5. Add camera_demo to enable camera integration with Tuya cloud
6. Add weather_get_demo to retrieve various weather information formats through Tuya cloud
7. Add LVGL game applications supporting multiple games including 2048, Klotski puzzle, and match-three games
8. Add T5AI DVP multiple video sensor drivers and examples
9. Add support for IR infrared remote control receiving and transmitting examples
10. Add support for WS2812 and other RGB LED drivers and examples

---

1. 优化 T5AI ，支持 SMP 架构
2. 新增 LVGL V8, 可在 V9/V8 中任意选择
3. 新增 TUYA_T5AI_Pocket、TUYA_T5A_Core、WAVESHARE_T5AI_TOUCH_AMOLED_1_75 等多款开发板
4. 新增 tuya_t5_pocket 游戏机应用
5. 新增 camera_demo，可实现摄像头接入 tuya 云
6. 新增 weather_get_demo，可通过 tuya 云获取多种格式的天气信息
7. 新增 lvgl gamae 应用，支持 2048、 华容道、消消乐等多款游戏
8. 新增 T5AI DVP 多种视频 sensor 驱动及 example
9. 新增支持 IR 红外遥控接收和发送 example
10. 新增支持 ws2812 等多款幻彩灯珠驱动及 example

TuyaOpen Release v1.4.0

1. Refactor the compilation system using Python-based build scripts, adding support for T5AI and ESP32 chips across Linux/Windows/Mac platforms
2. Fix LVGL GIF playback stuttering issues by upgrading to version 9.1.0
3. Add screen rotation functionality support
4. Adapt LVGL for monochrome screen drivers
5. Optimize display drivers with support for QSPI and MCU8080 interfaces
6. Add three new display drivers: SPI ST7305, QSPI ST7735S, and MCU8080_ST7796
7. Introduce new dual-screen emotional robot application
8. Optimize IO expansion chip XL9555 driver
9. Improve hardware compatibility for DNESP32-BOX ES8311 and NS4168 versions
10. Fix compilation warnings in ESP32 platform
11. Enhance chat_bot UI startup with optimized emotion display

---

1. 重构编译体系，使用python版本编译脚本，支持T5AI，ESP32 芯片 Linxu/Windows/Mac 全平台编译
2. 修复 lvgl 运行gif 卡顿的问题，lvgl 版本切换至9.1.0
3. 支持屏幕旋转的功能。
4. lvgl 适配了单色屏幕驱动。
5. 优化了显示驱动，支持 qspi, mcu8080 接口的屏幕驱动
6. 新增了 spi st7305，qspi st7735s，mcu8080_st7796 三款显示驱动
7. 新增双眼屏情绪机器人应用
8. 优化 IO 扩展芯片 XL9555 驱动
9. 优化支持了 DNESP32-BOX ES8311 和 NS4168 两个版本的硬件
10. 修复了 ESP32 platform 编译警告
11. 优化了 chat_bot UI 启动显示表情

TuyaOpen Release v1.3.1

1. Added Otto Robot AI application
2. Optimized AI voice interruption handling
3. Added TuyaOpen and platform info logging for easier issue tracking
4. Added dedicated PIDs for T5AI/ESP32 dev boards (improved management)
5. Added LED peripheral driver component
6. Added Docker-based build environment support
7. Added authorization code flashing feature
8. Optimized example project compilation method
9. Improved ESP32 AI/IoT build compatibility
10. Enhanced ESP32/T5AI display performance
11. Tuned audio effects for certain ESP32-S3 dev boards

For full details, see: merge dev branch to master (#164)

1. 新增 otto robot AI 应用
2. 优化 AI 语音打断
3. 新增 TuyaOpen 及 platform 信息打印，方便跟踪定位问题
4. 新增 T5AI/ESP32开发板专用 PID，方便后续管理
5. 新增 LED的外设组件
6. 新增 支持 docker 编译环境
7. 新增 授权码烧录功能
8. 优化 example 编译方式
9. 优化 ESP32 AI/IoT 编译兼容
10. 优化 ESP32/T5AI 显示效果
11. 优化 部分 ESP32-S3 开发板音频效果
    更多详细信息可在 merge dev branch to master (#164) 页面获取

TuyaOpen Release v1.3.0

1. Supports tuya.ai 2.0 capabilities, with enhanced multimodal capabilities for voice and video, enabling faster interactions.
2. Supports ESP32S3 integration with tuya.ai.
3. Supports T5AI development boards: T5AI_Board, T5AI_EVB, and T5AI_MOJI.
4. Supports four ESP32S3 development boards: ESP32 Breadboard, DNESP32S3_BOX, Waveshare ESP32 1.8 AMOLED, and wmnologo_xingzhi 0.96 OLED.
5. Fixes compatibility between ESP32 Tuya Cloud IoT applications and AI applications.
6. Enhances the voice interaction experience of your_chat_bot for smoother interactions.
7. Improves the UI of your_chat_bot to support emoji display.
8. your_chat_bot supports keyword wake-up and interruption

---

1. 支持 tuya.ai 2.0 能力，支持语音/视频多模态能力，交互速度更快
2. 支持 ESP32S3 接入 tuya.ai
3. 支持 T5AI 3款开发板：T5AI_Board、T5AI_EVB、T5AI_MOJI
4. 支持 ESP32S3 4款开发板：ESP32 面包板、正点原子 ESP32S3_BOX、waveshare esp32 1.8 AMOLD、星智 cube 0.96 oled
5. 修复 ESP32 tuya 云 IoT 应用与 AI 应用兼容
6. 完善 your_chat_bot 语音交互体验，交互更流畅
7. 完善 your_chat_bot 显示 UI，支持表情显示
8. your_chat_bot 支持关键词唤醒、打断

TuyaOpen Release v1.2.1

1. support ESP32S3
2. Optimize your_chat_bot for T5AI

TuyaOpen Release v1.2.0

1. Add tuya.ai support you_chat_bot AI application. your_chat_bot
2. Add lvgl demo to T5AI. lvgl_demo

TuyaOpen Release v1.1.1

1. Added support for audio collection and playback on the T5AI_Board
2. Added support for SPI-TFT screen driver (ILI9341，ST7789, GC9A01)
3. Added support for IIC touch chip driver (GT911, CST816X)
4. Add a knob driver
5. Add lvgl components and examples，support T3
6. Added support for BK7231N and LN882H chips
7. Update tos flash tool to support ESP32 series chip burning
8. Update the compilation tool ninja
9. Add the https client example

TuyaOpen Release v1.1.0

1. Support T5
2. Support ESP32/ESP32C3
3. Support PlatformIO
4. Upgrade tos tools
5. Upgrade the tls component to support mbedtsl v3.5