esp-adf-libs includes a hidden copy of libopus

[hossbeast]

I just discovered this, and in the interest of telling the folks who might be interested to know, I am posting it here.

As you know, esp-idf is the official sdk / build system / libraries for esp32. There is also esp-adf, which is specifically geared towards audio processing on the esp32. It bundles esp-idf (a slightly older version), and a bunch of extra components - mainly audio_pipeline_handle_t and all a load of audio elements you can use. Notably, there is an opus encoder element (components/esp-adf-libs/esp_codec/include/codec/opus_decoder.h) and opus decoder element (components/esp-adf-libs/esp_codec/include/codec/opus_encoder.h)

esp-idf and esp-adf libraries (they call them components) mostly consist of source code. You build those components on your machine when you build your project.

But esp-adf-libs is a distribution in binary form. People are already asking for the code (espressif/esp-adf-libs#12)

If you look at the header files for opus_decoder.h and opus_encoder.h in esp-adf-libs, the functionality is only exposed behind the audio pipeline abstraction (audio_element_handle_t).

How is the actual opus encoding/decoding being done? You might think, from reading the espressif documentation, that the "codec chip" is doing this in hardware.

Nope, they've built libopus and bundled it into esp-adf-libs.

todd@euclid ~/esp-adf
#
# some symbols defined in libopus are present in libesp_codec.a
#
1 (v2.4) % readelf -as components/esp-adf-libs/esp_codec/lib/esp32/libesp_codec.a | grep opus_decoder_create
    59: 00000000   149 FUNC    GLOBAL DEFAULT   35 opus_decoder_create

# 
# are these symbols declared in the libesp_codec header files? they are not
# 
0 (v2.4) % ag opus_decoder_create components/esp-adf-libs/esp_codec/include

I can confirm that you can just #include header files from https://github.com/xiph/opus in your esp-adf project, and it will build, link, and run. (Ok, I haven't tried the encoder, but the decoder works just fine).

Notably, this lets you decode opus and play it directly with the i2s driver, without using the audio_pipeline abstraction at all.

[hossbeast]

So in summary : do you need to build libopus for esp32 (this repo)? If you are using esp-idf, there's a case for doing so. (or if you want to build everything from source).

But you can also just use espressifs own esp-adf-libs/esp_codec/lib/esp32/libesp_codec.a from esp-adf

[stianeklund]

@hossbeast

How is the actual opus encoding/decoding being done? You might think, from reading the espressif documentation, that the "codec chip" is doing this in hardware.

Are you sure about this? I'm not sure it applies to the ESP32 Lyra series & or similar that have the 8388 on them?

Still a fascinating find. Thanks for commenting this.

[hossbeast]

LyraT v4.3 is the board I am testing on currently. I believe that the ES8388 is only for mixing and gain ; its digital inputs and outputs are PCM. At the very least, the header files I can find for it don't have any functionality for other encodings (esp-adf/components/audio_hal/driver/es8388/es8388.h )

[expresspotato]

@hossbeast is correct, there is no hardware decoding or encoding on the ES8388, it is simply a CODEC, aka: Audio (ADC / DAC)