Add microphone support via a new driver

[JesseTG]

This PR adds recording support to the audio driver, for use in emulating microphones. This PR was motivated by the Nintendo DS; at a later date, I'll submit a PR to the melonDS repo using the functionality introduced here.

This PR also introduces read support to the SDL audio driver; audio input for other drivers is not implemented at this time.

However, this PR isn't yet done. My main challenges are:

1. Understanding how to call the recording functionality. Writing the driver was easy enough, but I'm not sure I know what to do with it.
2. Testing this driver. Since there are no cores that use the microphone (yet), I have no idea how to ensure that this implementation is correct.

[JesseTG]

This is the interface I'm thinking about implementing for the microphone. Any thoughts on the design?

#define RETRO_ENVIRONMENT_GET_MICROPHONE_INTERFACE (72 | RETRO_ENVIRONMENT_EXPERIMENTAL)
                                           /* retro_microphone_interface ** --
                                            * Returns an interface that can be used to receive audio from
                                            * one or more microphones, depending on what's supported by the
                                            * audio driver.
                                            *
                                            * Will return NULL if the current audio driver or libretro implementation
                                            * doesn't support microphones.
                                            */

/* A few thousand lines later... */

/**
 * Enables or disables the microphone at the given port number.
 * Microphones are disabled by default, you will need to enable them.
 * You may enable microphones throughout the lifetime of a core,
 * or only in instances where they're needed.
 *
 * A driver might not support microphones, or it might only support one.
 * If only one microphone is supported, use a value of 0 for port.
 *
 * @param port The number of the microphone to set. Most likely will be 0,
 * although it's conceivable that a platform could put a microphone in each controller.
 * @param state True if the microphone should receive audio input, false if it
 * should be idle.
 * @returns true if the microphone's state was successfully set, false if port
 * does not indicate a valid microphone or if there was an error.
 */
typedef bool (RETRO_CALLCONV *retro_set_microphone_state_t)(unsigned port, bool state);

/**
 * @param port The number of the microphone to query. Most likely will be 0,
 * unless a platform uses multiple microphones for input (e.g. per controller).
 * @return true if the microphone given by port is active, false if not or if
 * port does not indicate a valid microphone.
 */
typedef bool (RETRO_CALLCONV *retro_get_microphone_state_t)(unsigned port);

/**
 * @return The number of microphones that the current audio driver supports.
 * 0 indicates that the audio driver does not support microphones.
 * A return value of INT_MAX indicates that the audio driver
 * supports as many microphones as the system permits.
 * I don't know why you'd want arbitrarily many microphones,
 * but I guess this is how you'd represent that.
 * @note Should be a constant value.
 */
typedef unsigned (RETRO_CALLCONV *retro_max_supported_microphones_t)(void);

/**
 * @return The number of microphones that are available.
 * 0 indicates that no microphones are available,
 * or that the current audio driver doesn't support them.
 */
typedef unsigned (RETRO_CALLCONV *retro_num_available_microphones_t)(void);

/**
 * Retrieves the input received by the microphone since the previous frame.
 *
 * @param port The number of the microphone to query
 * @param data The buffer that will be used to store the microphone's data
 * @param data_length The size of the data buffer, in samples.
 * @return The number of samples that were collected this frame. Will return 0
 * if the microphone is invalid or disabled.
 */
typedef size_t (RETRO_CALLCONV *retro_get_microphone_input_t)(unsigned port, int16_t* data, size_t data_length);

struct retro_microphone_interface
{
   retro_max_supported_microphones_t max_supported_microphones;
   retro_num_available_microphones_t num_available_microphones;
   retro_set_microphone_state_t set_microphone_state;
   retro_get_microphone_state_t get_microphone_state;
   retro_get_microphone_input_t get_microphone_input;
};

[JesseTG]

It works!

https://youtu.be/t2EvaHfs6Vw

Now that I have a working proof-of-concept, what will I need to do to get this ready for merging? Some things that come to mind include:

• Exposing microphone support and frequency in a new "Audio Input" subcategory
• Adding support for another core or two (here's a PR for melonDS already)
• Resolving merge conflicts that have arisen since I started this
• Adding support for floating-point samples (right now I only process 16-bit samples)
• I believe there's a bug in which mic support won't work properly if a core is started directly from the command line.

[hizzlekizzle]

Congratulations! This is a great addition.

I think the priorities you mentioned are right on target and would be something in the order of:

3, 5, 1, 2 and 4.

[JesseTG]

Ah, there are a couple of other things I need to do as well:

• Implement thread wrappers in audio_thread_wrapper
• Implement support for ALSA, XAudio2, and WASAPI (per @LibretroAdmin's request)
• I might want to run the mic input through the resampler, too. Not sure yet.

[bslenul]

Tested on Windows 10 with SDL2 as the audio driver, seems to work great overall! ❤️

Noticed this weird "null" setting in the "Resampler" menu:

Also noticed a few issues with the MelonDS PR:

• Changing mic core options seems to make it stop working completely (e.g. changing "Microphone Input" to "Blow Noise" then back to "Microphone Input", or switching the "Listen for Mic Input" option).
• When the "Listen for Mic Input" option is set to "Always" it also affects "Blow/White Noise", which is probably unwanted since it will input that noise non-stop :p

[JesseTG]

Tested on Windows 10 with SDL2 as the audio driver, seems to work great overall! ❤️

Thank you for testing this! What did you use it on, if I may ask? Play anything fun?

Noticed this weird "null" setting in the "Resampler" menu:

Ah, I know exactly what's wrong there. That's supposed to be the mic input rate. Must've missed a few text labels in the most recent merge.

Also noticed a few issues with the MelonDS PR:

* Changing mic core options seems to make it stop working completely (e.g. changing "Microphone Input" to "Blow Noise" then back to "Microphone Input", or switching the "Listen for Mic Input" option).

Does the issue go away if you reset (both soft-reset and unload-load cycle)?

* When the "Listen for Mic Input" option is set to "Always" it also affects "Blow/White Noise", which is probably unwanted since it will input that noise non-stop :p

Hm, that's technically a bug. I'll fix that.

[bslenul]

Thank you for testing this! What did you use it on, if I may ask? Play anything fun?

Mostly tested with http://drastic-ds.com/micrecord.nds and also Electroplankton as you suggested on Discord :D

Also briefly tested Zelda Phantom Hourglass (blowing candles) and Mario Kart (inflating balloons in battle mode).

Does the issue go away if you reset (both soft-reset and unload-load cycle)?

H hotkey or Quick Menu > Restart doesn't fix the issue, closing content and launching again fixes it however.

[JesseTG]

@hunterk @LibretroAdmin I now consider this PR complete and await your review. PRs for support in melonDS and a sample core will be ready soon.

I also intend to add support for flycast, citra, and one of the NES cores at a later date.

[JesseTG]

@Themaister I'm told that you would be a good person to review this. Would you mind taking a look when you get some time?

[LibretroAdmin]

Apart from this, I'd also appreciate a define 'HAVE_MICROPHONE' so we can bake out all the microphone code this adds to the codebase. This is for more lightweight platforms that have a limited amount of RAM and no microphone support to speak of, so we can bake out redundant code like this and not add to the executable size unnecessarily.

I can later go to the effort of adding HAVE_MICROPHONE to the platforms that actually have the support ready.

Another change I'd like to see, instead of a separate 'microphone/' dir, it'd be better to move the files to audio/. And then instead of microphone/drivers/, audio/drivers_microphone. I also think personally it's better if microphone_driver.c is just a part of audio_driver.c instead, since audio_driver.c is a relatively small file.

[LibretroAdmin]

There should always be a return at the end of a function with a return type. I'd suggest turning the default case into a break; and then returning "" at the end of the function.

[LibretroAdmin]

One line conditional blocks should have no brackets

[LibretroAdmin]

One line conditional blocks should have no brackets

[LibretroAdmin]

One line conditional blocks should have no brackets

[LibretroAdmin]

Bracket for struct on next line

[LibretroAdmin]

One line conditional blocks should have no brackets

[LibretroAdmin]

One line conditional blocks should have no brackets

[LibretroAdmin]

Is it a mistake to replace AUDIO_NULL with MICROPHONE_NULL here?

[JesseTG]

Nope. Each driver is given a distinct number, right? The value of the first constant in each enum here is equal to the value of the last constant in the previous enum.

[JesseTG]

Apart from this, I'd also appreciate a define 'HAVE_MICROPHONE' so we can bake out all the microphone code this adds to the codebase. This is for more lightweight platforms that have a limited amount of RAM and no microphone support to speak of, so we can bake out redundant code like this and not add to the executable size unnecessarily.

Okay, I'll do this.

I can later go to the effort of adding HAVE_MICROPHONE to the platforms that actually have the support ready.

I imagine that the vast majority of platforms do have native mic support, even if they don't yet have a working RetroArch driver. In your view, which platforms shouldn't define HAVE_MICROPHONE?

Another change I'd like to see, instead of a separate 'microphone/' dir, it'd be better to move the files to audio/. And then instead of microphone/drivers/, audio/drivers_microphone.

I can do this.

I also think personally it's better if microphone_driver.c is just a part of audio_driver.c instead, since audio_driver.c is a relatively small file.

I was originally doing this (check the commit history), but since XAudio doesn't provide mic support you can't assume that all audio APIs support microphones. On top of that, I noticed that the microphone behavior didn't really rely on the audio driver's state.

Or if you mean just moving the content into audio_driver.c? I'd still rather keep them apart to maintain clarity, since they're distinct drivers.

Thank you for your feedback, I appreciate it.

[LibretroAdmin]

You can keep the separate microphone_driver.c file for now. Do move the files to the audio/ folder though as previously suggested.

As for HAVE_MICROPHONE, it's so that we always keep the door open for size-optimized builds configured the way a user would want.