Skip to content

Feature Request- Implement Cross-Lingual Style Transfer Pipeline #155

@mnk-nasir

Description

@mnk-nasir

Objective: Extend the current implementation to support cross-lingual style transfer (e.g., using an English voice prompt to generate Spanish speech).

Technical Requirements:

Integration of a multilingual phonemizer (e.g., espeak-ng or gruut).

Update the audio_to_text alignment logic to handle non-English character sets.

Verification of the flow-matching objective's performance across different language embeddings.

Context: This would bring the repo closer to the full functionality described in the Meta Voicebox paper (Le et al., 2023).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions