.png){kind=link}
In this project, music understanding and visualization are combined to analyze songs and generate corresponding artwork. We present a system to automatically analyze song lyrics and audio chords to tag music and automatically visualize its mood and harmony. The system begins with some audio files, like WAV or MP3, and applies a music recognition API to recognize and process them. We use a lyrics-based auto-tagging using natural language processing (NLP) and a chord recognition method from audio using music information retrieval (MIR) techniques to preprocess the lyrics, tags, and chords. Then, multi-label classification will be used to generate labels into a text-to-image generation AI model. This will create an innovative picture of the song and describe the mood of the song. Moreover, we can analyze the chords and tones to output as Piet Mondrian-style color blocks based on the chords as an additional output result. In general, this project includes the processes of music analysis, deep learning, and generative art. The meaning of this project is to provide an efficient and accurate way to create a mood drawing or an album/song cover picture for songs.
Daming Wang
Haolin Liu