NIPS 2018 Creativity Art Gallery
Machine Learning for Creativity and Design
Title of musical artwork: Andalusian Fragrance
Year: 2018
Submission by: Fadi Al-Ghawanmeh and Kamel Smaïli
Technique: SMT Automatic Musical Accompaniment
Credits:
Audio file:
Improvisation is essential in Arab music. Mawwal (plural mawaweel) is a non-metric vocal improvisation and is often applied to narrative poetry. Upon the completion of each vocal sentence in the mawwal, the instrumentalist performs an instrumental recapitulation, or a translation of the vocal sentence. Mawwal is also called istikhbar in some Arab regions.
This musical submission features the new and improved version of the mawaweel computer application for automatic accompaniment to Arab vocal improvisation (mawwal). The new version is designed to provide a creative accompaniment that is primarily based on machine learning rather than solely musical knowledge. It is real-time and is capable of providing a relevant instrumental response, or translation, for any vocal sentence interpreted in any of the four Arab maqamat: bayati, saba, hijaz (on D) and nahawand (on G). The singer may select the corresponding button of the desired maqam in the application's GUI before or during performing each vocal sentence.
Due to not having sufficient datasets of instrumentally-accompanied Arab improvisations, and also to technical reasons preventing us from transcribing accompanied improvisations available as audio files on the Internet, we constructed our own parallel corpus. Because of the high expenses and great effort required to record the music necessary for building such corpora, our corpus was small and consisted of only 4,041 vocal phrases and their corresponding instrumental responses, or translations. Having such a very small corpus made it suitable to treat it the same as corpora of under-resourced natural languages were treated, and accordingly to apply statistical machine translation (SMT) rather than neural machine translation.
The new and improved version of the application kept the original mawaweel transcriber that was designed to transcribe the basic melody of the vocal signal and was based on a set of musical rules related to the maqam used. The same transcriber was used to automatically transcribe the vocal phrases in our parallel corpus (the instrumental phrases were in MIDI). The mawaweel application was built in Java and made good use of JSyn API for audio synthesis and JMSL API for algorithmic composition and performance. The new version makes use of the SMT engine Moses, in both the training as well as the translation.
The following two publications describe the Mawaweel model with further details:
"Andalusian Fragrance" is a mawwal narrating poetry excerpts from the dramatic love story of two very famous poets and lovers in Andalusia: Ibn Zaydún al-Makhzumi and Wallada bint al-Mustakfi. Throughout this six-minute artwork, the improviser Muhannad Alkhateeb used elaborative expressiveness, maqam modulation (four maqamat) and choice of register all to convey the deep and diverse feelings expressed in the poetry, such as: passion, doubt, anger, sorrow and hope. The computer successfully stood as a peer while translating the creative vocal sentences.
Just as the case can be in common studio recordings, as opposed to live concerts, we had the privilege of recording several takes and selecting the best ones from two perspectives: vocal and instrumental. We then applied slight reverb, EQ, and compression. However, the model is also robust for live performances.
- Snapshot of the GUI of the new Mawaweel application:
- The Mawaweel computer application is available for download at: http://www.mawaweel.com
- An example of instrumentally (MIDI keyboard) accompanied vocal improvisations used for the construction of our parallel corpus: