Google music instrument AI experiment

Photo Credit: Google

Google has created a new AI experiment, this time simulating 100 different musical instruments like you’d play a keyboard.

The new music instrument playground can simulate over 100 instruments from around the world, including some exotic instruments. The Veena from India, the Dizi from China, and the Mbira from Zimbabwe are all represented. Even the resonator guitar, but you’ll have to ask for it by brand-name (Dobro) rather than description.

Music LM can pair the instrument sound with adjectives like ‘melancholic’ or ‘dramatic’ to create a specific sound. For advanced musicians, the tool even features a sequencer to make it possible to layer and loop up to four different generated instruments—each with their own sound.

Google says the starting point for this experiment was exploring an interface based on Music LM that inspires creativity and the discovery of instruments around the world. You can experiment with a ‘festive crystal flute’ to imagine Lizzo playing a jaunty holiday tune for the month of December by trying out the playground yourself.

The playground is an extension of its MusicLM project, a generative AI model that can create high-fidelity music from text prompts. Google introduced MusicLM in February 2023, trained on a dataset of 280,000 hours of music to produce songs from text prompts. Researchers say the model has been built with the ability to transform a collection of sequentially written descriptions into a musical story to build on existing melodies that can be whistled, hummed, sung, or played on an instrument.

The ability to play over 100 instruments means Google has been hard at work training MusicLM on hundreds of additional hours of each of these instruments. Whether it’s a crystal flute, mandolin, or sitar—hundreds of hours of these instruments being played were fed to the algorithm in order for it to accurately re-create the sounds. Do the musicians who played those instruments know their recordings were used in this manner? Google has not revealed its training model for MusicLM, so we’re only left to guess.