Posts
'AI Music Generators Could Be a Boon For Artists - But Also Problematic' (techcrunch.com)29
"Our new robot overlords are making a whole lot of progress in the space of AI music generation," quips TechCrunch, discussing a new project called "Harmonai" backed by Stability AI (creators of the open source AI image generator Stable Diffusion):In late September, Harmonai released Dance Diffusion, an algorithm and set of tools that can generate clips of music by training on hundreds of hours of existing songs.... Dance Diffusion remains in the testing stages — at present, the system can only generate clips a few seconds long. But the early results provide a tantalizing glimpse at what could be the future of music creation, while at the same time raising questions about the potential impact on artists....
Google's AudioLM, detailed for the first time earlier this week, shows... an uncanny ability to generate piano music given a short snippet of playing. But it hasn't been open sourced. Dance Diffusion aims to overcome the limitations of previous open source tools by borrowing technology from image generators such as Stable Diffusion. The system is what's known as a diffusion model, which generates new data (e.g., songs) by learning how to destroy and recover many existing samples of data. As it's fed the existing samples — say, the entire Smashing Pumpkins discography — the model gets better at recovering all the data it had previously destroyed to create new works....
It's not the most intuitive idea. But as DALL-E 2, Stable Diffusion and other such systems have shown, the results can be remarkably realistic.
Its lyrics are gibberish, TechCrunch concedes — though their article also features several audio clips (including a style transfer of Smash Mouth's vocals onto the Tetris theme).
And the article also notes a new tool letting artists opt of of being used in AI training sets, before raising the obvious concern...
The project's lead stresses that "All of the models that are officially being released as part of Dance Diffusion are trained on public domain data, Creative Commons-licensed data and data contributed by artists in the community." But even with that, TechCrunch notes that "Assuming Dance Diffusion one day reaches the point where it can generate coherent whole songs, it seems inevitable that major ethical and legal issues will come to the fore."
For example, beyond the question of whether "training" is itself a copyright violation, there's the possibility that the algorithm might accidentally duplicate a copyrighted melody...
Google's AudioLM, detailed for the first time earlier this week, shows... an uncanny ability to generate piano music given a short snippet of playing. But it hasn't been open sourced. Dance Diffusion aims to overcome the limitations of previous open source tools by borrowing technology from image generators such as Stable Diffusion. The system is what's known as a diffusion model, which generates new data (e.g., songs) by learning how to destroy and recover many existing samples of data. As it's fed the existing samples — say, the entire Smashing Pumpkins discography — the model gets better at recovering all the data it had previously destroyed to create new works....
It's not the most intuitive idea. But as DALL-E 2, Stable Diffusion and other such systems have shown, the results can be remarkably realistic.
Its lyrics are gibberish, TechCrunch concedes — though their article also features several audio clips (including a style transfer of Smash Mouth's vocals onto the Tetris theme).
And the article also notes a new tool letting artists opt of of being used in AI training sets, before raising the obvious concern...
The project's lead stresses that "All of the models that are officially being released as part of Dance Diffusion are trained on public domain data, Creative Commons-licensed data and data contributed by artists in the community." But even with that, TechCrunch notes that "Assuming Dance Diffusion one day reaches the point where it can generate coherent whole songs, it seems inevitable that major ethical and legal issues will come to the fore."
For example, beyond the question of whether "training" is itself a copyright violation, there's the possibility that the algorithm might accidentally duplicate a copyrighted melody...
