AI Song Contest time is back again in 2021!
Last year was a great learning experience. I got to make music hands-on for the first time in different settings using deep learning algorithms, attended conferences and read more research papers and wrote more code. I also learnt about the art of music a little bit more. I was introduced to the creative process of making art and augmenting it using technology to create something unique. From around the end of the past year until today, I have been learning about various research directions in music-AI and understanding the stacks of technology on which AI algorithms would have to work in order to support people in playing or creating music. This stack looks something like -
- Music Theory (Notation, Scale, Key, Meter, Rhythm, Chords, Playing)
- Musical Instruments (Piano, Guitar)
-
- Music hardware (MIDI controller, mic)
- Digital Signal Processing (Mixer, connecting wires and math)
- Digital Audio Workstations (LMMS, Bitwig-8track, Ableton Live)
- Virtual Studio Technology
- Max device programming and M4L devices
- User Interface Design
- Musical tool design
- Efficient workflows for use of musical tools
- Human Computer Interaction for musical instruments
- Instruments for exploring musical ideas in novel ways
- Instruments capable of learning explorable representations of musical ideas from data
-
- Deep Learning (Models dealing with audio and MIDI data)
This time I will be participating as part of the team from OVGU AILab. We have a slightly larger team and a little more know-how, so thats good too! The plan was to test out various deep learning and musical ideas during January and February, and to then take the best bits and build musical AI tools by the end of April and to create a song using these tools until the deadline on May 19. Phew, sounds like a long plan.
So the rough timeline looks something like this from Jan - May 2021:
- December: Test transformers on MIDI, Learn M4L a little
- January: Learn about music composition, production, recording, M4L, Ableton, Magenta apps
- Febuary: Start coding for testing ideas out, figure out the tech stack
- March: Finalize tech stack, build custom AI ideas, discuss data collection ideas
- April: Finish coding tasks, train all AI models, setup all music interfaces and hardware
- May: Make music, create a song, document the process, participate!
Since the song contest focuses on the use of AI in the creative process, we want to have as many AI tools as we can afford. However, having too many tools can make processes very complex sometimes, because of the number of options that the provide. Hence, having a perfectly sized set of useful AI tools for expressing musical creativity is the aim. Each tool must be able to interact with the person who acts on it. Hence, the tool must possess an interface of some sort, where it can be acted upon to get a response. The interface maybe a Jupyter notebook or an M4L/VST device in Ableton Live. Lastly, the functionality of the tool must be powered by deep learning algorithms. The tool itself must be capable of adapting to the kind of data that it is fed with. The tools must adapt to data by learning a model over the dataset. The model can then be used to generate similar data towards certain specific tasks.
So the plan is to collect and organize data to train our tools and create music with them in novel ways. Lets start understanding the various parts of this puzzle now
Data
- So how do we use data to shape the functionality of our tools?
- Storytelling with data?
Deep Learning
- What deep learning techniques are interesting and why?
- What human-like musical tasks can we emulate with deep learning systems?
- How can we explore sounds in more intricate ways?
Musical Interfaces
- How can we interact with AI models trained on musical data?
- How can we integrate AI workflows with standard music making tools?
Musical setup with AI tools
- How can many such AI tools trained on different types of musical data be used to create a workflow for musical creativity?
Tell a story with these tools
- What stories can we tell with such tools, that we couldn’t have otherwise told?
These are some of the many questions that I wonder about sometimes. And these are the questions that I will be trying to answer through participating in the contest.
I will write more about each of these blocks going forward.