AI & Audio Engineering

AI is driving innovation in the field of audio production. Jonathan Bailey, the Chief Technology Officer of iZotope, a company pioneering advances with these technologies, talks about the state-of-the-art in audio software. As recently as 50 years ago, audio production required physical tools such as a soldering iron to achieve. With the rise of the personal computer these technical requirements have disappeared, replaced by software which handles all of the work with bits and bytes. From mixing to sound repair to post-production, machine learning-powered software like that offered by iZotope continues to automate an audio engineer’s workflow and even put professional audio production within the reach of amateurs.   Memorable Quotes “There's a macro trend which is actually bigger than sort of machine learning or AI, which is for the professional working in audio the last 50, 60, 70 plus years has been a transition away from the technical problem domain to the creative problem domain.” “You have a person that has a point of view that is guiding and steering the neural network. Now, there are new network architectures where a neural network can train another neural network, and those are pretty interesting, but there's still someone behind that, right? So they're, currently and for the foreseeable future, there's going to be kind of a guiding hand who's steering and curating what these things are capable of.” “There's a lot of buzz in the world of technology overall and I think probably a lot of snake oil and misunderstanding of what machine learning really is capable of, but on the other hand, it is a pretty spectacularly powerful technology and set of techniques that we can use in the world of music and audio.” “By being able to encode some of the best practices and some of the learning that only an audio engineer would have, and it's like your virtual audio engineer buddy, now people can create recordings that will sound good enough that they could be uploaded directly to Spotify or SoundCloud.” “As a team that works with ML all day long, we are just scratching the surface of what is even possible to do in terms of personalizing the experience to a specific user, in terms of continuing to enhance our algorithms in response to real-world data.” “We can really see a future where the audio engineer sits down, they've made a recording, it's de-noised, it's cleaned up. Everything works well together, and they can start getting creative, just painting with colors rather than having to fix a bunch of problems in the content that they produce. That's the world that we're trying to push those people towards.” “The sort of next horizon for both the world at large but definitely for audio is how can we use neural networks to generate content?” “We have a stream of audio coming into the product and a stream of audio leaving the product, and our job is to process that audio to make it sound better or make it sound more like the user wants us to.” “We can almost treat that representation like an image, and at each portion of that spectral representation, we can attempt to make a decision, for example, is this voice or not-voice?” “So we've trained a neural network to be able to make point-to-point decisions, both in time and in frequency.” “We had an idea that it might be possible to use machine learning to solve this problem.”   Who You'll Hear Dirk Knemeyer, Social Futurist and Producer of Creative Next (@dknemeyer) Jonathan Follett, Writer, Electronic Musician, Emerging Tech Researcher and Producer of Creative Next (@jonfollett) Jonathan Bailey, CTO, iZotope   Join The Conversation Website & Newsletter: www.creativenext.org Twitter: @GoCreativeNext Facebook: /GoCreativeNext Instagram: @GoCreativeNext   Sponsors GoInvo, A design practice dedicated to innovation in healthcare whose clients are as varied as AstraZeneca, 3M Health Information Services, and the U.S. Department of Health and Human Services. www.goinvo.com  Design Museum Foundation, A new kind of museum, they believe design can change the world. They’re online, nomadic, and focused on making design accessible to everyone. Their mission: bring the transformative power of design everywhere. You can learn about their exhibitions, events, magazine, and more. www.designmuseumfoundation.org BIF, As a purpose-driven firm, BIF is committed to bringing design strategy where it is needed most - health care, education, and public service to create value for our most vulnerable populations. www.bif.is