Every month, the joint laboratory invites external speakers to take part in seminars for its partners. The next seminar is …
Zhiyao Duan (University of Rochester): “Making Music with Artificial Intelligence.”
Abstract:
Music is considered one of the highest ingenuities of mankind. Music making has a long history, and it has always co-evolved with technology. Artificial Intelligence (AI), as the most transformative technology today, is also transforming how humans make music. Different from other technologies, however, AI has the potential to become a true music partner of humans, and making music with AI could become as natural as making music with friends. In this talk, I will share my thoughts on why and how we should make music with AI. I will give a bird-eye view of our recent work on important components of music intelligence toward human-computer collaborative music making. I will then introduce and demonstrate several preliminary systems designed for different populations and music making scenarios. Finally, I will share my thoughts on challenges and future directions of this exciting research area.
Bio:
Zhiyao Duan is a professor in Electrical and Computer Engineering, Computer Science, and Sound Arts and Engineering at the University of Rochester. He is the director of New York State Center of Excellence in Data Science and Artificial Intelligence. He is also a co-founder of Violy, a music tech company for instrument learning. He received his B.S. in Automation and M.S. in Control Science and Engineering from Tsinghua University, China, in 2004 and 2008, respectively, and his Ph.D. in Computer Science from Northwestern University, USA, in 2013. His research interest is in computer audition and its connections with computer vision, natural language processing, and computational neuroscience. His research has been funded by NSF, NIH, NIJ, New York State Center of Excellence in Data Science and Artificial Intelligence, Adobe, ByteDance, IngenID, Kwai, Meta, Microsoft, and University of Rochester internal awards on AR/VR, health analytics, and data science. He served as Scientific Program Co-Chair of ISMIR 2021 and ISMIR 2026, associate editor of IEEE Open Journal of Signal Processing, guest editor of Transactions of the International Society for Music Information Retrieval, and senior area editor of IEEE Signal Processing Letters. He is a member of the IEEE Signal Processing Society Audio and Acoustic Signal Processing Technical Committee. He was the President of the ISMIR society between 2024 and 2025.
More information is available on his
website.
