A timbre-aware interactive computer music system
Interactive computer music systems analyze the performance
of a human musician in real-time, and generate material adaptively to accompany or improvise with the human musician. Most existing systems work with high-level performance events, such as note on and off, pitch, and loudness. Timbral information, which may be a crucial component of an improviser's musical language, is generally ignored.
We are developing a system that is able to parse both timbral and gestural information from a real-time performance. It extracts features that may be important to a human improviser when guiding a response or developing a dialogue. In addition to pitch or loudness, we also use timbral characteristics such as brightness, pitch/noise content, roughness, inharmonicity, etc. This information will be used to guide real-time audio processing and/or material generation.
(This is a collaborative project with British saxophonist John
Bill Hsu is an associate professor of computer science at San Francisco State University. His research interests include computer music, computer architecture and performance evaluation.