Hack the Violin Part 2: The Advanced Stuff - This Time There's AI

The talk follows two violinists who set out to build AI-assisted tools for violin practice. They began with a pitch analyzer adapted from a speech-analysis library, but found its measures, including formants, jitter and shimmer, were designed for the human voice and hard to link to intonation. A rhythm analyzer built on a music audio library came next, but raised its own problems, such as accounting for rubato and aligning multiple recordings for comparison.

The main pivot was timbre analysis using mel-frequency cepstral coefficients, which describe the shape of a sound's spectrum in terms tied to how people perceive it. The presenters recorded the same notes played plainly and then expressively, and an AI chatbot described the difference in descriptive words that matched their own judgement. They then gave the model their teaching notes and several 18th and 19th century violin treatises, which produced practical advice such as singing a phrase before playing it and preparing the sound mentally before bowing.

In closing, the presenters said they had found no other system that turns spectral results into violin coaching advice, and they proposed a future virtual teacher built from master quotations and recordings. In the questions they noted that useful feedback depends on a comparison recording, and that the approach had not yet been tested on double stops or on instruments other than the violin.