Learning to play the piano by yourself isn’t easy. There are apps that can tell you which notes you hit, but they rarely analyze the way your hands move, whether your posture is working against you, or why a passage feels awkward.
This is one of the reasons human teachers have been such an advantage when it comes to learning to play an instrument. However, the one place human teachers fall short? They aren’t always available. You may have a class or two each week, but what about those moments when a wave of practice motivation strikes?
Music tech company ROLI believes it may have a new solution: an AI Music Coach that can see your hands, listen to what you play, and respond in real time. It’s a shift from passive feedback – and slightly more advanced versions of it like that offered by the ROLI Piano – toward active guidance, launching first for piano learning alongside ROLI’s new Airwave hardware.
Sitting at the core of ROLI’s new system is a sensing layer that goes beyond just listening for the right notes. An infrared hand-tracking camera watches both hands in real time, tracking all of each hand’s 27 joints at 90 frames per second. This means that the AI Music Coach doesn’t just respond to whether you hit the correct keys, but instead, how you play them: finger position, motion, and posture.
"The idea for Airwave and using hand tracking has existed for a very long time, but the technology wasn't ready," the company's Oli Snoddy told us. "The partnership with, and later acquisition of, UltraLeap really helped accelerate that part, and from there, the addition of language models as a mode of interaction and guidance felt almost inevitable. But yes, we've wanted to do this for a while, and now the world models and language models are ready, which is very exciting. We envision a much more natural way to practice alone, and also work together with a human teacher."
There’s a conversational layer on top of this, too. Players can talk to the app, ask questions, and receive guidance that adapts on the fly. Lessons continually adjust their pacing and difficulty, based on what the system sees and hears moment-to-moment. The feedback is immediate, much like a human teacher, and practice paths aren’t locked into rigid, pre-built lesson plans.
This approach sets the AI Music Coach apart from most other music-learning apps, which mainly rely on MIDI note data and fixed exercises. By being able to literally see micro-movements of the player’s hands, the AI can address technique issues that notes alone can’t reveal.
"A teacher looks at what you're doing with your hands, and our coach also does that (although clearly not in the same way a human teacher would)," explained Snoddy. "The simplest example is that it can tell you if you're playing with the right finger, not just if you're playing the right note. A learning app that's just listening to what you're playing won't know the difference. Our AI Piano Coach will know if you've crossed the thumb under or not, on a more tricky piece.
"Secondly, the AI Music Coach is using that information to determine when you're ready to speed up, slow down, move to the next section, switch to both hands, or move to views more akin to traditional sheet music. And finally, while we're starting with a closed beta, and starting simple, our hand tracking is detailed enough to sense not just if you're using the right finger, but also more nuanced aspects of your technique, such as whether your hand position is right. This is just the start, but our fundamental belief is that the hand tracking is the key unlock. And on a super simple level, just seeing your hands on screen makes it easier to not look down all the time, a challenge for new learners."
The AI Music Coach is the natural next step in ROLI’s foray into expressive, guided music learning. It builds on its earlier products like the ROLI Piano – and it feels like an organic evolution rather than a pivot.
Behind the scenes, the AI Music Coach combines advanced voice models with a deep musical knowledgebase. This foundation allows it to listen, watch, and respond in a way that feels natural and conversational rather than scripted.
The system is tightly integrated across the ROLI ecosystem, rather than as a standalone tool. Accessibility was a key part of the company’s design philosophy: the system supports over 40 languages, and doesn’t require its users to read sheet music. ROLI says that the app has already been tested globally, with beginners and professionals alike using it to refine their practice.
The AI Music Coach has now launched to a closed beta group, which will be followed by a public beta within the ROLI Learn app by the end of March, bundled with shipments of the new Airwave hardware. Pricing hasn’t been finalized at the time of writing.
ROLI has been careful not to position the system as a replacement for human teachers, who will always offer a certain touch that no AI can emulate. Instead, the company positions it as a scalable companion that’s always available. The AI Music Coach isn’t a shortcut; it’s a feedback amplifier, allowing music learning that adapts to you, rather than forcing you to adapt to a fixed syllabus.
Source: ROLI
New Atas may receive commission on purchases made through our links
