Technology

AR Pianist app is fun to watch, but that’s about it

AR and VR company Massive Technologies has just released an app called AR Pianist which superimposes virtual musicians on pianos to give you “piano concerts from the comfort of your living room.” The app was first teased last December, when a tweet went viral with a video that showed virtual pianists popping up on-screen as the camera scanned across several different pianos.

Here’s the thing: My one colleague, Nilay Patel, says AR Pianist is “so cool,” while my other colleague, Thomas Ricker, says it’s the “AR equivalent of a fart app” (as in: use once and delete). Who’s right?

The gist of AR Pianist is that you hold your phone up to a piano and a virtual pianist will instantly appear, seated and playing. There’s a bunch of musical pieces to choose from, and the sheet music scrolls along the bottom of the app. There’s also a bunch of overlays on the piano itself: notes are marked, the piano can be re-skinned, notes approach the keyboard as they’re about to be played a la Guitar Hero, and keys light up and actually appear to be pressed down as they’re played. Neat!

The bones of AR Pianist are very similar to another app Massive Technologies launched in 2017 called Piano 3D, but that was more focused on music creation and had no superimposed humans. While that app had a dedicated learn mode, the ability to write songs and export them, and full core MIDI support, it doesn’t appear that any of these features are included in AR Pianist. At least, not yet.

Fayez Salka of Massive Technologies says the app is for people who want to experience music performances in a new way, or those “learning the piano with an AR headset.” The company leans in on this being an education app, with the app’s description saying that the sheet music is interactive (how? Who knows!) and that songs can be slowed down up to 100 times to study hand motions and techniques in detail.

Clearly there’s entertainment value in watching an AR pianist perform, as shown by the tweet going viral, but the app’s fun factor seems ephemeral, not something I’d use time and again. As for the education portion, if you had a headset that could augment the keys, it could theoretically be handy. But there aren’t many devices that would do the kind of spatial tracking this would require (let alone devices that are affordable).

Massive Technologies says that future iterations will include real-time feedback on your playing skills using your device’s microphone, the ability to swap out piano models, and more. If the app becomes more robust, it could be something piano players could use to refine their hand and body postures. Salka said that the virtual pianists’ movements are modeled from real human players. The company filmed pianists like Ossi Tanner and Tatia Kukhianidze with over a dozen infrared cameras to record all their movements to 1 mm accuracy.

As the app currently operates, is it practical? Absolutely not. Is it cool? Yes, and I’d probably find it briefly entertaining. This means I side with both Nilay and Thomas: it’s a very impressive AR equivalent of a fart app.

The app is free, so if you want to try out watching an AR piano virtuoso perform, go right ahead and get AR Pianist for iOS on the App Store.