EasyARPiano: Piano Teaching Mobile App with Augmented Reality

Abstract

Music has evolved into a fascinating component of human life and a vital role player. Piano is one of the key fields of music, and music venues all over the world welcome a huge number of people who are interested in learning it. Despite this high demand, music institutions charge students a high price for piano lessons. The conducted studies related to this topic didn't solve the issue of learning costs since they required expensive devices to setup. The purpose of this study is to create an affordable, simple, easy-to-set-up, and accessible piano teaching mobile application that everybody can use with their basic augmented reality devices, such as smartphones, which have become an essential part of our lives in the last decades. This cross-platform app can be installed on Android or iOS smartphones. The player sits in front of a genuine MIDI keyboard and launches the app, selecting a music track and the proper mode to play in, whether it is beginner, medium, or expert. The app uses the smartphone's screen to provide hints by displaying falling notes on the piano keys that must be touched at the right time. It also uses the smartphone's camera to track the player's hand movements. Each song has a weighted score that the player should achieve. As the player continues playing, the earned score is displayed on the screen. After the game is completed, the final score is displayed on the screen as shared feedback on the player's performance. © 2021 IEEE.