Evaluating the impact of visual supports on piano learning through simulated learners

Titelangaben

Emond, Bruno ; Schoppek, Wolfgang:
Evaluating the impact of visual supports on piano learning through simulated learners.
In: Smith, Brian K. ; Borge, Marcela ; Sottilare, Robert A. ; Schwarz, Jessica (Hrsg.): HCI International 2025 - Late Breaking Papers : Proceedings. Part XIV. - Cham : Springer , 2026 . - S. 177-192 . - (Lecture Notes in Computer Science ; 16344 )
ISBN 978-3-032-13174-4
DOI: https://doi.org/10.1007/978-3-032-13174-4

Volltext

Link zum Volltext (externe URL):

Abstract

Learning piano requires integrating visual, auditory, and motor skills, a cognitively demanding process. Evidence increasingly supports computer-assisted instruction over traditional methods. Although visual supports such as finger-number annotations are widely used, emerging digital technologies, including dynamic music sheets, keyboard projections, and note rolls, offer novel instructional possibilities. Typically, validation of instructional features occurs post-development, emphasizing product validation over early-stage instructional design verification. This paper demonstrates a complementary approach — using simulated learners to evaluate visual supports’ effectiveness at system verification, prior to evaluating a system with human performance data. We employ the ACT-R cognitive architecture to simulate novice piano learners, modelling their perception, motor actions, learning processes, and decision-making. The simulated tasks involve playing simple two- or three-note sequences using either hand. Simulation results indicate that visual supports (note annotation and key-highlighting) interact with hand sides during both learning and transfer tasks. These findings suggest that while visual supports could improve immediate instructional outcomes, they need to be understood in conjunction with motor, memory, and auditory
cognitive resources. By employing simulated learners, instructional designers can better evaluate instructional features early in development, potentially saving resources and enhancing educational outcomes. The paper concludes by highlighting future research on modelling piano learning and performance using the ACT-R architecture.