Abstract:

This study focuses on the gestural technical learning of conducting by blind students, proposing the incorporation of haptic feedback as an additional tool in the monitoring and evaluation cycle of this technique. To implement this approach, students use a wearable device in the form of a bracelet, which emits vibrations in response to errors in the trajectory of the conducting technique. The research hypothesis seeks to investigate the effectiveness of this haptic device in correcting users' movements in relation to the predefined trajectory.

The wearable device (bracelet) integrates fundamentally with the Maestro v0.1 software, a system designed to optimize the teaching and learning process for visually impaired students in the field of conducting, especially during their individual practices. In the execution of a technical conducting pattern, a student can enhance their performance using the bracelet, which references the patterns stored in the database. At this stage, the student's movement is captured through the webcam (visual sensing), then compared with the previously recorded trajectory. The error, assessed by the computer vision software, is transmitted to the wearable prototype. It is anticipated that the movement correction will be achieved, as the device provides haptic feedback indicating the correct trajectory.

In order to provide haptic feedback to the student, a prototype was developed consisting of the following essential components: an ESP8266 board, an H bridge, and four vibration motors. It is worth noting that these vibration motors are elements commonly used in smartphones. The microcontroller, represented by the ESP8266 board, plays a crucial role in establishing a connection with a local network and communicating with the Maestro v0.1 software through the User Datagram Protocol. Each received package contains a vector that points to the next point in the correct trajectory. In the microcontroller, the X and Y coordinates are converted into values ranging from 0 to 1023, corresponding to Pulse Width Modulation. These values, in turn, determine the intensity of the motor vibration, indicating the magnitude of the necessary alteration. Both the microcontroller and the H bridge are soldered onto a board and housed in a 3D-printed box fixed to the user's arm. The motors are strategically positioned on the four sides of the wrist, attached to an elastic fabric bracelet. This configuration provides an integrated solution for delivering haptic feedback during the practice of the conducting technique pattern.

To investigate the proposed hypothesis, 25 voluntary participants with visual impairments and basic knowledge in conducting were invited to evaluate the bracelet prototype. The experiment was divided into two parts: the first consisted of a 30-minute practice without using the prototype, while the second comprised a 30-minute practice with the application of the device. Subsequently, the Mean Squared Error between the original trajectory and each test was calculated.

In summary, the analysis of the collected data highlights the effectiveness of the bracelet prototype, which provides haptic feedback by indicating corrections in the conducting technique for blind students in their individual studies. This result points to a promising path in the integration, inclusion, and accessibility of technical and performative conducting content for blind students, also serving as a potential educational resource for teachers.

Acknowledgement:
This work is financed by national funds through FCT – Fundação para a Ciência e a Tecnologia, I.P., within the scope of the Scientific Employment Stimulus – Individual Support – (2022.02280. CEECIND) and the CIDTFF R&D Unit (UIDB/00194/2020 – https://doi.org/10.54499/UIDB/00194/2020 – and UIDP/00194/2020 https://doi.org/10.54499/UIDP/00194/2020).

Keywords:

Maestro v0.1, Teaching and Learning of Conducting, Musical technical studies and visual impairment.