Machine Learning for Acoustic-Based Keystroke Recognition: A Study on Security Vulnerabilitiess

Niveau : Master 2

Encadrants: Kais Hassan (LAUM), Meysam Shamsi (LIUM)
Equipes d’accueil : Laboratoire d’Informatique de l’Université du Mans (LIUM) – Laboratoire d’Acoustique de l’Université du Mans (LAUM). Le stage se déroulera en présentiel.
Lieu : Le Mans Université
Début du stage : Février 2025
Contact : Kais Hassan, Meysam Shamsi (prénom.nom@univ-lemans.fr)

 

Candidature : Envoyer votre CV, une lettre de motivation adaptée au sujet proposé, ainsi que le relevé de notes le plus récent, (possibilité de joindre un avis ou des lettres de recommandations) à Kais Hassan ou Meysam Shamsi , avant le 10 décembre 2024

 
 

Description :
In an era where data breaches and cyber threats are becoming increasingly sophisticated, this project explores the vulnerabilities of everyday devices through Acoustic Side-Channel Attacks on Keyboards [1,2,3]. The goal is to demonstrate how the content of keystrokes can be compromised by simply recording the sounds produced by a keyboard. This research leverages cutting-edge technology to expose keystroke vulnerabilities, underscoring the need for robust security measures in the face of growing digital threats.

This internship is a preliminary study with three main objectives:

• Optimization of efficiency, minimization of data collection costs and maximization of keystrokes recognition accuracy: Develop efficient methods for collecting and synchronizing audio data to reduce overhead. Use advanced techniques to train a highly effective model across various conditions with minimal training data.
• Analyze the user behavior from acoustic signal: Categorize users’ typing behaviors based on acoustic signals and assess the model’s recognition accuracy. Use this analysis to establish security guidelines that address vulnerabilities in acoustic-based keystroke detection.
• Raise Security Awareness: Highlight the risks associated with acoustic side-channel attacks and propose countermeasures to protect sensitive information from these vulnerabilities.

 

Project Overview :

1. Data Collection Interface: Develop a synchronized recording system to capture keystrokes and the associated acoustic signals. This involves using two devices: one to log the exact timing of the keystrokes and another to record the corresponding sound. The challenge is to align these recordings with high precision to create a robust training dataset.
2. Machine Learning Model Training: Implement a deep neural network for keystroke recognition from the recorded audio. This includes adapting pre-trained models [4] used for speech recognition to identify individual keystrokes. The objective is to achieve high accuracy with minimal data by employing state-of-the-art techniques in audio classification.
3. Performance Evaluation: Assess the model’s effectiveness under various conditions. This involves testing with different keyboards, typists, environments, and microphones. The aim is to evaluate how the model performs across diverse scenarios and to identify potential weaknesses.
4. Analysis and Countermeasures: Conduct an in-depth analysis of typing behaviors and scenarios that may challenge the attack, e.g. [5]. Explore strategies to mitigate such acoustic attacks and enhance the security of keystroke data.

This project not only aims to expose a critical security vulnerability, but also to lay the foundation for long-term interdisciplinary research. State-of-the-art machine learning algorithms in speech processing have already shown promising results in decoding audio signals [4]. In the long term, a deeper study of human behavior, such as [6], and communication through acoustic signals can be envisioned.

 

Mots clés : Reconnaissance des frappes, apprentissage automatique, cybersécurité, catégorisation des signaux acoustiques

Le stagiaire bénéficiera de l’expertise des deux laboratoires tant sur la dimension acoustique (LAUM) que sur la dimension informatique et apprentissage automatique (LIUM).

Profil recherché : Candidat·e motivé·e par l’intelligence artificielle la cybersécurité et l’acoustique, actuellement inscrit en master informatique, acoustique, traitement du signal ou dans des domaines connexes.

 

Bibliographie

[1] Taheritajar, A., Harris, Z. M., & Rahaeimehr, R. (2023). A Survey on Acoustic Side Channel Attacks on Keyboards. arXiv preprint arXiv:2309.11012.

[2] Bai, J. X., Liu, B., & Song, L. (2021, October). I know your keyboard input: A robust keystroke eavesdropper based-on acoustic signals. In Proceedings of the 29th ACM International Conference on Multimedia (pp. 1239-1247).

[3]. Harrison, J., Toreini, E., & Mehrnezhad, M. (2023, July). A practical deep learning-based acoustic side channel attack on keyboards. In 2023 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) (pp. 270-280). IEEE

[4] Mohamed, A., Lee, H. Y., Borgholt, L., Havtorn, J. D., Edin, J., Igel, C., … & Watanabe, S. (2022). Self-supervised speech representation learning: A review. IEEE Journal of Selected Topics in Signal Processing, 16(6), 1179-1210.

[5]. Rodrigues, D., Macedo, G., Conti, M., & Pinto, P. (2024, June). A Prototype for Generating Random Key Sounds to Prevent Keyboard Acoustic Side-Channel Attacks. In 2024 IEEE 22nd Mediterranean Electrotechnical Conference (MELECON) (pp. 1287-1292). IEEE.

[6]. Kołakowska, A. (2015, June). Recognizing emotions on the basis of keystroke dynamics. In 2015 8th International Conference on Human System Interaction (HSI) (pp. 291-297). IEEE.