Title Detecting Lombard speech using deep learning approach /
Authors Kąkol, Krzysztof ; Korvel, Gražina ; Tamulevičius, Gintautas ; Kostek, Bożena
DOI 10.3390/s23010315
Full Text Download
Is Part of Sensors.. Basel : Sensors. 2023, vol. 23, no. 1, art. no. 315, p. [1-20].. eISSN 1424-8220
Keywords [eng] 2D feature representations ; Lombard speech ; machine learning ; speech recognition ; threshold-based averaging strategy
Abstract [eng] Robust Lombard speech-in-noise detecting is challenging. This study proposes a strategy to detect Lombard speech using a machine learning approach for applications such as public address systems that work in near real time. The paper starts with the background concerning the Lombard effect. Then, assumptions of the work performed for Lombard speech detection are outlined. The framework proposed combines convolutional neural networks (CNNs) and various two-dimensional (2D) speech signal representations. To reduce the computational cost and not resign from the 2D representation-based approach, a strategy for threshold-based averaging of the Lombard effect detection results is introduced. The pseudocode of the averaging process is also included. A series of experiments are performed to determine the most effective network structure and the 2D speech signal representation. Investigations are carried out on German and Polish recordings containing Lombard speech. All 2D signal speech representations are tested with and without augmentation. Augmentation means using the alpha channel to store additional data: gender of the speaker, F0 frequency, and first two MFCCs. The experimental results show that Lombard and neutral speech recordings can clearly be discerned, which is done with high detection accuracy. It is also demonstrated that the proposed speech detection process is capable of working in near real-time. These are the key contributions of this work.
Published Basel : Sensors
Type Journal article
Language English
Publication date 2023
CC license CC license description