FusedF0
A hybrid DSP + DNN pitch tracker that stays accurate in noise by fusing summary-correlograms with raw waveform representations.
Classical DSP pitch trackers such as the multi-band summary-correlogram (MBSC) are robust to noise. Neural networks that map raw waveform segments to F0 can be more accurate, but generalise less well and degrade faster as conditions get noisy. Rather than choosing between them, FusedF0 combines them.
The paper contributes a faster modified MBSC that keeps its accuracy, a DNN architecture that fuses those DSP features with raw waveform representations, and evidence that the auxiliary DSP features improve generalisation while keeping the network relatively small. On PTDB-TUG it beats both MBSC and CREPE, including optimised variants, for clean speech and at 20, 10 and 0 dB SNR.
Citation
@inproceedings{eren2023fusedf0,
title = {FusedF0: Improving DNN-Based F0 Estimation by Fusion of Summary-Correlograms and Raw Waveform Representations of Speech Signals},
author = {Eren, Eray and Tan, Lee Ngee and Alwan, Abeer},
booktitle = {Proc. INTERSPEECH},
pages = {4523--4527},
year = {2023},
doi = {10.21437/Interspeech.2023-2229}
}