Phoneme-Level Prosody Anchoring for Emotion-Preserving Singing Voice Conversion

Abstract

Singing voice conversion is a rapidly evolving domain within audio processing that aims to transform the vocal timbre of a source singer into that of a target singer while preserving the underlying musical content and linguistic information. A significant challenge in this field is the preservation of emotional expressivity, which is heavily reliant on intricate prosodic variations such as pitch fluctuations, vibrato, and dynamic energy contours. Traditional frame-level prosody modeling techniques often fail to capture the hierarchical structure of musical phrasing, leading to a loss of emotional fidelity in the converted audio. To address this limitation, this paper introduces a novel framework based on phoneme-level prosody anchoring. By extracting and aligning prosodic features including fundamental frequency, energy, and duration at the phoneme level, the proposed system establishes a robust structural anchor that guides the conversion process. This approach allows the generative model to disentangle emotional nuances from speaker-specific timbre more effectively. The methodology employs an advanced feature extraction module coupled with a phoneme-aligned text encoder and a dedicated emotion embedding network. Comprehensive experiments utilizing both objective acoustic metrics and subjective perceptual evaluations demonstrate that the proposed phoneme-level prosody anchoring framework significantly outperforms existing baseline models in maintaining emotional integrity. The results indicate a substantial reduction in fundamental frequency errors and a marked improvement in human perceptual scores for emotional expressiveness.