收藏切换
Neural Speech Codec Method Based on Mel Spectrogram and Squeeze-Excitation-Weighted Quantization
收藏切换
Junzuo Zhou1, Jiangyan Yi1, *, Jianhua Tao2, 3, Yong Ren1, Tao Wang1
Journal of Computer Research and Development | 2025, 62(11) : 2725 - 2736
Less
收藏切换
Journal of Computer Research and Development | 2025, 62(11): 2725-2736
• Artificial Intelligence •
Neural Speech Codec Method Based on Mel Spectrogram and Squeeze-Excitation-Weighted Quantization
Full
Junzuo Zhou1, Jiangyan Yi1, *, Jianhua Tao2, 3, Yong Ren1, Tao Wang1
Affiliations
  • 1Institute of Automation, Chinese Academy of Sciences, Beijing 100190
  • 2Department of Automation, Tsinghua University, Beijing 100084
  • 3Beijing National Research Center for Information Science and Technology (Tsinghua University), Beijing 100084
  • born in 2000. Master candidate. His main research interest includes speech synthesis

    born in 1984. PhD, master supervisor. Her main research interests include speech information processing, speech generation and identification, and continuous learning

    born in 1972. PhD, PhD supervisor. His main research interests include intelligent information fusion and processing, speech processing, affective computing, and big data analysis

    born in 1998. PhD candidate. His main research interest includes speech synthesis

    born in 1996. PhD. His main research interest includes speech synthesis

About Author:

born in 2000. Master candidate. His main research interest includes speech synthesis

born in 1984. PhD, master supervisor. Her main research interests include speech information processing, speech generation and identification, and continuous learning

born in 1972. PhD, PhD supervisor. His main research interests include intelligent information fusion and processing, speech processing, affective computing, and big data analysis

born in 1998. PhD candidate. His main research interest includes speech synthesis

born in 1996. PhD. His main research interest includes speech synthesis

Published: 2025-11-01 doi: 10.7544/issn1000-1239.202440329
Outline
收藏切换

At present, end-to-end speech neural codecs, represented by SoundStream, have demonstrated outstanding performance in reconstructed speech quality. However, these methods require extensive convolutional computations, leading to lengthy encoding times. To address this issue, we introduce a neural speech codec method based on Mel spectrogram and squeeze-excitation-weighted quantization. This method aims to maintain high speech perceptual quality while reducing computational costs and increasing operational speed, thereby minimizing latency. Specifically, we utilize Mel spectrogram features as input, capitalize on the temporal compression properties during Mel spectrogram extraction, and combine a lower-layer convolutional encoder to simplify the computation process. Additionally, inspired by squeezed excitation network concepts, we extract excitation weights for each dimension of the output features from the encoder’s final layer. These weights are used as the weighting coefficients for each dimension of the compressed features when calculating codebook distances in the quantizer, thus enabling the learning of correlations between features and enhancing the performance of quantization. Experimental results on the LibriTTS and VCTK datasets indicate that this method significantly enhances the computational speed of the encoder and improves the reconstructed speech quality at lower bit rates (≤3 Kbps). For instance, at a bitrate of 1.5 Kbps, the real-time factor (RTF) of encoding computations can increase by up to 4.6 times. Regarding perceptual quality, at a bitrate of 0.75 Kbps, objective metrics such as short-time objective intelligibility (STOI) and virtual speech quality objective listener (VISQOL) show an average improvement of 8.72% compared with the baseline. Additionally, ablation studies not only demonstrate that the optimization effect of compressed excitation weight methods is inversely correlated with bitrate, but also reveal that, compared with the periodic activation function Snake, the Relu activation function can significantly speed up processing while maintaining comparable speech perceptual quality.

speech codec  /  Mel-spectrogram  /  squeeze-and-excitation networks  /  residual vector quantization  /  generative adversarial network
周俊佐, 易江燕, 陶建华, 任勇, 汪涛. 基于梅尔谱与压缩激励加权量化的语音神经编解码方法. 计算机研究与发展, 2025 , 62 (11) : 2725 -2736 . DOI: 10.7544/issn1000-1239.202440329
Junzuo Zhou, Jiangyan Yi, Jianhua Tao, Yong Ren, Tao Wang. Neural Speech Codec Method Based on Mel Spectrogram and Squeeze-Excitation-Weighted Quantization[J]. Journal of Computer Research and Development, 2025 , 62 (11) : 2725 -2736 . DOI: 10.7544/issn1000-1239.202440329
Year 2025 volume 62 Issue 11
114
53
Cite this Article
BibTeX
Article Info
doi: 10.7544/issn1000-1239.202440329
  • Receive Date:2024-05-21
  • Online Date:2026-11-10
  • Published:2025-11-01
Article Data
Affiliations
History
  • Received:2024-05-21
  • Revised:2025-03-27
  • Accepted:2025-04-04
Affiliations
    1Institute of Automation, Chinese Academy of Sciences, Beijing 100190
    2Department of Automation, Tsinghua University, Beijing 100084
    3Beijing National Research Center for Information Science and Technology (Tsinghua University), Beijing 100084
References
Share
https://castjournals.cast.org.cn/joweb/jsjyjyfz/EN/10.7544/issn1000-1239.202440329
Share to
QR

Scan QR to access full text

Cite this article
BibTeX
Citations
表12种不同金属材料的力学参数
科
Family
属数
Number of
genus
种数
Number of
species
占总种数比例
Percentage of
total species (%)
属
Genus
种数
Number of
species
占总种数比例
Percentage of total
species (%)
鹅膏菌科Amanitaceae 2 11 5.26 鹅膏菌属 Amanita 10 4.78
小菇科 Mycenaceae 2 12 5.74 丝盖伞属 Inocybe 5 2.39
多孔菌科 Polyporaceae 8 14 6.70 蜡蘑属 Laccaria 5 2.39
红菇科 Russulaceae 3 23 11.00 小皮伞属 Marasmius 6 2.87
小菇属 Mycena 11 5.26
光柄菇属 Pluteus 5 2.39
红菇属 Russula 17 8.13
栓菌属 Trametes 5 2.39
关闭全屏
  • BibTeX
  • EndNote
  • RefWorks
  • TxT