Neural Sync Active

Learning Objectives

Registry Synced

Learning Objectives

424 words

2 min read

Reading compass

Now · 1. Speech Enhancement Goal

Learning Objectives

Apply spectral subtraction and Wiener filtering
Understand deep learning for enhancement
Make ASR robust to noise
Signal processing fundamentals
Understanding of noise

1. Speech Enhancement Goal

Improve quality and intelligibility of noisy speech. Pre-processing for ASR, hearing aids, communication.

2. Classical Methods

Spectral Subtraction: Estimate noise spectrum from silence regions. Subtract from noisy speech spectrum. Simple but creates "musical noise" artifacts. Wiener Filter: Optimal linear filter minimizing MSE between clean and estimated speech. Requires noise statistics. Can be frame-wise adaptive.

3. Deep Learning for Enhancement

Noise Suppression (DNS): DNN predicts clean speech from noisy input (mapping or masking). Mapping-based: DNN learns direct mapping (noisy spectrogram -> clean spectrogram). U-Net architecture (encoder-decoder with skip connections). Masking-based: DNN predicts time-frequency mask (0-1) to apply to noisy spectrogram. Ideal Binary Mask (0/1) or Ideal Ratio Mask (continuous). Phase-sensitive mask improves quality.

4. Robust ASR

Front-end: Enhancement + feature normalization (CMVN: cepstral mean and variance normalization) Back-end: Multi-condition training (train on noisy data), noise-aware training (append noise estimate), augmentation (add noise during training)

Q1: What is spectral subtraction and what are its limitations?

Estimate noise spectrum from silence, subtract from noisy spectrum. Limitations: musical noise artifacts (random peaks), phase estimation error, non-stationary noise difficult to estimate. Q2: How do DNNs improve speech enhancement?

Learn complex mapping from noisy to clean speech. Better at handling non-stationary noise, preserving speech quality. U-Net architecture captures multi-scale patterns. Masking-based approaches (predict T-F mask) often better than mapping. Q3: What is multi-condition training for robust ASR?

Train acoustic model on both clean and noisy data (artificially add noise at various SNRs). Model learns noise-invariant features. Simple and effective: single model works across conditions. Q4: What is CMVN and why is it used?

Cepstral Mean and Variance Normalization. Per utterance: subtract mean, divide by std. Reduces channel effects and slowly-varying noise. Standard pre-processing for robust ASR. Q5: How is speech enhancement evaluated?

PESQ (Perceptual Evaluation of Speech Quality), STOI (Short-Time Objective Intelligibility), SNR improvement. PESQ: -0.5 to 4.5. STOI: 0-1 (intelligibility). Also subjective: MOS. Q6: Simple spectral subtraction:
python
import numpy as np
# Estimate noise from first 10 frames
noise_power = np.mean(np.abs(STFT(signal[:10*frame_size]))**2, axis=0)
# Subtract from full signal
clean_power = np.maximum(signal_power - noise_power, 0)
# Reconstruct with noisy phase
clean_stft = np.sqrt(clean_power) * np.exp(1j * np.angle(noisy_stft))

Join Discord PreviousSpeaker RecognitionNextSpeech Applications & Multilingual Speech