Learning Objectives
424 words
2 min read
Visual companion
Python
Type and operator map
Python Week 1: the first filter for runtime behavior
View
Revision summary
What this note is really saying
Short form
# Learning Objectives - Apply spectral subtraction and Wiener filtering - Understand deep learning for enhancement - Make ASR robust to noise - Signal processing fundamentals - Understanding of noise ## 1. Speech Enhancement Goal Improve quality and intelligibility of noisy speech.
Learning Objectives
- Apply spectral subtraction and Wiener filtering
- Understand deep learning for enhancement
- Make ASR robust to noise
- Signal processing fundamentals
- Understanding of noise
1. Speech Enhancement Goal
Improve quality and intelligibility of noisy speech. Pre-processing for ASR, hearing aids, communication.
2. Classical Methods
Spectral Subtraction: Estimate noise spectrum from silence regions. Subtract from noisy speech spectrum. Simple but creates "musical noise" artifacts.
Wiener Filter: Optimal linear filter minimizing MSE between clean and estimated speech. Requires noise statistics. Can be frame-wise adaptive.
3. Deep Learning for Enhancement
Noise Suppression (DNS): DNN predicts clean speech from noisy input (mapping or masking).
Mapping-based: DNN learns direct mapping (noisy spectrogram -> clean spectrogram). U-Net architecture (encoder-decoder with skip connections).
Masking-based: DNN predicts time-frequency mask (0-1) to apply to noisy spectrogram. Ideal Binary Mask (0/1) or Ideal Ratio Mask (continuous). Phase-sensitive mask improves quality.
4. Robust ASR
Front-end: Enhancement + feature normalization (CMVN: cepstral mean and variance normalization) Back-end: Multi-condition training (train on noisy data), noise-aware training (append noise estimate), augmentation (add noise during training)
Q1: What is spectral subtraction and what are its limitations?Estimate noise spectrum from silence, subtract from noisy spectrum. Limitations: musical noise artifacts (random peaks), phase estimation error, non-stationary noise difficult to estimate. Q2: How do DNNs improve speech enhancement?Learn complex mapping from noisy to clean speech. Better at handling non-stationary noise, preserving speech quality. U-Net architecture captures multi-scale patterns. Masking-based approaches (predict T-F mask) often better than mapping. Q3: What is multi-condition training for robust ASR?Train acoustic model on both clean and noisy data (artificially add noise at various SNRs). Model learns noise-invariant features. Simple and effective: single model works across conditions. Q4: What is CMVN and why is it used?Cepstral Mean and Variance Normalization. Per utterance: subtract mean, divide by std. Reduces channel effects and slowly-varying noise. Standard pre-processing for robust ASR. Q5: How is speech enhancement evaluated?PESQ (Perceptual Evaluation of Speech Quality), STOI (Short-Time Objective Intelligibility), SNR improvement. PESQ: -0.5 to 4.5. STOI: 0-1 (intelligibility). Also subjective: MOS. Q6: Simple spectral subtraction:pythonimport numpy as np # Estimate noise from first 10 frames noise_power = np.mean(np.abs(STFT(signal[:10*frame_size]))**2, axis=0) # Subtract from full signal clean_power = np.maximum(signal_power - noise_power, 0) # Reconstruct with noisy phase clean_stft = np.sqrt(clean_power) * np.exp(1j * np.angle(noisy_stft))