Learning Objectives

• Understand TTS architectures
• Apply neural TTS models (Tacotron, FastSpeech)
• Evaluate synthetic speech quality
• Week 7: Deep learning for speech
• Understanding of sequence-to-sequence models

1. Classical TTS

2. Neural TTS

• Text -> Character embeddings -> Encoder (CBHG) -> Attention -> Decoder (auto-regressive) -> Mel-spectrogram -> Vocoder (WaveNet, Griffin-Lim) -> Audio
• Handles text normalization, prosody prediction, spectrogram generation FastSpeech (Non-AutoRegressive):
• Encoder -> Duration Predictor (from teacher Tacotron) -> Decoder -> Mel-spectrogram
• Faster (parallel generation), more robust (no attention errors) WaveNet: Deep autoregressive model generating raw audio samples. Conditioned on linguistic features. High quality but slow for real-time.

3. Vocoders

Q1: What is the difference between concatenative and parametric TTS?

Concatenative: splices recorded speech (natural, large database needed, one voice). Parametric: generates from parameters (flexible, smaller footprint, robotic quality). Neural TTS has largely replaced both. Q2: How does Tacotron work?

Text -> character embeddings -> encoder (CBHG + bidirectional RNN) -> attention (aligns text to audio) -> autoregressive decoder (predicts spectrogram frames) -> post-net (refines spectrogram) -> vocoder -> audio. Q3: What is the advantage of FastSpeech over Tacotron?

Tacotron is autoregressive (slow, error propagation). FastSpeech is non-autoregressive (parallel generation = faster, more robust). Uses duration predictor learned from Tacotron alignments. Q4: What is a vocoder and what does it do?

Converts spectrogram features to time-domain audio waveform. Key for neural TTS quality. Options: Griffin-Lim (fast, lower quality), WaveNet (slow, best quality), HiFi-GAN (fast, near-WaveNet quality). Q5: How is TTS quality evaluated?

Mean Opinion Score (MOS): 1-5 subjective rating by listeners. Naturalness, intelligibility, prosody. 5 = indistinguishable from human. Also objective: MCD (Mel Cepstral Distortion), PESQ (perceptual evaluation). Q6: TTS inference example:

python
Run Inline
from TTS.api import TTS
tts = TTS(model_name="tts_models/en/ljspeech/tacotron2-DDC")
tts.tts_to_file(text="Hello, welcome to speech technology", file_path="output.wav")