---
language:
  - en
license: other
library_name: nemo
base_model: nvidia/parakeet-tdt-0.6b-v2
pipeline_tag: automatic-speech-recognition
tags:
  - nemo
  - parakeet
  - tdt
  - automatic-speech-recognition
  - air-traffic-control
  - par
  - singapore
  - military
metrics:
  - wer
---

# Parakeet-TDT 0.6B v2 - PAR Talkdown

NVIDIA Parakeet-TDT 0.6B v2 fine-tuned for Precision Approach Radar (PAR) talkdown speech: fast, clipped controller
calls and pilot replies, as used by the PAR training simulator.

## Usage

`model.nemo` is self-contained (weights, config and tokenizer). Audio is 16 kHz mono.

```python
from nemo.collections.asr.models import EncDecRNNTBPEModel

model = EncDecRNNTBPEModel.restore_from("par/ASR/model.nemo")
model.eval().cuda()
print(model.transcribe(["audio.wav"])[0].text)
```

Tested with `nemo_toolkit[asr]==2.7.3`.

## Output Format

Lowercase spoken words with no punctuation, for a downstream phrase matcher. Numbers, callsign numbers and
abbreviations are spoken digit by digit and letter by letter. ICAO "tree" and "fife" come out as three and five,
"niner" and "nine" are kept as spoken, and hesitations come out as `uh` or `um`.

## Results

WER on held-out test sets, best epoch (14 of 15). Every epoch is in [training/results.txt](training/results.txt).

| Test set | Clips | WER |
|---|---|---|
| PAR, held-out phrases (half sped up with radio noise) | 3,390 | **0.17%** |
| PAR, real recordings | 20 | 0.00% |
| General English (Loquacious test) | 461 | 8.43% |
| Singapore English (MNSC) | 267 | 0.95% |
| Real ATC radio (DeepDML) | 142 | 4.31% |

The PAR held-out phrases are synthetic and split by sentence, but share templates with training phrases, and the
20 real recordings are one speaker whose other recordings were trained on. New speakers are the real test. The
model was not trained on ASTRA phraseology and outputs PAR callsigns for unfamiliar ones, so use the
[ASTRA ASR](../../astra/ASR) for ASTRA. Gate it with VAD or push-to-talk, since noise-only audio can still produce
words.

## Training

Fine-tuned from `nvidia/parakeet-tdt-0.6b-v2` on 79,242 clips (93.7 h). 10% of PAR phrases were held out by
sentence across all voices for the PAR test set; the other test sets come from separate splits of each source.

| Source | Hours |
|---|---|
| PAR synthetic speech (six TTS voices) | 51.4 |
| ASTRA PAR-like synthetic speech and real recordings | 9.3 |
| General English replay (Loquacious: audiobooks, VoxPopuli, Common Voice, YODAS) | 22.5 |
| Singapore English (MNSC) | 7.6 |
| Real ATC radio (DeepDML) | 2.6 |
| Real PAR recordings | 0.3 |

Augmentation: speed copies of PAR speech from 0.8x to 2.5x (most at 1.3x to 2.0x, capped at 7 words/s); radio
band-pass, drive, noise, codec and push-to-talk clicks; and on about 4% of clips one light effect (mic hiss,
clipping, fade, one distant talker or quiet babble).

AdamW at 1e-4 with one epoch of warmup and cosine decay to 1e-6, weight decay 1e-3, batch 64, bf16, 15 epochs,
best checkpoint by PAR WER. About 10.5 h on one RTX 4090. Loss and WER per step and epoch are in
[training/train.log](training/train.log).