Audio Classification
LiteRT
LiteRT
LiteRT
on-device
android
gpu
audio-tagging
audioset
sound-event-detection
panns
cnn14
fully-gpu
Instructions to use litert-community/PANNs-CNN14-AudioSet-LiteRT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- LiteRT
How to use litert-community/PANNs-CNN14-AudioSet-LiteRT with LiteRT:
# No code snippets available yet for this library. # To use this model, check the repository files and the library's documentation. # Want to help? PRs adding snippets are welcome at: # https://github.com/huggingface/huggingface.js
- Notebooks
- Google Colab
- Kaggle
Add minimal usage snippets (Kotlin + Python)
Browse files
README.md
CHANGED
|
@@ -44,6 +44,47 @@ So the spectral front-end is computed on the **CPU** (the Whisper/Kokoro pattern
|
|
| 44 |
|
| 45 |
Mono **32 kHz**, padded/truncated to 10 s (320000 samples), values in [-1, 1]. Compute the log-mel as above → [1,1,1001,64]. The output 527 sigmoid probabilities are per-class (multi-label); take the top-K as tags.
|
| 46 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 47 |
## License
|
| 48 |
|
| 49 |
Code [Apache-2.0](https://github.com/qiuqiangkong/audioset_tagging_cnn/blob/master/LICENSE); weights `Cnn14_mAP=0.431.pth` [CC-BY-4.0](https://zenodo.org/record/3987831) (Zenodo). AudioSet ontology © Google, [CC-BY-4.0](https://research.google.com/audioset/). Upstream: [qiuqiangkong/audioset_tagging_cnn](https://github.com/qiuqiangkong/audioset_tagging_cnn).
|
|
|
|
| 44 |
|
| 45 |
Mono **32 kHz**, padded/truncated to 10 s (320000 samples), values in [-1, 1]. Compute the log-mel as above → [1,1,1001,64]. The output 527 sigmoid probabilities are per-class (multi-label); take the top-K as tags.
|
| 46 |
|
| 47 |
+
## Minimal usage
|
| 48 |
+
|
| 49 |
+
**Android (Kotlin, CompiledModel GPU)**
|
| 50 |
+
|
| 51 |
+
```kotlin
|
| 52 |
+
// staged into filesDir by an install script (162 MB — too big for assets)
|
| 53 |
+
val model = CompiledModel.create(File(ctx.filesDir, "cnn14_audioset_fp16.tflite").absolutePath,
|
| 54 |
+
CompiledModel.Options(Accelerator.GPU), null)
|
| 55 |
+
val inputs = model.createInputBuffers(); val outputs = model.createOutputBuffers()
|
| 56 |
+
inputs[0].writeFloat(logmel) // [1,1,1001,64] host log-mel (see Python below)
|
| 57 |
+
model.run(inputs, outputs)
|
| 58 |
+
val probs = outputs[0].readFloat() // [527] sigmoid, multi-label -> top-K tags
|
| 59 |
+
```
|
| 60 |
+
|
| 61 |
+
**Python (desktop verification)**
|
| 62 |
+
|
| 63 |
+
```python
|
| 64 |
+
import numpy as np, soundfile as sf
|
| 65 |
+
from ai_edge_litert.interpreter import Interpreter
|
| 66 |
+
|
| 67 |
+
SR, NFFT, HOP, NMEL, CLIP = 32000, 1024, 320, 64, 320000
|
| 68 |
+
wav, _ = sf.read("clip_32k.wav", dtype="float32") # mono 32 kHz
|
| 69 |
+
x = np.zeros(CLIP, np.float32); n = min(len(wav), CLIP); x[:n] = wav[:n]
|
| 70 |
+
|
| 71 |
+
# torchlibrosa-exact log-mel: center reflect-pad, periodic Hann, |rFFT|^2, mel, 10*log10
|
| 72 |
+
pad = np.pad(x, NFFT // 2, mode="reflect")
|
| 73 |
+
win = 0.5 - 0.5 * np.cos(2 * np.pi * np.arange(NFFT) / NFFT)
|
| 74 |
+
frames = 1 + CLIP // HOP # 1001
|
| 75 |
+
power = np.stack([np.abs(np.fft.rfft(pad[t*HOP:t*HOP+NFFT] * win))**2 for t in range(frames)])
|
| 76 |
+
fb = np.fromfile("mel_basis.bin", np.float32).reshape(NMEL, 513)
|
| 77 |
+
logmel = (10.0 * np.log10(np.maximum(power @ fb.T, 1e-10))).astype(np.float32)
|
| 78 |
+
|
| 79 |
+
it = Interpreter(model_path="cnn14_audioset_fp16.tflite"); it.allocate_tensors()
|
| 80 |
+
it.set_tensor(it.get_input_details()[0]["index"], logmel[None, None]); it.invoke()
|
| 81 |
+
probs = it.get_tensor(it.get_output_details()[0]["index"])[0] # [527]
|
| 82 |
+
|
| 83 |
+
labels = open("audioset_labels.txt").read().splitlines()
|
| 84 |
+
for i in probs.argsort()[::-1][:5]:
|
| 85 |
+
print(f"{probs[i]:.3f} {labels[i]}")
|
| 86 |
+
```
|
| 87 |
+
|
| 88 |
## License
|
| 89 |
|
| 90 |
Code [Apache-2.0](https://github.com/qiuqiangkong/audioset_tagging_cnn/blob/master/LICENSE); weights `Cnn14_mAP=0.431.pth` [CC-BY-4.0](https://zenodo.org/record/3987831) (Zenodo). AudioSet ontology © Google, [CC-BY-4.0](https://research.google.com/audioset/). Upstream: [qiuqiangkong/audioset_tagging_cnn](https://github.com/qiuqiangkong/audioset_tagging_cnn).
|