mlboydaisuke commited on
Commit
61f8a27
·
verified ·
1 Parent(s): a224538

Add minimal usage snippets (Kotlin + Python)

Browse files
Files changed (1) hide show
  1. README.md +41 -0
README.md CHANGED
@@ -44,6 +44,47 @@ So the spectral front-end is computed on the **CPU** (the Whisper/Kokoro pattern
44
 
45
  Mono **32 kHz**, padded/truncated to 10 s (320000 samples), values in [-1, 1]. Compute the log-mel as above → [1,1,1001,64]. The output 527 sigmoid probabilities are per-class (multi-label); take the top-K as tags.
46
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
47
  ## License
48
 
49
  Code [Apache-2.0](https://github.com/qiuqiangkong/audioset_tagging_cnn/blob/master/LICENSE); weights `Cnn14_mAP=0.431.pth` [CC-BY-4.0](https://zenodo.org/record/3987831) (Zenodo). AudioSet ontology © Google, [CC-BY-4.0](https://research.google.com/audioset/). Upstream: [qiuqiangkong/audioset_tagging_cnn](https://github.com/qiuqiangkong/audioset_tagging_cnn).
 
44
 
45
  Mono **32 kHz**, padded/truncated to 10 s (320000 samples), values in [-1, 1]. Compute the log-mel as above → [1,1,1001,64]. The output 527 sigmoid probabilities are per-class (multi-label); take the top-K as tags.
46
 
47
+ ## Minimal usage
48
+
49
+ **Android (Kotlin, CompiledModel GPU)**
50
+
51
+ ```kotlin
52
+ // staged into filesDir by an install script (162 MB — too big for assets)
53
+ val model = CompiledModel.create(File(ctx.filesDir, "cnn14_audioset_fp16.tflite").absolutePath,
54
+ CompiledModel.Options(Accelerator.GPU), null)
55
+ val inputs = model.createInputBuffers(); val outputs = model.createOutputBuffers()
56
+ inputs[0].writeFloat(logmel) // [1,1,1001,64] host log-mel (see Python below)
57
+ model.run(inputs, outputs)
58
+ val probs = outputs[0].readFloat() // [527] sigmoid, multi-label -> top-K tags
59
+ ```
60
+
61
+ **Python (desktop verification)**
62
+
63
+ ```python
64
+ import numpy as np, soundfile as sf
65
+ from ai_edge_litert.interpreter import Interpreter
66
+
67
+ SR, NFFT, HOP, NMEL, CLIP = 32000, 1024, 320, 64, 320000
68
+ wav, _ = sf.read("clip_32k.wav", dtype="float32") # mono 32 kHz
69
+ x = np.zeros(CLIP, np.float32); n = min(len(wav), CLIP); x[:n] = wav[:n]
70
+
71
+ # torchlibrosa-exact log-mel: center reflect-pad, periodic Hann, |rFFT|^2, mel, 10*log10
72
+ pad = np.pad(x, NFFT // 2, mode="reflect")
73
+ win = 0.5 - 0.5 * np.cos(2 * np.pi * np.arange(NFFT) / NFFT)
74
+ frames = 1 + CLIP // HOP # 1001
75
+ power = np.stack([np.abs(np.fft.rfft(pad[t*HOP:t*HOP+NFFT] * win))**2 for t in range(frames)])
76
+ fb = np.fromfile("mel_basis.bin", np.float32).reshape(NMEL, 513)
77
+ logmel = (10.0 * np.log10(np.maximum(power @ fb.T, 1e-10))).astype(np.float32)
78
+
79
+ it = Interpreter(model_path="cnn14_audioset_fp16.tflite"); it.allocate_tensors()
80
+ it.set_tensor(it.get_input_details()[0]["index"], logmel[None, None]); it.invoke()
81
+ probs = it.get_tensor(it.get_output_details()[0]["index"])[0] # [527]
82
+
83
+ labels = open("audioset_labels.txt").read().splitlines()
84
+ for i in probs.argsort()[::-1][:5]:
85
+ print(f"{probs[i]:.3f} {labels[i]}")
86
+ ```
87
+
88
  ## License
89
 
90
  Code [Apache-2.0](https://github.com/qiuqiangkong/audioset_tagging_cnn/blob/master/LICENSE); weights `Cnn14_mAP=0.431.pth` [CC-BY-4.0](https://zenodo.org/record/3987831) (Zenodo). AudioSet ontology © Google, [CC-BY-4.0](https://research.google.com/audioset/). Upstream: [qiuqiangkong/audioset_tagging_cnn](https://github.com/qiuqiangkong/audioset_tagging_cnn).