Live or Studio?

When you drop an audio file, your browser:

1. Decodes it locally (the browser handles MP3/WAV/FLAC/M4A).
2. Resamples to 16 kHz mono and pulls out three 30-second windows.
3. Runs each window through the Audio Spectrogram Transformer — an 86-million-parameter model pretrained on AudioSet — to produce a 768-dimensional embedding per window.
4. Feeds that embedding into a tiny linear classifier (1,538 parameters) trained on 24,000 audio clips to distinguish live from studio.
5. Takes the majority vote across the three windows.

The models run in your browser via ONNX Runtime Web for inference, plus transformers.js for the mel-spectrogram preprocessing. Inference uses WebAssembly (the WebGPU backend silently mis-quantizes this model). The AST model downloads once (~85 MB), then the browser caches it. Your audio never leaves your device.

On a held-out test set drawn from the same distribution as training data, the classifier scores about 94.5% accuracy. On a separate adversarial test set of 150 shows tagged "soundboard," "audience," or "matrix" mix from the Internet Archive's Live Music Archive — recordings deliberately chosen to be hard — it still scores 93.9%. The model generalizes well, but it's not perfect: it tends to mis-classify quiet, lo-fi, or room-recorded studio tracks as live, and very clean soundboard-style live recordings as studio.

24,227 30-second clips. Live data: ~10,000 clips drawn from the Internet Archive's Live Music Archive, plus ~6,000 from a personal collection. Studio data: ~8,000 clips from the Free Music Archive (small subset). Training used a frozen AST backbone and a single linear layer trained with class-weighted cross-entropy.