aboutsummaryrefslogtreecommitdiff

pyseg

pyannote/segmentation-3.0 speaker segmentation in dependency-free C.

The model reads ten seconds of 16 kHz mono audio and returns, for each of 589 output frames, a distribution over seven classes: no speaker, one of three speakers alone, or one of the three pairs. That is enough for voice activity detection, overlapped speech detection and per-frame speaker counting over a recording of any length, all of which this library does.

The library is C11 and libc. The weights are compiled into the binary. Building the weight blob needs PyTorch and pyannote.audio once.

Build

make                    # library and CLI, weights embedded
make test               # all gates (needs the fixtures below)
make bench              # timings

Options: THREADS=0 drops the pthread dependency, EMBED=0 leaves the weights out of the binary so it loads --weights pyseg_weights.bin instead, STATIC=1 links a self-contained executable, OPT=... overrides the compiler flags.

Do not add -ffast-math. It changes how the reductions contract and moves the results outside the tolerances above.

Command line

pyseg_cli [options] file.wav

  --mode speech|overlap|count|frames   what to print (default speech)
  --step SECONDS       window hop, default 1.0 (a tenth of the window)
  --threads N          windows to run in parallel, default 1
  --batch N            windows per engine call, default 4
  --weights FILE       weight blob; default is the embedded copy
  --onset X            score above which a region opens, default 0.5
  --offset X           score below which it closes, default 0.5
  --min-on SECONDS     drop regions shorter than this, default 0
  --min-off SECONDS    fill gaps shorter than this, default 0
  --scalar             portable C kernels instead of SIMD

Input must be 16 kHz 16-bit PCM WAV; the first channel is used. speech and overlap print start end pairs in seconds, count prints the expected number of speakers per frame, and frames prints all three tracks.

$ pyseg_cli --threads 8 meeting.wav
0.031 0.588
2.427 18.593
18.897 27.149

Cost scales with --step: the default hop of one second means every part of the recording is covered by ten windows, which is what pyannote's pipelines are tuned for. A larger hop costs proportionally less and blurs the aggregate.

Library

#include "pyseg.h"

pyseg_t *v = pyseg_create();                 /* embedded weights */
float probs[PYSEG_FRAMES * PYSEG_CLASSES];
pyseg_process_f32(v, samples, probs);        /* 160000 samples in */
pyseg_destroy(v);

probs holds one distribution over the seven powerset classes per frame. pyseg_to_multilabel turns that into per-speaker activity, and frame f covers the 991 samples starting at f * PYSEG_FRAME_STEP.

For a whole recording:

pyseg_slide_params p;
pyseg_slide_params_default(&p);
p.threads = 8;

size_t n = pyseg_slide_frames(num_samples, p.step);
float *tracks = malloc(n * PYSEG_TRACKS * sizeof(float));
pyseg_slide_s16(samples, num_samples, &p, tracks);

pyseg_segment segs[256];
size_t count = pyseg_binarize(tracks, n, PYSEG_TRACK_SPEECH, NULL, segs, 256);

pyseg_slide covers the recording with overlapping windows and averages them exactly as pyannote.audio.Inference does. pyseg_binarize turns one track into segments the way pyannote.audio.utils.signal.Binarize does.

No allocation, syscall or lock happens inside pyseg_process_*. An engine is not thread-safe and is not meant to be: pyseg_slide gives each thread its own, and so should any other caller.

Layout

path content
src/pyseg.h the public interface
src/pyseg.c the forward pass
src/kernels.c the one dense primitive, scalar and SIMD
src/weights.c weight blob reader
src/slide.c sliding window, aggregation, binarization
tools/reference.py numpy forward pass, the executable specification
tools/pipeline.py the aggregation reference
tools/export_weights.py checkpoint to pyseg_weights.bin
tools/embed.py blob to C source, standard library only
tools/verify_cli.py the built binary against pyannote's pipelines

Regenerating the fixtures

The parity tests need captured references, which need the Python environment:

uv venv --python 3.12 .venv
VIRTUAL_ENV=.venv uv pip install "pyannote.audio>=3.1" numpy soundfile
make weights            # only after a layout change; the blob is checked in
make golden             # capture reference outputs
make test
make verify             # end to end against pyannote's own pipelines

make golden needs a 16 kHz WAV at testdata_aepyx.wav; override with AUDIO=....

Licence

MIT license.

The model weights are pyannote/segmentation-3.0, by Alexis Plaquet and Hervé Bredin; see the model card and the paper.