diff options
| author | Onur Gündüz <ogunduz@gmail.com> | 2026-09-13 20:03:39 +0200 |
|---|---|---|
| committer | Onur Gündüz <ogunduz@gmail.com> | 2026-09-13 20:13:28 +0200 |
| commit | 64c3a1b528a88b4ae21bf3cbcff2537200477806 (patch) | |
| tree | 85da00748d7bbb5036aeb5a7a4b60b5abf75aeea /README.md | |
| download | pyseg-64c3a1b528a88b4ae21bf3cbcff2537200477806.tar.gz pyseg-64c3a1b528a88b4ae21bf3cbcff2537200477806.zip | |
Init
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 148 |
1 files changed, 148 insertions, 0 deletions
diff --git a/README.md b/README.md new file mode 100644 index 0000000..f16a9ff --- /dev/null +++ b/README.md @@ -0,0 +1,148 @@ +# pyseg + +`pyannote/segmentation-3.0` speaker segmentation in dependency-free C. + +The model reads ten seconds of 16 kHz mono audio and returns, for each of 589 +output frames, a distribution over seven classes: no speaker, one of three +speakers alone, or one of the three pairs. That is enough for voice activity +detection, overlapped speech detection and per-frame speaker counting over a +recording of any length, all of which this library does. + +The library is C11 and libc. The weights are compiled into the binary. +Building the weight blob needs PyTorch and pyannote.audio once. + +## Build + +```sh +make # library and CLI, weights embedded +make test # all gates (needs the fixtures below) +make bench # timings +``` + +Options: `THREADS=0` drops the pthread dependency, `EMBED=0` leaves the weights +out of the binary so it loads `--weights pyseg_weights.bin` instead, `STATIC=1` +links a self-contained executable, `OPT=...` overrides the compiler flags. + +Do not add `-ffast-math`. It changes how the reductions contract and moves the +results outside the tolerances above. + +## Command line + +``` +pyseg_cli [options] file.wav + + --mode speech|overlap|count|frames what to print (default speech) + --step SECONDS window hop, default 1.0 (a tenth of the window) + --threads N windows to run in parallel, default 1 + --batch N windows per engine call, default 4 + --weights FILE weight blob; default is the embedded copy + --onset X score above which a region opens, default 0.5 + --offset X score below which it closes, default 0.5 + --min-on SECONDS drop regions shorter than this, default 0 + --min-off SECONDS fill gaps shorter than this, default 0 + --scalar portable C kernels instead of SIMD +``` + +Input must be 16 kHz 16-bit PCM WAV; the first channel is used. `speech` and +`overlap` print `start end` pairs in seconds, `count` prints the expected +number of speakers per frame, and `frames` prints all three tracks. + +```console +$ pyseg_cli --threads 8 meeting.wav +0.031 0.588 +2.427 18.593 +18.897 27.149 +``` + +Cost scales with `--step`: the default hop of one second means every part of +the recording is covered by ten windows, which is what pyannote's pipelines are +tuned for. A larger hop costs proportionally less and blurs the aggregate. + +## Library + +```c +#include "pyseg.h" + +pyseg_t *v = pyseg_create(); /* embedded weights */ +float probs[PYSEG_FRAMES * PYSEG_CLASSES]; +pyseg_process_f32(v, samples, probs); /* 160000 samples in */ +pyseg_destroy(v); +``` + +`probs` holds one distribution over the seven powerset classes per frame. +`pyseg_to_multilabel` turns that into per-speaker activity, and frame `f` +covers the 991 samples starting at `f * PYSEG_FRAME_STEP`. + +For a whole recording: + +```c +pyseg_slide_params p; +pyseg_slide_params_default(&p); +p.threads = 8; + +size_t n = pyseg_slide_frames(num_samples, p.step); +float *tracks = malloc(n * PYSEG_TRACKS * sizeof(float)); +pyseg_slide_s16(samples, num_samples, &p, tracks); + +pyseg_segment segs[256]; +size_t count = pyseg_binarize(tracks, n, PYSEG_TRACK_SPEECH, NULL, segs, 256); +``` + +`pyseg_slide` covers the recording with overlapping windows and averages them +exactly as `pyannote.audio.Inference` does. `pyseg_binarize` turns one track +into segments the way `pyannote.audio.utils.signal.Binarize` does. + +No allocation, syscall or lock happens inside `pyseg_process_*`. An engine is +not thread-safe and is not meant to be: `pyseg_slide` gives each thread its +own, and so should any other caller. + +### What this does not do + +Speaker identities hold only inside a single ten-second window. Two +overlapping windows may number the same person differently, so joining windows +into whole-recording speakers needs a speaker embedding model and clustering, +which is a separate model and outside this library. Speech activity, +overlapped speech and the per-frame speaker count do carry across windows, and +those are what `pyseg_slide` aggregates. + +The model is 16 kHz mono only, and the window length is fixed at ten seconds +because the network was trained that way; per-channel statistics are computed +over the whole window, so it also cannot be run as a stream. + +## Layout + +| path | content | +|---|---| +| `src/pyseg.h` | the public interface | +| `src/pyseg.c` | the forward pass | +| `src/kernels.c` | the one dense primitive, scalar and SIMD | +| `src/weights.c` | weight blob reader | +| `src/slide.c` | sliding window, aggregation, binarization | +| `tools/reference.py` | numpy forward pass, the executable specification | +| `tools/pipeline.py` | the aggregation reference | +| `tools/export_weights.py` | checkpoint to `pyseg_weights.bin` | +| `tools/embed.py` | blob to C source, standard library only | +| `tools/verify_cli.py` | the built binary against pyannote's pipelines | + +## Regenerating the fixtures + +The parity tests need captured references, which need the Python environment: + +```sh +uv venv --python 3.12 .venv +VIRTUAL_ENV=.venv uv pip install "pyannote.audio>=3.1" numpy soundfile +make weights # only after a layout change; the blob is checked in +make golden # capture reference outputs +make test +make verify # end to end against pyannote's own pipelines +``` + +`make golden` needs a 16 kHz WAV at `testdata_aepyx.wav`; override with +`AUDIO=...`. + +## Licence + +The code here is yours to license. The model weights are +`pyannote/segmentation-3.0`, MIT, by Alexis Plaquet and Hervé Bredin; see the +[model card](https://hf.co/pyannote/segmentation-3.0) and the +[paper](https://www.isca-speech.org/archive/interspeech_2023/plaquet23_interspeech.html). |
