aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorOnur Gündüz <ogunduz@gmail.com>2026-09-13 20:03:39 +0200
committerOnur Gündüz <ogunduz@gmail.com>2026-09-13 20:13:28 +0200
commit64c3a1b528a88b4ae21bf3cbcff2537200477806 (patch)
tree85da00748d7bbb5036aeb5a7a4b60b5abf75aeea /README.md
downloadpyseg-64c3a1b528a88b4ae21bf3cbcff2537200477806.tar.gz
pyseg-64c3a1b528a88b4ae21bf3cbcff2537200477806.zip
Init
Diffstat (limited to 'README.md')
-rw-r--r--README.md148
1 files changed, 148 insertions, 0 deletions
diff --git a/README.md b/README.md
new file mode 100644
index 0000000..f16a9ff
--- /dev/null
+++ b/README.md
@@ -0,0 +1,148 @@
+# pyseg
+
+`pyannote/segmentation-3.0` speaker segmentation in dependency-free C.
+
+The model reads ten seconds of 16 kHz mono audio and returns, for each of 589
+output frames, a distribution over seven classes: no speaker, one of three
+speakers alone, or one of the three pairs. That is enough for voice activity
+detection, overlapped speech detection and per-frame speaker counting over a
+recording of any length, all of which this library does.
+
+The library is C11 and libc. The weights are compiled into the binary.
+Building the weight blob needs PyTorch and pyannote.audio once.
+
+## Build
+
+```sh
+make # library and CLI, weights embedded
+make test # all gates (needs the fixtures below)
+make bench # timings
+```
+
+Options: `THREADS=0` drops the pthread dependency, `EMBED=0` leaves the weights
+out of the binary so it loads `--weights pyseg_weights.bin` instead, `STATIC=1`
+links a self-contained executable, `OPT=...` overrides the compiler flags.
+
+Do not add `-ffast-math`. It changes how the reductions contract and moves the
+results outside the tolerances above.
+
+## Command line
+
+```
+pyseg_cli [options] file.wav
+
+ --mode speech|overlap|count|frames what to print (default speech)
+ --step SECONDS window hop, default 1.0 (a tenth of the window)
+ --threads N windows to run in parallel, default 1
+ --batch N windows per engine call, default 4
+ --weights FILE weight blob; default is the embedded copy
+ --onset X score above which a region opens, default 0.5
+ --offset X score below which it closes, default 0.5
+ --min-on SECONDS drop regions shorter than this, default 0
+ --min-off SECONDS fill gaps shorter than this, default 0
+ --scalar portable C kernels instead of SIMD
+```
+
+Input must be 16 kHz 16-bit PCM WAV; the first channel is used. `speech` and
+`overlap` print `start end` pairs in seconds, `count` prints the expected
+number of speakers per frame, and `frames` prints all three tracks.
+
+```console
+$ pyseg_cli --threads 8 meeting.wav
+0.031 0.588
+2.427 18.593
+18.897 27.149
+```
+
+Cost scales with `--step`: the default hop of one second means every part of
+the recording is covered by ten windows, which is what pyannote's pipelines are
+tuned for. A larger hop costs proportionally less and blurs the aggregate.
+
+## Library
+
+```c
+#include "pyseg.h"
+
+pyseg_t *v = pyseg_create(); /* embedded weights */
+float probs[PYSEG_FRAMES * PYSEG_CLASSES];
+pyseg_process_f32(v, samples, probs); /* 160000 samples in */
+pyseg_destroy(v);
+```
+
+`probs` holds one distribution over the seven powerset classes per frame.
+`pyseg_to_multilabel` turns that into per-speaker activity, and frame `f`
+covers the 991 samples starting at `f * PYSEG_FRAME_STEP`.
+
+For a whole recording:
+
+```c
+pyseg_slide_params p;
+pyseg_slide_params_default(&p);
+p.threads = 8;
+
+size_t n = pyseg_slide_frames(num_samples, p.step);
+float *tracks = malloc(n * PYSEG_TRACKS * sizeof(float));
+pyseg_slide_s16(samples, num_samples, &p, tracks);
+
+pyseg_segment segs[256];
+size_t count = pyseg_binarize(tracks, n, PYSEG_TRACK_SPEECH, NULL, segs, 256);
+```
+
+`pyseg_slide` covers the recording with overlapping windows and averages them
+exactly as `pyannote.audio.Inference` does. `pyseg_binarize` turns one track
+into segments the way `pyannote.audio.utils.signal.Binarize` does.
+
+No allocation, syscall or lock happens inside `pyseg_process_*`. An engine is
+not thread-safe and is not meant to be: `pyseg_slide` gives each thread its
+own, and so should any other caller.
+
+### What this does not do
+
+Speaker identities hold only inside a single ten-second window. Two
+overlapping windows may number the same person differently, so joining windows
+into whole-recording speakers needs a speaker embedding model and clustering,
+which is a separate model and outside this library. Speech activity,
+overlapped speech and the per-frame speaker count do carry across windows, and
+those are what `pyseg_slide` aggregates.
+
+The model is 16 kHz mono only, and the window length is fixed at ten seconds
+because the network was trained that way; per-channel statistics are computed
+over the whole window, so it also cannot be run as a stream.
+
+## Layout
+
+| path | content |
+|---|---|
+| `src/pyseg.h` | the public interface |
+| `src/pyseg.c` | the forward pass |
+| `src/kernels.c` | the one dense primitive, scalar and SIMD |
+| `src/weights.c` | weight blob reader |
+| `src/slide.c` | sliding window, aggregation, binarization |
+| `tools/reference.py` | numpy forward pass, the executable specification |
+| `tools/pipeline.py` | the aggregation reference |
+| `tools/export_weights.py` | checkpoint to `pyseg_weights.bin` |
+| `tools/embed.py` | blob to C source, standard library only |
+| `tools/verify_cli.py` | the built binary against pyannote's pipelines |
+
+## Regenerating the fixtures
+
+The parity tests need captured references, which need the Python environment:
+
+```sh
+uv venv --python 3.12 .venv
+VIRTUAL_ENV=.venv uv pip install "pyannote.audio>=3.1" numpy soundfile
+make weights # only after a layout change; the blob is checked in
+make golden # capture reference outputs
+make test
+make verify # end to end against pyannote's own pipelines
+```
+
+`make golden` needs a 16 kHz WAV at `testdata_aepyx.wav`; override with
+`AUDIO=...`.
+
+## Licence
+
+The code here is yours to license. The model weights are
+`pyannote/segmentation-3.0`, MIT, by Alexis Plaquet and Hervé Bredin; see the
+[model card](https://hf.co/pyannote/segmentation-3.0) and the
+[paper](https://www.isca-speech.org/archive/interspeech_2023/plaquet23_interspeech.html).