diff options
| author | Onur Gündüz <ogunduz@gmail.com> | 2026-06-14 11:58:38 +0200 |
|---|---|---|
| committer | Onur Gündüz <ogunduz@gmail.com> | 2026-06-14 11:58:38 +0200 |
| commit | 769cb9a45807f15968c170e3210e95269d460420 (patch) | |
| tree | ac15cede711e7d4b4d053bc887d8fbd0e01e82ea | |
| parent | 347ba945ec5bea9326c474561f7ddad21b636651 (diff) | |
| download | ffwd-769cb9a45807f15968c170e3210e95269d460420.tar.gz ffwd-769cb9a45807f15968c170e3210e95269d460420.zip | |
docs: update README with the model support and usage examples
| -rw-r--r-- | README.md | 65 |
1 files changed, 62 insertions, 3 deletions
@@ -1,9 +1,8 @@ # embed.c -`embed.c` implements inference for Perplexity +`embed.c` implements inference for [pplx-embed](https://huggingface.co/collections/perplexity-ai/pplx-embed) -dense/contextual text embedding models plus the recently added MaxSim -late-interaction model. +dense/contextual and late-interaction text embedding models. Supported platforms are CUDA/cuBLAS on Linux/NVIDIA, Apple Silicon MLX, and CPU/BLAS. @@ -20,3 +19,63 @@ CPU/BLAS. - Run `make gpu` to auto-select the GPU backend (MLX on macOS, CUDA on Linux). - Run `make cpu` to build the CPU backend only (BLAS: Accelerate on macOS, OpenBLAS on Linux). + +## Usage + +### embed + +Pass one text to get its embedding, two or more to get a cosine similarity +matrix: + +```bash +./embed -d ./model "What is the capital of France?" +./embed -d ./model "What is the capital of France?" \ + "Paris is the capital of France." \ + "Berlin is the capital of Germany." +``` + +Pipe in lines and use `--stream` to get one JSON embedding per line: + +```bash +cat texts.txt | ./embed -d ./model --stream -b 8 +# {"embedding":[...],"dim":2048,"tokens":7,"ms":12.3,"workspace_bytes":1048576} +``` + +Without `--stream`, reading from stdin accumulates all lines then prints the +similarity matrix. + +Use `--mlx` or `--cuda` to select a GPU backend (build-dependent). Default is +CPU. `-t` sets thread count, `-b` sets batch size, `-v` enables verbose output. + +### embed-server + +Serves the HTTP API. Takes one or more `--model ID=DIR` pairs: + +```bash +./embed-server \ + --model pplx-embed-v1-0.6b=./model \ + --port 8000 +``` + +Perplexity API compatible endpoints: + +- `POST /v1/embeddings` — pooled embeddings +- `POST /v1/contextualizedembeddings` — contextual document embeddings + +Late-interaction reranking endpoint: + +- `POST /v1/rerank` — late-interaction MaxSim reranking + +Example: + +```bash +./embed-server --model pplx-embed-v1-late-0.6b=./model +# can be tested with: +curl -s http://127.0.0.1:8000/v1/rerank \ + -H 'Content-Type: application/json' \ + -d '{ + "model":"pplx-embed-v1-late-0.6b", + "query":"scientific curiosity", + "documents":["Scientists explore from curiosity.","SQLite stores data."] + }' +``` |
