aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorOnur Gündüz <ogunduz@gmail.com>2026-06-14 11:58:38 +0200
committerOnur Gündüz <ogunduz@gmail.com>2026-06-14 11:58:38 +0200
commit769cb9a45807f15968c170e3210e95269d460420 (patch)
treeac15cede711e7d4b4d053bc887d8fbd0e01e82ea
parent347ba945ec5bea9326c474561f7ddad21b636651 (diff)
downloadffwd-769cb9a45807f15968c170e3210e95269d460420.tar.gz
ffwd-769cb9a45807f15968c170e3210e95269d460420.zip
docs: update README with the model support and usage examples
-rw-r--r--README.md65
1 files changed, 62 insertions, 3 deletions
diff --git a/README.md b/README.md
index 8f80cb2..977576e 100644
--- a/README.md
+++ b/README.md
@@ -1,9 +1,8 @@
# embed.c
-`embed.c` implements inference for Perplexity
+`embed.c` implements inference for
[pplx-embed](https://huggingface.co/collections/perplexity-ai/pplx-embed)
-dense/contextual text embedding models plus the recently added MaxSim
-late-interaction model.
+dense/contextual and late-interaction text embedding models.
Supported platforms are CUDA/cuBLAS on Linux/NVIDIA, Apple Silicon MLX, and
CPU/BLAS.
@@ -20,3 +19,63 @@ CPU/BLAS.
- Run `make gpu` to auto-select the GPU backend (MLX on macOS, CUDA on Linux).
- Run `make cpu` to build the CPU backend only (BLAS: Accelerate on macOS,
OpenBLAS on Linux).
+
+## Usage
+
+### embed
+
+Pass one text to get its embedding, two or more to get a cosine similarity
+matrix:
+
+```bash
+./embed -d ./model "What is the capital of France?"
+./embed -d ./model "What is the capital of France?" \
+ "Paris is the capital of France." \
+ "Berlin is the capital of Germany."
+```
+
+Pipe in lines and use `--stream` to get one JSON embedding per line:
+
+```bash
+cat texts.txt | ./embed -d ./model --stream -b 8
+# {"embedding":[...],"dim":2048,"tokens":7,"ms":12.3,"workspace_bytes":1048576}
+```
+
+Without `--stream`, reading from stdin accumulates all lines then prints the
+similarity matrix.
+
+Use `--mlx` or `--cuda` to select a GPU backend (build-dependent). Default is
+CPU. `-t` sets thread count, `-b` sets batch size, `-v` enables verbose output.
+
+### embed-server
+
+Serves the HTTP API. Takes one or more `--model ID=DIR` pairs:
+
+```bash
+./embed-server \
+ --model pplx-embed-v1-0.6b=./model \
+ --port 8000
+```
+
+Perplexity API compatible endpoints:
+
+- `POST /v1/embeddings` — pooled embeddings
+- `POST /v1/contextualizedembeddings` — contextual document embeddings
+
+Late-interaction reranking endpoint:
+
+- `POST /v1/rerank` — late-interaction MaxSim reranking
+
+Example:
+
+```bash
+./embed-server --model pplx-embed-v1-late-0.6b=./model
+# can be tested with:
+curl -s http://127.0.0.1:8000/v1/rerank \
+ -H 'Content-Type: application/json' \
+ -d '{
+ "model":"pplx-embed-v1-late-0.6b",
+ "query":"scientific curiosity",
+ "documents":["Scientists explore from curiosity.","SQLite stores data."]
+ }'
+```