aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
Diffstat (limited to 'README.md')
-rw-r--r--README.md138
1 files changed, 103 insertions, 35 deletions
diff --git a/README.md b/README.md
index c5351a1..03b05f8 100644
--- a/README.md
+++ b/README.md
@@ -1,12 +1,12 @@
-# pplx-embed
+# pplx-embed.c
-Pure C inference for [pplx-embed](https://research.perplexity.ai/articles/pplx-embed-state-of-the-art-embedding-models-for-web-scale-retrieval) embedding models by [Perplexity AI](https://www.perplexity.ai/). Supports safetensors and hardware-accelerated execution (BLAS/mlx-c) for generating float32 vectors.
+C implementation of the inference pipeline for [pplx-embed](https://research.perplexity.ai/articles/pplx-embed-state-of-the-art-embedding-models-for-web-scale-retrieval) embedding models.
Based on math kernels from [antirez](https://github.com/antirez)'s [qwen-asr](https://github.com/antirez/qwen-asr).
-## Supported models
+### Supported models
-All [pplx-embed](https://huggingface.co/collections/perplexity-ai/pplx-embed-v1-68126b2e7cc987cd820ebbbd) models are supported.
+All [pplx-embed](https://huggingface.co/collections/perplexity-ai/pplx-embed) models are supported.
| Model | Dims | Params | Size |
|---|---|---|---|
@@ -15,75 +15,140 @@ All [pplx-embed](https://huggingface.co/collections/perplexity-ai/pplx-embed-v1-
| [pplx-embed-context-v1-0.6B](https://huggingface.co/perplexity-ai/pplx-embed-context-v1-0.6B) | 1024 | 0.6B | ~2.5 GB |
| [pplx-embed-context-v1-4B](https://huggingface.co/perplexity-ai/pplx-embed-context-v1-4B) | 2560 | 4B | ~16 GB |
-## Quickstart
+## Building
```bash
-make mlx # Apple Silicon (fastest)
-# or: make blas # CPU with BLAS (any platform)
+make blas # CPU with BLAS (Accelerate on macOS, OpenBLAS on Linux)
+make mlx # Apple Silicon GPU via mlx-c (recommended on Apple Silicon)
+make debug # Debug build with AddressSanitizer
+```
+
+* For `make mlx`, install MLX first: `brew install mlx mlx-c`
+* For `make blas` on Linux: `sudo apt install libopenblas-dev`
+
+## Usage
+
+```
+./pplx_embed -d <model_dir> [options] [text...]
+
+Options:
+ -d <dir> Model directory (required)
+ --mlx Use Apple MLX GPU backend
+ --daemon Read texts from stdin, write JSON embeddings to stdout
+ -t <n> CPU thread count (default: all cores)
+ -e Print raw embedding vectors (with multiple texts)
+ -v Verbose (-vv for debug)
+```
+
+**Single text**: prints the embedding vector as space-separated floats.
+```bash
./pplx_embed -d /path/to/model "query: what is the capital of France?"
# prints 1024 (or 2560) floats to stdout
+```
+
+**Multiple texts**: prints a cosine similarity matrix.
+```bash
./pplx_embed -d /path/to/model \
"query: what is the capital of France?" \
"document: Paris is the capital of France." \
"document: Berlin is the capital of Germany."
-# prints cosine similarity matrix
+# prints a NxN similarity matrix to stdout
```
-### Daemon mode
-
-Start the process once and feed texts through stdin. The model stays loaded in memory.
+**Daemon mode** (`--daemon`): reads one text per line from stdin, writes one JSON object per line to stdout. EOF or empty line exits.
```bash
-# One embedding per line, output as JSON
./pplx_embed -d /path/to/model --daemon --mlx <<'EOF'
query: what is the capital of France?
document: Paris is the capital of France.
document: Berlin is the capital of Germany.
-document: Istanbul is the capital of Germany.
EOF
```
-Output (one JSON object per line):
+Output is a line-delimited stream of JSON objects with the embedding vector, dimension, token count, and latency in milliseconds:
-```json
+```
{"embedding":[0.0231,-0.0412,...], "dim":1024, "tokens":9, "ms":15.2}
{"embedding":[0.0187,-0.0339,...], "dim":1024, "tokens":9, "ms":14.8}
...
```
-## Building
+## C API
-```bash
-make blas # CPU with BLAS (Accelerate on macOS, OpenBLAS on Linux)
-make mlx # Apple Silicon GPU via mlx-c (recommended on Apple Silicon)
-make debug # Debug build with AddressSanitizer
-```
+**Single text:**
-For `make mlx`, install MLX first: `brew install mlx mlx-c`
+```c
+#include "pplx_embed.h"
+#include "qwen_asr_tokenizer.h"
-For `make blas` on Linux: `sudo apt install libopenblas-dev`
+/* Load model */
+pplx_ctx_t *ctx = pplx_load("/path/to/model");
+int dim = ctx->config.hidden_size; /* 1024 or 2560 */
-## Usage
+/* Load tokenizer */
+qwen_tokenizer_t *tok = qwen_tokenizer_load("/path/to/model/vocab.json");
+
+/* Embed a text */
+int n_tokens;
+int *ids = qwen_tokenizer_encode(tok, "query: what is AI?", &n_tokens);
+float *emb = pplx_embed(ctx, ids, n_tokens);
+/* emb is a unit-length float[dim] vector. Use it, then free it. */
+store_in_database(emb, dim);
+free(emb);
+free(ids);
+
+/* Cleanup */
+pplx_free(ctx);
+qwen_tokenizer_free(tok);
```
-./pplx_embed -d <model_dir> [options] [text...]
-Options:
- -d <dir> Model directory (required)
- --mlx Use Apple MLX GPU backend
- --daemon Read texts from stdin, write JSON embeddings to stdout
- -t <n> CPU thread count (default: all cores)
- -e Print raw embedding vectors (with multiple texts)
- -v Verbose (-vv for debug)
+**Processing a batch of texts:**
+
+```c
+pplx_ctx_t *ctx = pplx_load(model_dir);
+qwen_tokenizer_t *tok = qwen_tokenizer_load(vocab_path);
+int dim = ctx->config.hidden_size;
+
+const char *texts[] = {
+ "document: Paris is the capital of France.",
+ "document: The Eiffel Tower is in Paris.",
+ "document: Berlin is the capital of Germany.",
+};
+
+for (int i = 0; i < 3; i++) {
+ int n;
+ int *ids = qwen_tokenizer_encode(tok, texts[i], &n);
+ float *emb = pplx_embed(ctx, ids, n);
+
+ /* emb[0..dim-1] is the L2-normalized embedding */
+ printf("text %d: %d tokens, first dim = %f\n", i, n, emb[0]);
+
+ free(emb);
+ free(ids);
+}
+
+pplx_free(ctx);
+qwen_tokenizer_free(tok);
```
-**Single text**: prints the embedding vector as space-separated floats.
+**Full per-token output** (for contextual/late-chunking models):
-**Multiple texts**: prints a cosine similarity matrix.
+```c
+float *all = pplx_forward(ctx, ids, n_tokens);
+/* all[i * dim .. (i+1) * dim - 1] = hidden state for token i */
+/* Do your own pooling (e.g., mean-pool a chunk range) */
+free(all);
+```
-**Daemon mode** (`--daemon`): reads one text per line from stdin, writes one JSON object per line to stdout. EOF or empty line exits.
+**Cosine similarity:**
+
+```c
+float sim = pplx_cosine_similarity(emb_a, emb_b, dim);
+/* For L2-normalized vectors this is just the dot product */
+```
## Benchmark results
@@ -131,3 +196,6 @@ Options:
* Load times: C/BLAS loads fastest (mmap, no conversion). MLX has a one-time Metal kernel compilation cost. PyTorch's 12-second load for 4B is because the first inference faults in the entire mmap'd file.
* Accuracy is identical (cosine sim = 1.000000, max diff < 2e-6) between PyTorch and C implementations.
+## License
+
+MIT