aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorOnur Gündüz <ogunduz@gmail.com>2026-06-17 02:55:07 +0200
committerOnur Gündüz <ogunduz@gmail.com>2026-06-17 02:55:07 +0200
commitc45ea7fc7b175a5995a8120f504063388242cda9 (patch)
treeb9a6a83e6cd6ac82b7a7b2efb96c750280311de2
parent4f743e18aecda8f5ebf25b400452bd4c3409e4aa (diff)
downloadffwd-c45ea7fc7b175a5995a8120f504063388242cda9.tar.gz
ffwd-c45ea7fc7b175a5995a8120f504063388242cda9.zip
chore: document XLM-R models, F16 weights, and config-driven pooling
-rw-r--r--README.md56
1 files changed, 32 insertions, 24 deletions
diff --git a/README.md b/README.md
index d3a7eaf..a2cdada 100644
--- a/README.md
+++ b/README.md
@@ -3,31 +3,35 @@
`embed.c` implements inference for several text embedding model families —
[pplx-embed](https://huggingface.co/collections/perplexity-ai/pplx-embed),
[Qwen3-Embedding](https://huggingface.co/collections/Qwen/qwen3-embedding),
-GTE-Qwen2, and BERT/BGE encoders. It serves them through an
-OpenAI/Perplexity-compatible HTTP API and runs on CPU, Apple MLX, and NVIDIA
-CUDA. See [Supported models](#supported-models).
+GTE-Qwen2, BERT/BGE encoders, and multilingual XLM-R encoders (multilingual-E5,
+Snowflake Arctic). It serves them through an OpenAI/Perplexity-compatible HTTP
+API and runs on CPU, Apple MLX, and NVIDIA CUDA. See
+[Supported models](#supported-models).
## Supported models
The server registers these model IDs out of the box; pass any of them as
`--model ID=DIR`, loading a Hugging Face safetensors checkpoint from `DIR`.
-Every model runs on all three backends, and `dimensions` truncates
-Matryoshka-trained vectors (pplx-embed down to 128, Qwen3-Embedding down to 32).
+Weights load directly from F32, BF16, or F16 safetensors. Every model runs on
+all three backends, and `dimensions` truncates Matryoshka-trained vectors
+(pplx-embed down to 128, Qwen3-Embedding down to 32, arctic-embed down to 256).
**Pooled embeddings** — one vector per text, via `POST /v1/embeddings`:
-| Model ID | Family | Dim | Pooling | Output |
-| ----------------------- | --------------- | ---- | ---------- | ---------------------- |
-| pplx-embed-v1-0.6b | pplx-embed | 1024 | mean | int8, unnormalized |
-| pplx-embed-v1-4b | pplx-embed | 2560 | mean | int8, unnormalized |
-| Qwen3-Embedding-0.6B | Qwen3-Embedding | 1024 | last token | float32, L2-normalized |
-| Qwen3-Embedding-4B | Qwen3-Embedding | 2560 | last token | float32, L2-normalized |
-| Qwen3-Embedding-8B | Qwen3-Embedding | 4096 | last token | float32, L2-normalized |
-| gte-Qwen2-1.5B-instruct | GTE-Qwen2 | 1536 | last token | float32, L2-normalized |
-| all-MiniLM-L6-v2 | BERT (MiniLM) | 384 | mean | float32, L2-normalized |
-| bge-small-en-v1.5 | BERT (BGE) | 384 | CLS | float32, L2-normalized |
-| bge-base-en-v1.5 | BERT (BGE) | 768 | CLS | float32, L2-normalized |
-| bge-large-en-v1.5 | BERT (BGE) | 1024 | CLS | float32, L2-normalized |
+| Model ID | Family | Dim | Pooling | Output |
+| ------------------------------ | --------------- | ---- | ---------- | ---------------------- |
+| pplx-embed-v1-0.6b | pplx-embed | 1024 | mean | int8, unnormalized |
+| pplx-embed-v1-4b | pplx-embed | 2560 | mean | int8, unnormalized |
+| Qwen3-Embedding-0.6B | Qwen3-Embedding | 1024 | last token | float32, L2-normalized |
+| Qwen3-Embedding-4B | Qwen3-Embedding | 2560 | last token | float32, L2-normalized |
+| Qwen3-Embedding-8B | Qwen3-Embedding | 4096 | last token | float32, L2-normalized |
+| gte-Qwen2-1.5B-instruct | GTE-Qwen2 | 1536 | last token | float32, L2-normalized |
+| all-MiniLM-L6-v2 | BERT (MiniLM) | 384 | mean | float32, L2-normalized |
+| bge-small-en-v1.5 | BERT (BGE) | 384 | CLS | float32, L2-normalized |
+| bge-base-en-v1.5 | BERT (BGE) | 768 | CLS | float32, L2-normalized |
+| bge-large-en-v1.5 | BERT (BGE) | 1024 | CLS | float32, L2-normalized |
+| multilingual-e5-large-instruct | XLM-R (E5) | 1024 | mean | float32, L2-normalized |
+| snowflake-arctic-embed-l-v2.0 | XLM-R (Arctic) | 1024 | CLS | float32, L2-normalized |
**Contextual embeddings** — one vector per document chunk, each embedded with
its document as context, via `POST /v1/contextualizedembeddings`:
@@ -45,10 +49,14 @@ its document as context, via `POST /v1/contextualizedembeddings`:
| pplx-embed-v1-late-0.6b | 128 | MaxSim rerank score |
The engine handles three transformer blocks — the Qwen3 block (pplx-embed and
-Qwen3-Embedding), the Qwen2 block (GTE-Qwen2), and the BERT block (MiniLM, BGE)
-— and selects the tokenizer from the model files (byte-level BPE for the Qwen
-families, WordPiece for BERT). pplx-embed vectors are unnormalized int8 (rank
-them by cosine similarity); the other families emit L2-normalized float32.
+Qwen3-Embedding), the Qwen2 block (GTE-Qwen2), and the BERT/RoBERTa block (MiniLM,
+BGE, and the XLM-R multilingual encoders E5 and Arctic) — and selects the
+tokenizer from the model files: byte-level BPE for the Qwen families, WordPiece
+for BERT, and SentencePiece (Unigram) for XLM-R. Pooling and L2-normalization
+are read from the checkpoint's Sentence Transformers config
+(`1_Pooling/config.json` and `modules.json`), so a model's pooling mode does not
+have to be hardcoded. pplx-embed vectors are unnormalized int8 (rank them by
+cosine similarity); the other families emit L2-normalized float32.
## Building
@@ -185,9 +193,9 @@ Returns a list of embeddings, one per input text:
`encoding_format` follows the model family:
-- The OpenAI-API models (Qwen3-Embedding, GTE-Qwen2, MiniLM, BGE) default to
- `float` (the true float32 vector) and also accept `base64` (base64 of
- float32).
+- The OpenAI-API models (Qwen3-Embedding, GTE-Qwen2, MiniLM, BGE,
+ multilingual-E5, Snowflake Arctic) default to `float` (the true float32 vector)
+ and also accept `base64` (base64 of float32).
- pplx-embed (Perplexity-compatible) defaults to `base64_int8` and also accepts
`base64_binary` and `float` (the decoded int8 view).