aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorOnur Gündüz <ogunduz@gmail.com>2026-03-28 23:38:06 +0100
committerOnur Gündüz <ogunduz@gmail.com>2026-03-28 23:38:06 +0100
commit353aa5394a11ef8a9c69e73338b95a3f7eac5855 (patch)
tree414a8e39d7e19b2704088f8c7d691281625f1394 /README.md
parentc47e900b94d18cca1bf600e58f10dbcc2e34c1f7 (diff)
downloadffwd-353aa5394a11ef8a9c69e73338b95a3f7eac5855.tar.gz
ffwd-353aa5394a11ef8a9c69e73338b95a3f7eac5855.zip
Add benchmark script
Diffstat (limited to 'README.md')
-rw-r--r--README.md46
1 files changed, 46 insertions, 0 deletions
diff --git a/README.md b/README.md
index bbc205a..c5351a1 100644
--- a/README.md
+++ b/README.md
@@ -85,3 +85,49 @@ Options:
**Daemon mode** (`--daemon`): reads one text per line from stdin, writes one JSON object per line to stdout. EOF or empty line exits.
+## Benchmark results
+
+### pplx-embed-v1-0.6b
+
+1024-dim, 28 layers, 2.38 GB
+
+**Inference latency** (median, 5 runs, Apple M4 Pro 12-core):
+
+| Sentence | Tokens | PyTorch | C/BLAS | Speedup | C/MLX | Speedup |
+| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
+| short | 2 | 76.2 ms | 53.9 ms | 1.4x | 15.2 ms | 5.0x |
+| medium | 13 | 75.7 ms | 50.7 ms | 1.5x | 15.4 ms | 4.9x |
+| long | 63 | 97.4 ms | 58.9 ms | 1.7x | 20.0 ms | 4.9x |
+| 256tok | 199 | 193.5 ms | 178.2 ms | 1.1x | 53.9 ms | 3.6x |
+
+**Load/memory**:
+
+| | PyTorch | C/BLAS | C/MLX |
+| :--- | :--- | :--- | :--- |
+| **Load time** | 186 ms | 195 ms | 363 ms |
+| **RSS** | ~2163 MB | ~1737 MB | ~4603 MB |
+
+### pplx-embed-v1-4b
+
+2560-dim, 36 layers, 16.09 GB
+
+| Sentence | Tokens | PyTorch | C/BLAS | Speedup | C/MLX | Speedup |
+| :--- | :--- | :--- | :--- | :--- | :--- | :--- |
+| short | 2 | 462 ms | 402 ms | 1.1x | 97 ms | 4.8x |
+| medium | 13 | 307 ms | 234 ms | 1.3x | 101 ms | 3.0x |
+| long | 63 | 411 ms | 350 ms | 1.2x | 121 ms | 3.4x |
+| 256tok | 199 | 875 ms | 919 ms | 1.0x | 376 ms | 2.3x |
+
+| | PyTorch | C/BLAS | C/MLX |
+| :--- | :--- | :--- | :--- |
+| **Load time** | 12094 ms | 1202 ms | 6134 ms |
+| **RSS** | ~14253 MB | ~13902 MB | ~15483 MB |
+
+**Summary**:
+
+* C/MLX is up to **5x** faster than PyTorch on Apple M4 Pro, and C/BLAS is up to **1.7x** faster.
+* C/BLAS uses ~20% less memory than PyTorch (1737 vs 2163 MB for 0.6B) because it mmap's weights directly instead of copying them into Python objects.
+* MLX RSS is higher (~4.6 GB for 0.6B) because Metal duplicates weight data into GPU-accessible memory. This is the cost of GPU speed.
+* Load times: C/BLAS loads fastest (mmap, no conversion). MLX has a one-time Metal kernel compilation cost. PyTorch's 12-second load for 4B is because the first inference faults in the entire mmap'd file.
+* Accuracy is identical (cosine sim = 1.000000, max diff < 2e-6) between PyTorch and C implementations.
+