diff options
| author | Onur Gündüz <ogunduz@gmail.com> | 2026-07-14 02:38:03 +0200 |
|---|---|---|
| committer | Onur Gündüz <ogunduz@gmail.com> | 2026-07-14 02:38:03 +0200 |
| commit | be9162fe19d29c56419dfbcc642283065765c671 (patch) | |
| tree | fce37f8afff0457655e81d93005411ae5e04d59a | |
| parent | 7ad23531777042e88f197cc400651cb80ae83723 (diff) | |
| download | ffwd-be9162fe19d29c56419dfbcc642283065765c671.tar.gz ffwd-be9162fe19d29c56419dfbcc642283065765c671.zip | |
chore: update README
| -rw-r--r-- | README.md | 26 |
1 files changed, 0 insertions, 26 deletions
@@ -53,32 +53,6 @@ Once the dependencies are in place, build the project by running: make # or make gpu for GPU build ``` -## Performance - -When comparing ffwd with other open-source solutions, the main differentiator in -terms of speed is attention performance. - -ffwd implements custom FlashAttention-style CUDA kernels from scratch (optimized -for Blackwell and Ada architectures), without relying on libraries such as -CUTLASS or cuDNN. - -It is very fast and competitive at short and medium context lengths, up to -around 16k tokens for Qwen3, mainly because the work surrounding the attention -kernel is kept extremely low-overhead. - -However, once pushed toward saturation, especially with a single long context or -very large batches, the bottleneck becomes attention-kernel throughput. -Unsurprisingly, this is where engines such as TEI, which leverages CuTe-based -[Flash Attention](https://github.com/HazyResearch/flash-attention), began to -outperform ffwd in my benchmarks. - -That said, the v0.2.0 release was also a useful reality check and a good -learning experience: it is very hard to beat machine-generated kernels, no -matter how many sub-agents you fork to benchmark and hypertune your code across -architectures 😄 At some point, you either end up writing a DSL or moving toward -heavily templated code, which is exactly what these libraries are designed to -provide. - ## Acknowledgements - This project started as a fork of |
