<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ffwd/libffwd/cuda.cu, branch v0.2.0</title>
<subtitle>Fastforward embedding inference engine</subtitle>
<id>https://git.onurgunduz.com/ffwd/atom/libffwd/cuda.cu?h=v0.2.0</id>
<link rel='self' href='https://git.onurgunduz.com/ffwd/atom/libffwd/cuda.cu?h=v0.2.0'/>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/'/>
<updated>2026-06-25T23:38:00Z</updated>
<entry>
<title>cuda: skip Qwen flash softmax rescale when running max is stable</title>
<updated>2026-06-25T23:38:00Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T23:38:00Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=0f861b7f3098750ddee4f6570f3be228d1da5320'/>
<id>urn:sha1:0f861b7f3098750ddee4f6570f3be228d1da5320</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: load Qwen flash V with ldmatrix.trans instead of gather</title>
<updated>2026-06-25T22:52:05Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T22:52:05Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=733ec2cb70551a85032b44281623d1b3b43da511'/>
<id>urn:sha1:733ec2cb70551a85032b44281623d1b3b43da511</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: skip O(L^2) score buffer for qwen flash path (enables 32k)</title>
<updated>2026-06-25T18:44:28Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T18:44:28Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=6a9aa3302362e7d129b804cd621d39a1c1e7c43d'/>
<id>urn:sha1:6a9aa3302362e7d129b804cd621d39a1c1e7c43d</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: stage compact GQA K/V in qwen attention, halve the expand</title>
<updated>2026-06-25T18:06:54Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T18:06:54Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=c09ea7df78546e6c4ff73fe4e8e66cd5a964ea97'/>
<id>urn:sha1:c09ea7df78546e6c4ff73fe4e8e66cd5a964ea97</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: skip per-warp future key tiles in causal qwen attention</title>
<updated>2026-06-24T22:07:10Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T22:07:10Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=b673eb9a72138b89f1efecf52578020cb43857ff'/>
<id>urn:sha1:b673eb9a72138b89f1efecf52578020cb43857ff</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: double-buffer qwen flash K/V staging with cp.async</title>
<updated>2026-06-24T18:58:05Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T18:58:05Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=ec3966ee8ef5bb694d312d4c0d9db16df19630d5'/>
<id>urn:sha1:ec3966ee8ef5bb694d312d4c0d9db16df19630d5</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: prefetch bert flash QK ahead of softmax to overlap the tensor cores</title>
<updated>2026-06-24T16:37:14Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T16:37:14Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=8d53da182857369e48dc82d8226cc0e212b81e3e'/>
<id>urn:sha1:8d53da182857369e48dc82d8226cc0e212b81e3e</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: read bert flash K directly, drop the shared-memory transpose</title>
<updated>2026-06-24T15:16:45Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T15:16:45Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=3bee45859fa41e444d6ba707c4138a2f02a89589'/>
<id>urn:sha1:3bee45859fa41e444d6ba707c4138a2f02a89589</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: broaden bert flash kv pipeline to all flash batches</title>
<updated>2026-06-24T14:17:11Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T14:17:11Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=88aed70366f99520a7ed4e86a101a41d33aa4542'/>
<id>urn:sha1:88aed70366f99520a7ed4e86a101a41d33aa4542</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: fix bert reduced-precision ffn gate-up output aliasing its input</title>
<updated>2026-06-24T11:13:44Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T11:13:44Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=e7e9ad4bcda6be5395da521d86f5ced11220ec60'/>
<id>urn:sha1:e7e9ad4bcda6be5395da521d86f5ced11220ec60</id>
<content type='text'>
</content>
</entry>
</feed>
