<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ffwd/libffwd/cuda.cu, branch v0.3.0</title>
<subtitle>Fastforward embedding inference engine</subtitle>
<id>https://git.onurgunduz.com/ffwd/atom/libffwd/cuda.cu?h=v0.3.0</id>
<link rel='self' href='https://git.onurgunduz.com/ffwd/atom/libffwd/cuda.cu?h=v0.3.0'/>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/'/>
<updated>2026-07-02T18:18:05Z</updated>
<entry>
<title>cuda: default cuDNN attention to Qwen only, BERT opt-in</title>
<updated>2026-07-02T18:18:05Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-07-02T18:18:05Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=4d79e6337c05d0a768de80eeb164f1de3b1624d3'/>
<id>urn:sha1:4d79e6337c05d0a768de80eeb164f1de3b1624d3</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: add cuDNN fused attention path with runtime fallback</title>
<updated>2026-07-02T17:44:08Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-07-02T17:44:08Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=6a10d3577232f1fd646a20f22a4a1ab4ec29f28a'/>
<id>urn:sha1:6a10d3577232f1fd646a20f22a4a1ab4ec29f28a</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: skip Qwen flash softmax rescale when running max is stable</title>
<updated>2026-06-25T23:38:00Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T23:38:00Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=0f861b7f3098750ddee4f6570f3be228d1da5320'/>
<id>urn:sha1:0f861b7f3098750ddee4f6570f3be228d1da5320</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: load Qwen flash V with ldmatrix.trans instead of gather</title>
<updated>2026-06-25T22:52:05Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T22:52:05Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=733ec2cb70551a85032b44281623d1b3b43da511'/>
<id>urn:sha1:733ec2cb70551a85032b44281623d1b3b43da511</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: skip O(L^2) score buffer for qwen flash path (enables 32k)</title>
<updated>2026-06-25T18:44:28Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T18:44:28Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=6a9aa3302362e7d129b804cd621d39a1c1e7c43d'/>
<id>urn:sha1:6a9aa3302362e7d129b804cd621d39a1c1e7c43d</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: stage compact GQA K/V in qwen attention, halve the expand</title>
<updated>2026-06-25T18:06:54Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-25T18:06:54Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=c09ea7df78546e6c4ff73fe4e8e66cd5a964ea97'/>
<id>urn:sha1:c09ea7df78546e6c4ff73fe4e8e66cd5a964ea97</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: skip per-warp future key tiles in causal qwen attention</title>
<updated>2026-06-24T22:07:10Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T22:07:10Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=b673eb9a72138b89f1efecf52578020cb43857ff'/>
<id>urn:sha1:b673eb9a72138b89f1efecf52578020cb43857ff</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: double-buffer qwen flash K/V staging with cp.async</title>
<updated>2026-06-24T18:58:05Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T18:58:05Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=ec3966ee8ef5bb694d312d4c0d9db16df19630d5'/>
<id>urn:sha1:ec3966ee8ef5bb694d312d4c0d9db16df19630d5</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: prefetch bert flash QK ahead of softmax to overlap the tensor cores</title>
<updated>2026-06-24T16:37:14Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T16:37:14Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=8d53da182857369e48dc82d8226cc0e212b81e3e'/>
<id>urn:sha1:8d53da182857369e48dc82d8226cc0e212b81e3e</id>
<content type='text'>
</content>
</entry>
<entry>
<title>cuda: read bert flash K directly, drop the shared-memory transpose</title>
<updated>2026-06-24T15:16:45Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-24T15:16:45Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=3bee45859fa41e444d6ba707c4138a2f02a89589'/>
<id>urn:sha1:3bee45859fa41e444d6ba707c4138a2f02a89589</id>
<content type='text'>
</content>
</entry>
</feed>
