<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ffwd/wordpiece_tokenizer.c, branch master</title>
<subtitle>Fastforward embedding inference engine</subtitle>
<id>https://git.onurgunduz.com/ffwd/atom/wordpiece_tokenizer.c?h=master</id>
<link rel='self' href='https://git.onurgunduz.com/ffwd/atom/wordpiece_tokenizer.c?h=master'/>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/'/>
<updated>2026-06-16T14:27:47Z</updated>
<entry>
<title>chore: move sources to src/ with proper names, drop qwen_ prefix</title>
<updated>2026-06-16T14:27:47Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-16T14:27:47Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=d1ba1ebc06ef0d62ca932388fae83e6992594719'/>
<id>urn:sha1:d1ba1ebc06ef0d62ca932388fae83e6992594719</id>
<content type='text'>
</content>
</entry>
<entry>
<title>tokenizer: Unicode case-fold and accent strip for multilingual WordPiece</title>
<updated>2026-06-16T06:27:41Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-16T06:27:41Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=187e5e8ad924cde6436ed799a0b8f3a7e0ed2efd'/>
<id>urn:sha1:187e5e8ad924cde6436ed799a0b8f3a7e0ed2efd</id>
<content type='text'>
</content>
</entry>
<entry>
<title>tokenizer: full-Unicode punctuation and control classification</title>
<updated>2026-06-16T06:18:12Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-16T06:18:12Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=35400f3a6b3b38d86eba4ab99a2984f584b4c961'/>
<id>urn:sha1:35400f3a6b3b38d86eba4ab99a2984f584b4c961</id>
<content type='text'>
</content>
</entry>
<entry>
<title>tokenizer: add WordPiece backend for BERT-family models</title>
<updated>2026-06-15T12:46:04Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-15T12:46:04Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=8e8836796e7dd22820c820c0916e83ecdbf38368'/>
<id>urn:sha1:8e8836796e7dd22820c820c0916e83ecdbf38368</id>
<content type='text'>
</content>
</entry>
</feed>
