<feed xmlns='http://www.w3.org/2005/Atom'>
<title>ffwd/tools/gen_wordpiece_unicode.py, branch master</title>
<subtitle>Fastforward embedding inference engine</subtitle>
<id>https://git.onurgunduz.com/ffwd/atom/tools/gen_wordpiece_unicode.py?h=master</id>
<link rel='self' href='https://git.onurgunduz.com/ffwd/atom/tools/gen_wordpiece_unicode.py?h=master'/>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/'/>
<updated>2026-06-21T04:00:26Z</updated>
<entry>
<title>devtools: add scripts for unicode generation, model  inspection,  embedding ranking, and more</title>
<updated>2026-06-21T04:00:26Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-20T18:31:06Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=dbdf714bb57578431924e5d069089bdfd9f4c604'/>
<id>urn:sha1:dbdf714bb57578431924e5d069089bdfd9f4c604</id>
<content type='text'>
</content>
</entry>
<entry>
<title>tools: point wordpiece unicode generator at src/tokenizer_unicode.h</title>
<updated>2026-06-16T14:51:35Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-16T14:51:35Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=51a716cc7bc555d765cf4ab97c6ee4935683d129'/>
<id>urn:sha1:51a716cc7bc555d765cf4ab97c6ee4935683d129</id>
<content type='text'>
</content>
</entry>
<entry>
<title>tokenizer: Unicode case-fold and accent strip for multilingual WordPiece</title>
<updated>2026-06-16T06:27:41Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-16T06:27:41Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=187e5e8ad924cde6436ed799a0b8f3a7e0ed2efd'/>
<id>urn:sha1:187e5e8ad924cde6436ed799a0b8f3a7e0ed2efd</id>
<content type='text'>
</content>
</entry>
<entry>
<title>tokenizer: full-Unicode punctuation and control classification</title>
<updated>2026-06-16T06:18:12Z</updated>
<author>
<name>Onur Gündüz</name>
<email>ogunduz@gmail.com</email>
</author>
<published>2026-06-16T06:18:12Z</published>
<link rel='alternate' type='text/html' href='https://git.onurgunduz.com/ffwd/commit/?id=35400f3a6b3b38d86eba4ab99a2984f584b4c961'/>
<id>urn:sha1:35400f3a6b3b38d86eba4ab99a2984f584b4c961</id>
<content type='text'>
</content>
</entry>
</feed>
