aboutsummaryrefslogtreecommitdiff
path: root/src/tokenizer_bpe.c (unfollow)
Commit message (Expand)AuthorFilesLines
2026-06-21all: split Makefile and reorg library/toolsOnur Gündüz1-2018/+0
2026-06-16chore: move sources to src/ with proper names, drop qwen_ prefixOnur Gündüz1-43/+43
2026-06-15chore: run clang-formatOnur Gündüz1-368/+630
2026-06-14tests: tokenizer decode/NFC/contraction/loader-edge coverageOnur Gündüz1-13/+2
2026-06-14server: resolve contextual separator from vocab, test contextual pathOnur Gündüz1-0/+6
2026-06-14tokenizer: ASCII class table for pretokenizer codepoint checksOnur Gündüz1-9/+19
2026-06-14tokenizer: integer-pair BPE merging with linked-list symbolsOnur Gündüz1-5/+233
2026-06-14kernels: drop asr_ prefix namingOnur Gündüz1-2/+2
2026-06-14chore: address static analyzer warningsOnur Gündüz1-10/+41
2026-06-14tokenizer: add reusable encode workspaceOnur Gündüz1-0/+427
2026-06-14tokenizer: reduce merge allocationOnur Gündüz1-10/+41
2026-06-14tokenizer: refactor encoding pipeline to support pre-tokenizationOnur Gündüz1-5/+397
2026-03-28Refactor and enhance pplx_embed and pplx_embed_mlx for improved model supportOnur Gündüz1-8/+0
2026-03-28InitOnur Gündüz1-0/+650