diff options
| author | Onur Gündüz <ogunduz@gmail.com> | 2026-06-27 00:58:05 +0200 |
|---|---|---|
| committer | Onur Gündüz <ogunduz@gmail.com> | 2026-06-27 01:03:41 +0200 |
| commit | af55803d0dff9eb692c760c8d486e736c945061d (patch) | |
| tree | 0875a1112bf98304e445e0544c4309670d6f4769 /tests/test_bpe.c | |
| parent | 0f861b7f3098750ddee4f6570f3be228d1da5320 (diff) | |
| download | ffwd-af55803d0dff9eb692c760c8d486e736c945061d.tar.gz ffwd-af55803d0dff9eb692c760c8d486e736c945061d.zip | |
chore: clean up comment formatting and wording
Diffstat (limited to 'tests/test_bpe.c')
| -rw-r--r-- | tests/test_bpe.c | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/tests/test_bpe.c b/tests/test_bpe.c index 1674b4e..02e5078 100644 --- a/tests/test_bpe.c +++ b/tests/test_bpe.c @@ -1,8 +1,9 @@ -/* tests/test_tokenizer.c - hermetic byte-level BPE tokenizer tests. +/* Hermetic byte-level BPE tokenizer tests. * Synthesizes a complete GPT-2-style vocab.json (all 256 byte tokens plus * merged tokens) and merges.txt, then checks exact token ids through the * integer-pair fast path, and that dropping the integer tables (forcing the - * string-based fallback) produces identical ids. Runs via `make test`. */ + * string-based fallback) produces identical ids. Runs via make test. + */ #include "bpe.h" #include "tok_fixture.h" |
