diff options
| author | Onur Gündüz <ogunduz@gmail.com> | 2026-06-27 00:58:05 +0200 |
|---|---|---|
| committer | Onur Gündüz <ogunduz@gmail.com> | 2026-06-27 01:03:41 +0200 |
| commit | af55803d0dff9eb692c760c8d486e736c945061d (patch) | |
| tree | 0875a1112bf98304e445e0544c4309670d6f4769 /tests/test_wordpiece.c | |
| parent | 0f861b7f3098750ddee4f6570f3be228d1da5320 (diff) | |
| download | ffwd-af55803d0dff9eb692c760c8d486e736c945061d.tar.gz ffwd-af55803d0dff9eb692c760c8d486e736c945061d.zip | |
chore: clean up comment formatting and wording
Diffstat (limited to 'tests/test_wordpiece.c')
| -rw-r--r-- | tests/test_wordpiece.c | 8 |
1 files changed, 4 insertions, 4 deletions
diff --git a/tests/test_wordpiece.c b/tests/test_wordpiece.c index 289260c..2e95f3e 100644 --- a/tests/test_wordpiece.c +++ b/tests/test_wordpiece.c @@ -1,7 +1,7 @@ -/* tests/test_wordpiece.c - hermetic WordPiece tokenizer checks. A synthetic - * vocab exercises greedy ## matching, [UNK], casing, accent stripping, - * punctuation splitting, CJK char-splitting, special-token resolution, and the - * workspace encode paths. No model files or network needed. */ +/* Hermetic WordPiece tokenizer checks. A synthetic vocab exercises greedy + * ## matching, [UNK], casing, accent stripping, punctuation splitting, CJK char-splitting, + * special-token resolution, and the workspace encode paths. + */ #include "wordpiece.h" |
