diff options
| author | Onur Gündüz <ogunduz@gmail.com> | 2026-06-20 20:36:55 +0200 |
|---|---|---|
| committer | Onur Gündüz <ogunduz@gmail.com> | 2026-06-21 06:00:26 +0200 |
| commit | 6652fcbc653d84e75cb5c08c5f5edf52263ed50f (patch) | |
| tree | 1ccffa48e1dc86c59c61dcd4672ffedcf1e0c873 /tests/test-vectors/wordpiece/tokenizer_config.json | |
| parent | dbdf714bb57578431924e5d069089bdfd9f4c604 (diff) | |
| download | ffwd-6652fcbc653d84e75cb5c08c5f5edf52263ed50f.tar.gz ffwd-6652fcbc653d84e75cb5c08c5f5edf52263ed50f.zip | |
tests: upgrade unit tests and add integration/parity tests
Diffstat (limited to 'tests/test-vectors/wordpiece/tokenizer_config.json')
| -rw-r--r-- | tests/test-vectors/wordpiece/tokenizer_config.json | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/tests/test-vectors/wordpiece/tokenizer_config.json b/tests/test-vectors/wordpiece/tokenizer_config.json new file mode 100644 index 0000000..dd43f9a --- /dev/null +++ b/tests/test-vectors/wordpiece/tokenizer_config.json @@ -0,0 +1 @@ +{"do_lower_case": true} |
