aboutsummaryrefslogtreecommitdiff
path: root/README.md
diff options
context:
space:
mode:
authorOnur Gündüz <ogunduz@gmail.com>2026-09-29 14:04:04 +0200
committerOnur Gündüz <ogunduz@gmail.com>2026-09-29 14:04:04 +0200
commit6f98fc8cf3f7cc4b9d523d16492a07d3e2aea8c9 (patch)
tree427e45b901aec902ceba72428fe5de9689d3001b /README.md
parentcb1a04331b27cc19445de63cedcd4fd30b8a9016 (diff)
downloadlangdet-6f98fc8cf3f7cc4b9d523d16492a07d3e2aea8c9.tar.gz
langdet-6f98fc8cf3f7cc4b9d523d16492a07d3e2aea8c9.zip
Add CPU reference comparison and repeatable API tests
Diffstat (limited to 'README.md')
-rw-r--r--README.md40
1 files changed, 36 insertions, 4 deletions
diff --git a/README.md b/README.md
index bdab9bc..23a6612 100644
--- a/README.md
+++ b/README.md
@@ -8,8 +8,8 @@ runtime is required for inference.
## Build and run
Use an Apple Silicon Mac with Xcode Command Line Tools installed
-(`xcode-select --install` if needed). Python 3 is required only for downloading
-the model.
+(`xcode-select --install` if needed). Python 3 is used for downloading the model
+and running the optional reference checks, not for native inference.
```sh
python3 download_model.py
@@ -108,6 +108,37 @@ Downloads go to `.part` files and become final files only after verification.
Interrupted files restart on the next run; they do not resume from their previous
byte offset.
+## Tests
+
+With the model downloaded, install `uv` and have Python 3.10 through 3.13
+available, then run:
+
+```sh
+make test
+```
+
+Use `make test MODEL=/path/to/model PYTHON=python3.12` to select a different
+checkpoint directory or Python executable. The first run needs internet access
+for the test-only Python packages. `uv` manages the pinned dependencies declared
+in `tools/verify.py`; its cache stays in `.uv-cache/` inside this project.
+The Transformers reference uses only the local model files and runs offline on
+CPU in F32, with dropout disabled and eager attention.
+
+The target builds and runs:
+
+- `tools/test_api.c`: repeated predictions, score normalization, result ownership,
+ empty input, truncation, invalid arguments, error buffers, and detector lifecycle.
+- `tools/token_ids.c`: a test-only adapter that returns the native token IDs.
+- `tools/verify.py`: exact token-ID comparison and all 20 language scores against
+ Transformers on 28 inputs. These cover the 20 languages, empty and whitespace
+ input, Unicode normalization, multiple lines, and lengths below, at, and above
+ the 512-token limit. Scores must differ by no more than `1e-4` in absolute value,
+ and the predicted labels must match, with both one and four CPU kernel threads.
+
+The Python checks also verify stdin behavior, invalid CLI arguments, NUL rejection,
+and failures for missing or invalid configuration, weights, and tokenizer files.
+No Python package is linked into or required by the native library.
+
## Verification status
- The executable builds and runs as native macOS ARM64, linked to Accelerate and
@@ -120,5 +151,6 @@ byte offset.
- Standalone C and C++ applications link using only `langdet.h` and the static
archive. Prediction results, error returns, and detector lifecycle checks pass.
- The downloaded checkpoint files match Hugging Face's sizes and hashes.
-- Numerical equivalence with Hugging Face Transformers has not been established.
- These smoke checks are not an accuracy benchmark or exhaustive tokenizer test.
+- The reference comparisons pass on the 28 test inputs: token IDs match exactly,
+ labels agree, and probability differences stay below the `1e-4` tolerance.
+ These regression checks are not an accuracy benchmark or exhaustive tokenizer test.