diff options
Diffstat (limited to 'tests/check_bert_parity.py')
| -rwxr-xr-x | tests/check_bert_parity.py | 5 |
1 files changed, 2 insertions, 3 deletions
diff --git a/tests/check_bert_parity.py b/tests/check_bert_parity.py index 2500a5f..93a51f4 100755 --- a/tests/check_bert_parity.py +++ b/tests/check_bert_parity.py @@ -1,5 +1,5 @@ #!/usr/bin/env python3 -"""Compare the C BERT-family forward against SentenceTransformers. +"""Compare the BERT-family forward against SentenceTransformers. Builds a small C driver that loads the WordPiece tokenizer and the BERT model from a sentence-transformers checkpoint, wraps each text with [CLS]/[SEP], @@ -7,8 +7,7 @@ encodes it (BERT forward + mean pool + L2 normalize), and prints the vector. The reference is SentenceTransformer(model).encode(normalize_embeddings=True), which runs the same pipeline. Defaults to all-MiniLM-L6-v2. -This one needs the real model weights to run the C forward, so it stays a -manual --model-dir check rather than a hermetic stored-vector test. +Needs the real model weights to run the C forward. uv run --python 3.12 --with-requirements requirements-reference.txt \ tests/check_bert_parity.py --model-dir DIR |
