valendra/privacy-filter-mini

Valendra Privacy Filter Mini

An experimental, locally runnable token-classification model for finding potentially sensitive spans in English and Spanish text. This is a full fine-tune of the Qwen3-0.6B-Base backbone with a 41-label BIOES classification head; it is not a text-generation model.

This release contains the validation-selected step-4,000 checkpoint from an early mixed-data fine-tuning phase. It is a research snapshot, not a validated production privacy or anonymization system. The Safetensors checkpoint contains FP32 weights, approximately 2.38 GB on disk; it is not quantized.

Labels

The output vocabulary consists of O and the B-, I-, E-, and S- variants of ten categories:

PERSON, EMAIL, PHONE, ADDRESS, GOVERNMENT_ID, FINANCIAL_ID, SECRET, IP_ADDRESS, DATE_OF_BIRTH, and URL.

Character offsets and complete entities must be reconstructed from token outputs. Generic dates were not automatically treated as birth dates. The model may split one sensitive value into adjacent fragments with different types, including a secret split into PERSON and SECRET fragments.

Source code

The training, inference and comparison pipeline is maintained at valendra-tech/valendra-privacy-filter under the Apache License 2.0. The evaluation reported below used the original constrained decoder at commit a855583. The checkpoint itself loads with the standard Transformers implementation, without custom remote code.

Loading with Transformers

The checkpoint loads with the standard Qwen3ForTokenClassification implementation and does not require trust_remote_code. The tested version is Transformers 4.55.4 with PyTorch 2.8.0.

import torch
from transformers import AutoModelForTokenClassification, AutoTokenizer

model_id = "valendra/privacy-filter-mini"
tokenizer = AutoTokenizer.from_pretrained(model_id, use_fast=True)
model = AutoModelForTokenClassification.from_pretrained(
    model_id, use_safetensors=True, trust_remote_code=False
).eval()

text = "Contact me at demo@example.org."
inputs = tokenizer(
    text, return_tensors="pt", return_offsets_mapping=True,
    add_special_tokens=False, truncation=False,
)
offsets = inputs.pop("offset_mapping")[0].tolist()
if inputs["input_ids"].shape[1] > 512:
    raise ValueError("Use overlapping windows for texts longer than 512 tokens")

with torch.inference_mode():
    logits = model(**inputs).logits[0]

tags = logits.argmax(dim=-1).tolist()
print([(model.config.id2label[tag], start, end)
       for tag, (start, end) in zip(tags, offsets)])

This minimal example returns per-token argmax tags, not a redaction pipeline. Independent argmax can produce invalid BIOES sequences and does not reproduce the evaluation below. The evaluated implementation used constrained Viterbi decoding, original Unicode character offsets, and 512-token windows with a 64-token overlap. Longer texts require careful windowing and offset reconciliation; do not silently truncate sensitive input. Token and span scores are not calibrated probabilities.

Training and provenance

• Base: Qwen/Qwen3-0.6B-Base, revision da87bfb608c14b7cf20ba1ce41287e8de496c0cd; causal attention; the full backbone and classification head were trainable.
• Data: 268,298 mixed training documents from OpenPII, NVIDIA Nemotron-PII and project-generated synthetic data; Spanish training windows were repeated four times per epoch.
• The mixed-data phase initialized both backbone and classifier weights from a prior step-13,500 checkpoint, with a new AdamW optimizer and learning-rate schedule. Training used BF16 autocast with FP32 master parameters and optimizer states.
• Released weights: step 4,000 of the mixed-data phase; Safetensors SHA-256 130ea7b3a1de096aa5a1c32500d82a8157546c83607d5f71e361df0710fb9d7b.
• Mixed dataset manifest SHA-256: 9af56087c7655f676c1cb64d214ed2d9d1a5cac78e09aaaee59abd16cd3a0f18.

The inference artifact contains only model and tokenizer files; the training datasets, optimizer state and other checkpoints are not included. The repository also contains license, attribution and aggregate evaluation files.

Evaluation

The following results use complete synthetic reserved splits and the original constrained decoder, before the adjacent-secret issue noted above is addressed. All models processed the same documents sequentially on an RTX 4090. Valendra and GLiNER used BF16; OpenAI used its native BF16/FP32 parameters. Untyped F1 requires exact character-span boundaries without requiring a matching category. Typed-common F1 additionally requires the correct category among PERSON, EMAIL, PHONE, ADDRESS, URL and SECRET. Model label policies and GLiNER prompts differ.

Split	Documents	Model	Untyped precision	Untyped recall	Untyped F1	Typed-common F1
Test	22,262	Valendra, step 4,000	0.805	0.905	0.852	0.801
Test	22,262	OpenAI Privacy Filter	0.813	0.656	0.727	0.605
Test	22,262	NVIDIA GLiNER-PII	0.504	0.573	0.536	0.369
Challenge	14,461	Valendra, step 4,000	0.575	0.672	0.620	0.564
Challenge	14,461	OpenAI Privacy Filter	0.728	0.678	0.702	0.587
Challenge	14,461	NVIDIA GLiNER-PII	0.386	0.364	0.374	0.246

The checkpoint was selected using validation, and both test splits are synthetic. Each split contains only two clean negative documents, so these results do not provide a reliable negative-document false-positive rate. Template, exact-text and selected-identifier overlap checks are heuristic and cannot prove complete independence. Both Valendra and GLiNER were trained on Nemotron-derived data. These results should not be interpreted as a real-world ranking or a guarantee of anonymization.

On the test split, Valendra exactly matched 2,500 of 5,000 annotated URLs. On the challenge split, it had no exact-boundary matches for 1,982 address annotations or 2,478 IP address annotations; partial or expanded detections also count as failures under this metric. Separately sourced, independently annotated data and more negative examples are needed to assess deployment suitability.

The aggregate-only test report and challenge report include model revisions, configuration, counts, language breakdowns and runtime measurements. They contain no input texts or predicted values.

Attribution and licenses

• Model and code license: Apache License 2.0. The base model is distributed by the Qwen team under Apache 2.0.
• OpenPII 1M, Ai4Privacy / Ai Suisse SA, CC BY 4.0; revision ecfdc547f4a0955600cfe6ab98ba2a162207fcc0.
• Nemotron-PII, NVIDIA Corporation, CC BY 4.0; revision b70ffaf5ff39e079776134c5bf4381f00a9fd1ed.

Dataset licenses and attribution requirements are separate from the model license. See NOTICE.