Token Classification
GLiNER
PyTorch
ONNX
Portuguese
English
ner
pii
mdeberta-v3
span-level
markerV0
domain-adaptation
spam
legacy
experiment-log
Instructions to use arthrod/c5k-deberta_base-token_level-1-2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- GLiNER
How to use arthrod/c5k-deberta_base-token_level-1-2 with GLiNER:
from gliner import GLiNER model = GLiNER.from_pretrained("arthrod/c5k-deberta_base-token_level-1-2") text = "Cristiano Ronaldo dos Santos Aveiro was born on 5 February 1985 in Funchal, Madeira, Portugal." labels = ["person", "date", "location"] entities = model.predict_entities(text, labels) for entity in entities: print(entity["text"], "=>", entity["label"]) - Notebooks
- Google Colab
- Kaggle
Training in progress, step 500
Browse files- .gitattributes +1 -0
- config.yaml +117 -0
- pytorch_model.bin +3 -0
- summary_20260224T084509Z.txt +118 -0
- tokenizer.json +3 -0
- tokenizer_config.json +123 -0
- training_args.bin +3 -0
- validation_20260224T084509Z.log +260 -0
.gitattributes
CHANGED
|
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
| 33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
| 34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
| 35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
| 36 |
+
tokenizer.json filter=lfs diff=lfs merge=lfs -text
|
config.yaml
ADDED
|
@@ -0,0 +1,117 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
run:
|
| 2 |
+
name: b-gliner-pii-token-multipii-v1-spam-v1
|
| 3 |
+
description: Domain adaptation for PII-in-spam using token-level GLiNER starting
|
| 4 |
+
from gliner_multi_pii-v1.
|
| 5 |
+
tags:
|
| 6 |
+
- pii
|
| 7 |
+
- spam
|
| 8 |
+
- gliner
|
| 9 |
+
- token_level
|
| 10 |
+
- domain-adapt
|
| 11 |
+
- multipii-v1
|
| 12 |
+
seed: 42
|
| 13 |
+
model:
|
| 14 |
+
model_name: microsoft/deberta-v3-base
|
| 15 |
+
name: gliner-pii-token-multipii-v1-spam
|
| 16 |
+
labels_encoder: null
|
| 17 |
+
labels_decoder: null
|
| 18 |
+
decoder_mode: null
|
| 19 |
+
full_decoder_context: true
|
| 20 |
+
blank_entity_prob: 0.1
|
| 21 |
+
decoder_loss_coef: 0.5
|
| 22 |
+
relations_layer: null
|
| 23 |
+
triples_layer: null
|
| 24 |
+
embed_rel_token: true
|
| 25 |
+
rel_token_index: -1
|
| 26 |
+
rel_token: <<REL>>
|
| 27 |
+
adjacency_loss_coef: 1.0
|
| 28 |
+
relation_loss_coef: 1.0
|
| 29 |
+
span_mode: token_level
|
| 30 |
+
max_width: 12
|
| 31 |
+
represent_spans: false
|
| 32 |
+
neg_spans_ratio: 1.0
|
| 33 |
+
hidden_size: 768
|
| 34 |
+
dropout: 0.25
|
| 35 |
+
fine_tune: true
|
| 36 |
+
subtoken_pooling: first
|
| 37 |
+
fuse_layers: false
|
| 38 |
+
post_fusion_schema: ''
|
| 39 |
+
num_post_fusion_layers: 1
|
| 40 |
+
num_rnn_layers: 0
|
| 41 |
+
max_len: 384
|
| 42 |
+
max_types: 100
|
| 43 |
+
max_neg_type_ratio: 2
|
| 44 |
+
words_splitter_type: whitespace
|
| 45 |
+
embed_ent_token: true
|
| 46 |
+
class_token_index: -1
|
| 47 |
+
ent_token: <<ENT>>
|
| 48 |
+
sep_token: <<SEP>>
|
| 49 |
+
vocab_size: -1
|
| 50 |
+
token_loss_coef: 1.0
|
| 51 |
+
span_loss_coef: 1.0
|
| 52 |
+
encoder_config: null
|
| 53 |
+
_attn_implementation: flash_attention_2
|
| 54 |
+
labels_encoder_config: null
|
| 55 |
+
labels_decoder_config: null
|
| 56 |
+
data:
|
| 57 |
+
root_dir: gliner_logs/gliner-pii-token-multipii-v1-spam-v1
|
| 58 |
+
train_data: arthrod/gliner-flex-pii-ready-v5::train
|
| 59 |
+
val_data_dir: arthrod/gliner-flex-pii-ready-v5::eval
|
| 60 |
+
training:
|
| 61 |
+
prev_path: urchade/gliner_multi_pii-v1
|
| 62 |
+
num_steps: 12000
|
| 63 |
+
scheduler_type: cosine
|
| 64 |
+
warmup_ratio: 0.05
|
| 65 |
+
train_batch_size: 32
|
| 66 |
+
eval_batch_size: 64
|
| 67 |
+
gradient_accumulation_steps: 1
|
| 68 |
+
max_grad_norm: 1.0
|
| 69 |
+
optimizer: adamw_torch
|
| 70 |
+
lr_encoder: 8.0e-06
|
| 71 |
+
lr_others: 4.0e-05
|
| 72 |
+
weight_decay_encoder: 0.01
|
| 73 |
+
weight_decay_other: 0.01
|
| 74 |
+
loss_alpha: 0.75
|
| 75 |
+
loss_gamma: 2.0
|
| 76 |
+
loss_prob_margin: 0.0
|
| 77 |
+
label_smoothing: 0.0
|
| 78 |
+
loss_reduction: sum
|
| 79 |
+
negatives: 2.0
|
| 80 |
+
masking: none
|
| 81 |
+
eval_every: 500
|
| 82 |
+
save_total_limit: 20
|
| 83 |
+
logging_steps: 10
|
| 84 |
+
bf16: true
|
| 85 |
+
fp16: false
|
| 86 |
+
use_cpu: false
|
| 87 |
+
dataloader_num_workers: 18
|
| 88 |
+
dataloader_pin_memory: true
|
| 89 |
+
dataloader_persistent_workers: false
|
| 90 |
+
dataloader_prefetch_factor: 2
|
| 91 |
+
freeze_components: null
|
| 92 |
+
compile_model: false
|
| 93 |
+
gradient_checkpointing: false
|
| 94 |
+
eval_steps: null
|
| 95 |
+
eval_on_start: false
|
| 96 |
+
lora:
|
| 97 |
+
enabled: false
|
| 98 |
+
r: 16
|
| 99 |
+
lora_alpha: 32
|
| 100 |
+
lora_dropout: 0.05
|
| 101 |
+
bias: none
|
| 102 |
+
target_modules:
|
| 103 |
+
- all-linear
|
| 104 |
+
task_type: TOKEN_CLS
|
| 105 |
+
modules_to_save: null
|
| 106 |
+
init_lora_weights: true
|
| 107 |
+
use_rslora: false
|
| 108 |
+
fan_in_fan_out: false
|
| 109 |
+
environment:
|
| 110 |
+
push_to_hub: true
|
| 111 |
+
hub_model_id: arthrod/gliner-pii-token-multipii-v1-spam-v1
|
| 112 |
+
hf_token: null
|
| 113 |
+
report_to: wandb
|
| 114 |
+
wandb_project: gliner-pii
|
| 115 |
+
wandb_entity: null
|
| 116 |
+
wandb_api_key: null
|
| 117 |
+
cuda_visible_devices: '0'
|
pytorch_model.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:2de5a72ba0f93e12dd7caf7c3826d37ebc3f99138715338bff92da18e87f46c5
|
| 3 |
+
size 1155880867
|
summary_20260224T084509Z.txt
ADDED
|
@@ -0,0 +1,118 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
================================================================================
|
| 2 |
+
Configuration Validation Summary
|
| 3 |
+
================================================================================
|
| 4 |
+
run.name OK b-gliner-pii-token-multipii-v1-spam-v1
|
| 5 |
+
run.description OK Domain adaptation for PII-in-spam using token-level GLiNER starting from gliner_multi_pii-v1.
|
| 6 |
+
run.tags OK ['pii', 'spam', 'gliner', 'token_level', 'domain-adapt', 'multipii-v1']
|
| 7 |
+
run.seed OK 42
|
| 8 |
+
model.model_name OK microsoft/deberta-v3-base
|
| 9 |
+
model.name OK gliner-pii-token-multipii-v1-spam
|
| 10 |
+
model.labels_encoder OK None
|
| 11 |
+
model.labels_decoder OK None
|
| 12 |
+
model.decoder_mode OK None
|
| 13 |
+
model.full_decoder_context OK True
|
| 14 |
+
model.blank_entity_prob OK 0.1
|
| 15 |
+
model.decoder_loss_coef OK 0.5
|
| 16 |
+
model.relations_layer OK None
|
| 17 |
+
model.triples_layer OK None
|
| 18 |
+
model.embed_rel_token OK True
|
| 19 |
+
model.rel_token_index OK -1
|
| 20 |
+
model.rel_token OK <<REL>>
|
| 21 |
+
model.adjacency_loss_coef OK 1.0
|
| 22 |
+
model.relation_loss_coef OK 1.0
|
| 23 |
+
model.span_mode OK token_level
|
| 24 |
+
model.max_width OK 12
|
| 25 |
+
model.represent_spans OK False
|
| 26 |
+
model.neg_spans_ratio OK 1.0
|
| 27 |
+
model.hidden_size OK 768
|
| 28 |
+
model.dropout OK 0.25
|
| 29 |
+
model.fine_tune OK True
|
| 30 |
+
model.subtoken_pooling OK first
|
| 31 |
+
model.fuse_layers OK False
|
| 32 |
+
model.post_fusion_schema OK
|
| 33 |
+
model.num_post_fusion_layers OK 1
|
| 34 |
+
model.num_rnn_layers OK 0
|
| 35 |
+
model.max_len OK 384
|
| 36 |
+
model.max_types OK 100
|
| 37 |
+
model.max_neg_type_ratio OK 2
|
| 38 |
+
model.words_splitter_type OK whitespace
|
| 39 |
+
model.embed_ent_token OK True
|
| 40 |
+
model.class_token_index OK -1
|
| 41 |
+
model.ent_token OK <<ENT>>
|
| 42 |
+
model.sep_token OK <<SEP>>
|
| 43 |
+
model.token_loss_coef OK 1.0
|
| 44 |
+
model.span_loss_coef OK 1.0
|
| 45 |
+
model.encoder_config OK None
|
| 46 |
+
model.labels_encoder_config DEFAULT None
|
| 47 |
+
model.labels_decoder_config DEFAULT None
|
| 48 |
+
model._attn_implementation OK flash_attention_2
|
| 49 |
+
model.vocab_size OK -1
|
| 50 |
+
data.root_dir OK gliner_logs/gliner-pii-token-multipii-v1-spam-v1
|
| 51 |
+
data.train_data OK arthrod/gliner-flex-pii-ready-v5::train
|
| 52 |
+
data.val_data_dir OK arthrod/gliner-flex-pii-ready-v5::eval
|
| 53 |
+
training.prev_path OK urchade/gliner_multi_pii-v1
|
| 54 |
+
training.num_steps OK 12000
|
| 55 |
+
training.scheduler_type OK cosine
|
| 56 |
+
training.warmup_ratio OK 0.05
|
| 57 |
+
training.train_batch_size OK 32
|
| 58 |
+
training.eval_batch_size OK 64
|
| 59 |
+
training.gradient_accumulation_steps OK 1
|
| 60 |
+
training.max_grad_norm OK 1.0
|
| 61 |
+
training.optimizer OK adamw_torch
|
| 62 |
+
training.lr_encoder OK 8e-06
|
| 63 |
+
training.lr_others OK 4e-05
|
| 64 |
+
training.weight_decay_encoder OK 0.01
|
| 65 |
+
training.weight_decay_other OK 0.01
|
| 66 |
+
training.loss_alpha OK 0.75
|
| 67 |
+
training.loss_gamma OK 2.0
|
| 68 |
+
training.loss_prob_margin OK 0.0
|
| 69 |
+
training.label_smoothing OK 0.0
|
| 70 |
+
training.loss_reduction OK sum
|
| 71 |
+
training.negatives OK 2.0
|
| 72 |
+
training.masking OK none
|
| 73 |
+
training.eval_every OK 500
|
| 74 |
+
training.save_total_limit OK 20
|
| 75 |
+
training.logging_steps OK 10
|
| 76 |
+
training.bf16 OK True
|
| 77 |
+
training.fp16 OK False
|
| 78 |
+
training.use_cpu OK False
|
| 79 |
+
training.dataloader_num_workers OK 18
|
| 80 |
+
training.dataloader_pin_memory OK True
|
| 81 |
+
training.dataloader_persistent_workers OK False
|
| 82 |
+
training.dataloader_prefetch_factor OK 2
|
| 83 |
+
training.freeze_components OK None
|
| 84 |
+
training.gradient_checkpointing DEFAULT False
|
| 85 |
+
training.eval_steps DEFAULT None
|
| 86 |
+
training.eval_on_start DEFAULT False
|
| 87 |
+
training.compile_model OK False
|
| 88 |
+
lora.enabled OK False
|
| 89 |
+
lora.r OK 16
|
| 90 |
+
lora.lora_alpha OK 32
|
| 91 |
+
lora.lora_dropout OK 0.05
|
| 92 |
+
lora.bias OK none
|
| 93 |
+
lora.target_modules OK ['all-linear']
|
| 94 |
+
lora.task_type OK TOKEN_CLS
|
| 95 |
+
lora.modules_to_save OK None
|
| 96 |
+
lora.init_lora_weights DEFAULT True
|
| 97 |
+
lora.use_rslora DEFAULT False
|
| 98 |
+
lora.fan_in_fan_out DEFAULT False
|
| 99 |
+
environment.push_to_hub OK True
|
| 100 |
+
environment.hub_model_id OK arthrod/gliner-pii-token-multipii-v1-spam-v1
|
| 101 |
+
environment.hf_token OK None
|
| 102 |
+
environment.report_to OK wandb
|
| 103 |
+
environment.wandb_project OK gliner-pii
|
| 104 |
+
environment.wandb_entity OK None
|
| 105 |
+
environment.wandb_api_key OK None
|
| 106 |
+
environment.cuda_visible_devices OK 0
|
| 107 |
+
|
| 108 |
+
WARNINGS:
|
| 109 |
+
[DEFAULT] 'model.labels_encoder_config' not set -- using default: None
|
| 110 |
+
[DEFAULT] 'model.labels_decoder_config' not set -- using default: None
|
| 111 |
+
[DEFAULT] 'training.gradient_checkpointing' not set -- using default: False
|
| 112 |
+
[DEFAULT] 'training.eval_steps' not set -- using default: None
|
| 113 |
+
[DEFAULT] 'training.eval_on_start' not set -- using default: False
|
| 114 |
+
[DEFAULT] 'lora.init_lora_weights' not set -- using default: True
|
| 115 |
+
[DEFAULT] 'lora.use_rslora' not set -- using default: False
|
| 116 |
+
[DEFAULT] 'lora.fan_in_fan_out' not set -- using default: False
|
| 117 |
+
|
| 118 |
+
Total: 103 fields | 8 warnings | 0 errors
|
tokenizer.json
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:506beabd886b40a1fe008af4ea41be28e5a723410e5bc6ccc18850b1fea61bad
|
| 3 |
+
size 16034669
|
tokenizer_config.json
ADDED
|
@@ -0,0 +1,123 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"add_prefix_space": true,
|
| 3 |
+
"backend": "tokenizers",
|
| 4 |
+
"bos_token": "[CLS]",
|
| 5 |
+
"cls_token": "[CLS]",
|
| 6 |
+
"do_lower_case": false,
|
| 7 |
+
"eos_token": "[SEP]",
|
| 8 |
+
"extra_special_tokens": [
|
| 9 |
+
"[PAD]",
|
| 10 |
+
"[CLS]",
|
| 11 |
+
"[SEP]",
|
| 12 |
+
"▁<extra_id_99>",
|
| 13 |
+
"▁<extra_id_98>",
|
| 14 |
+
"▁<extra_id_97>",
|
| 15 |
+
"▁<extra_id_96>",
|
| 16 |
+
"▁<extra_id_95>",
|
| 17 |
+
"▁<extra_id_94>",
|
| 18 |
+
"▁<extra_id_93>",
|
| 19 |
+
"▁<extra_id_92>",
|
| 20 |
+
"▁<extra_id_91>",
|
| 21 |
+
"▁<extra_id_90>",
|
| 22 |
+
"▁<extra_id_89>",
|
| 23 |
+
"▁<extra_id_88>",
|
| 24 |
+
"▁<extra_id_87>",
|
| 25 |
+
"▁<extra_id_86>",
|
| 26 |
+
"▁<extra_id_85>",
|
| 27 |
+
"▁<extra_id_84>",
|
| 28 |
+
"▁<extra_id_83>",
|
| 29 |
+
"▁<extra_id_82>",
|
| 30 |
+
"▁<extra_id_81>",
|
| 31 |
+
"▁<extra_id_80>",
|
| 32 |
+
"▁<extra_id_79>",
|
| 33 |
+
"▁<extra_id_78>",
|
| 34 |
+
"▁<extra_id_77>",
|
| 35 |
+
"▁<extra_id_76>",
|
| 36 |
+
"▁<extra_id_75>",
|
| 37 |
+
"▁<extra_id_74>",
|
| 38 |
+
"▁<extra_id_73>",
|
| 39 |
+
"▁<extra_id_72>",
|
| 40 |
+
"▁<extra_id_71>",
|
| 41 |
+
"▁<extra_id_70>",
|
| 42 |
+
"▁<extra_id_69>",
|
| 43 |
+
"▁<extra_id_68>",
|
| 44 |
+
"▁<extra_id_67>",
|
| 45 |
+
"▁<extra_id_66>",
|
| 46 |
+
"▁<extra_id_65>",
|
| 47 |
+
"▁<extra_id_64>",
|
| 48 |
+
"▁<extra_id_63>",
|
| 49 |
+
"▁<extra_id_62>",
|
| 50 |
+
"▁<extra_id_61>",
|
| 51 |
+
"▁<extra_id_60>",
|
| 52 |
+
"▁<extra_id_59>",
|
| 53 |
+
"▁<extra_id_58>",
|
| 54 |
+
"▁<extra_id_57>",
|
| 55 |
+
"▁<extra_id_56>",
|
| 56 |
+
"▁<extra_id_55>",
|
| 57 |
+
"▁<extra_id_54>",
|
| 58 |
+
"▁<extra_id_53>",
|
| 59 |
+
"▁<extra_id_52>",
|
| 60 |
+
"▁<extra_id_51>",
|
| 61 |
+
"▁<extra_id_50>",
|
| 62 |
+
"▁<extra_id_49>",
|
| 63 |
+
"▁<extra_id_48>",
|
| 64 |
+
"▁<extra_id_47>",
|
| 65 |
+
"▁<extra_id_46>",
|
| 66 |
+
"▁<extra_id_45>",
|
| 67 |
+
"▁<extra_id_44>",
|
| 68 |
+
"▁<extra_id_43>",
|
| 69 |
+
"▁<extra_id_42>",
|
| 70 |
+
"▁<extra_id_41>",
|
| 71 |
+
"▁<extra_id_40>",
|
| 72 |
+
"▁<extra_id_39>",
|
| 73 |
+
"▁<extra_id_38>",
|
| 74 |
+
"▁<extra_id_37>",
|
| 75 |
+
"▁<extra_id_36>",
|
| 76 |
+
"▁<extra_id_35>",
|
| 77 |
+
"▁<extra_id_34>",
|
| 78 |
+
"▁<extra_id_33>",
|
| 79 |
+
"▁<extra_id_32>",
|
| 80 |
+
"▁<extra_id_31>",
|
| 81 |
+
"▁<extra_id_30>",
|
| 82 |
+
"▁<extra_id_29>",
|
| 83 |
+
"▁<extra_id_28>",
|
| 84 |
+
"▁<extra_id_27>",
|
| 85 |
+
"▁<extra_id_26>",
|
| 86 |
+
"▁<extra_id_25>",
|
| 87 |
+
"▁<extra_id_24>",
|
| 88 |
+
"▁<extra_id_23>",
|
| 89 |
+
"▁<extra_id_22>",
|
| 90 |
+
"▁<extra_id_21>",
|
| 91 |
+
"▁<extra_id_20>",
|
| 92 |
+
"▁<extra_id_19>",
|
| 93 |
+
"▁<extra_id_18>",
|
| 94 |
+
"▁<extra_id_17>",
|
| 95 |
+
"▁<extra_id_16>",
|
| 96 |
+
"▁<extra_id_15>",
|
| 97 |
+
"▁<extra_id_14>",
|
| 98 |
+
"▁<extra_id_13>",
|
| 99 |
+
"▁<extra_id_12>",
|
| 100 |
+
"▁<extra_id_11>",
|
| 101 |
+
"▁<extra_id_10>",
|
| 102 |
+
"▁<extra_id_9>",
|
| 103 |
+
"▁<extra_id_8>",
|
| 104 |
+
"▁<extra_id_7>",
|
| 105 |
+
"▁<extra_id_6>",
|
| 106 |
+
"▁<extra_id_5>",
|
| 107 |
+
"▁<extra_id_4>",
|
| 108 |
+
"▁<extra_id_3>",
|
| 109 |
+
"▁<extra_id_2>",
|
| 110 |
+
"▁<extra_id_1>",
|
| 111 |
+
"▁<extra_id_0>"
|
| 112 |
+
],
|
| 113 |
+
"is_local": false,
|
| 114 |
+
"mask_token": "[MASK]",
|
| 115 |
+
"model_max_length": 1000000000000000019884624838656,
|
| 116 |
+
"pad_token": "[PAD]",
|
| 117 |
+
"sep_token": "[SEP]",
|
| 118 |
+
"split_by_punct": false,
|
| 119 |
+
"tokenizer_class": "DebertaV2Tokenizer",
|
| 120 |
+
"unk_id": 3,
|
| 121 |
+
"unk_token": "[UNK]",
|
| 122 |
+
"vocab_type": "spm"
|
| 123 |
+
}
|
training_args.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:56e7a494ca68f698936f2394ec3e507d8cb5d23b525b61a2ee8d69d800f92cc7
|
| 3 |
+
size 5585
|
validation_20260224T084509Z.log
ADDED
|
@@ -0,0 +1,260 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
2026-02-24 08:45:09,750 | INFO | Log file: runs/b-token-v1/validation_20260224T084509Z.log
|
| 2 |
+
2026-02-24 08:45:09,751 | INFO | Timestamp: 20260224T084509Z
|
| 3 |
+
2026-02-24 08:45:09,751 | INFO | [OK] 'run.name' = 'b-gliner-pii-token-multipii-v1-spam-v1'
|
| 4 |
+
2026-02-24 08:45:09,752 | INFO | [OK] 'run.description' = 'Domain adaptation for PII-in-spam using token-level GLiNER starting from gliner_multi_pii-v1.'
|
| 5 |
+
2026-02-24 08:45:09,752 | INFO | [OK] 'run.tags' = ['pii', 'spam', 'gliner', 'token_level', 'domain-adapt', 'multipii-v1']
|
| 6 |
+
2026-02-24 08:45:09,753 | INFO | [OK] 'run.seed' = 42
|
| 7 |
+
2026-02-24 08:45:09,754 | INFO | [OK] 'model.model_name' = 'microsoft/deberta-v3-base'
|
| 8 |
+
2026-02-24 08:45:09,754 | INFO | [OK] 'model.name' = 'gliner-pii-token-multipii-v1-spam'
|
| 9 |
+
2026-02-24 08:45:09,755 | INFO | [OK] 'model.labels_encoder' = None
|
| 10 |
+
2026-02-24 08:45:09,755 | INFO | [OK] 'model.labels_decoder' = None
|
| 11 |
+
2026-02-24 08:45:09,755 | INFO | [OK] 'model.decoder_mode' = None
|
| 12 |
+
2026-02-24 08:45:09,756 | INFO | [OK] 'model.full_decoder_context' = True
|
| 13 |
+
2026-02-24 08:45:09,756 | INFO | [OK] 'model.blank_entity_prob' = 0.1
|
| 14 |
+
2026-02-24 08:45:09,757 | INFO | [OK] 'model.decoder_loss_coef' = 0.5
|
| 15 |
+
2026-02-24 08:45:09,757 | INFO | [OK] 'model.relations_layer' = None
|
| 16 |
+
2026-02-24 08:45:09,758 | INFO | [OK] 'model.triples_layer' = None
|
| 17 |
+
2026-02-24 08:45:09,758 | INFO | [OK] 'model.embed_rel_token' = True
|
| 18 |
+
2026-02-24 08:45:09,758 | INFO | [OK] 'model.rel_token_index' = -1
|
| 19 |
+
2026-02-24 08:45:09,759 | INFO | [OK] 'model.rel_token' = '<<REL>>'
|
| 20 |
+
2026-02-24 08:45:09,759 | INFO | [OK] 'model.adjacency_loss_coef' = 1.0
|
| 21 |
+
2026-02-24 08:45:09,760 | INFO | [OK] 'model.relation_loss_coef' = 1.0
|
| 22 |
+
2026-02-24 08:45:09,760 | INFO | [OK] 'model.span_mode' = 'token_level'
|
| 23 |
+
2026-02-24 08:45:09,761 | INFO | [OK] 'model.max_width' = 12
|
| 24 |
+
2026-02-24 08:45:09,761 | INFO | [OK] 'model.represent_spans' = False
|
| 25 |
+
2026-02-24 08:45:09,761 | INFO | [OK] 'model.neg_spans_ratio' = 1.0
|
| 26 |
+
2026-02-24 08:45:09,762 | INFO | [OK] 'model.hidden_size' = 768
|
| 27 |
+
2026-02-24 08:45:09,762 | INFO | [OK] 'model.dropout' = 0.25
|
| 28 |
+
2026-02-24 08:45:09,763 | INFO | [OK] 'model.fine_tune' = True
|
| 29 |
+
2026-02-24 08:45:09,763 | INFO | [OK] 'model.subtoken_pooling' = 'first'
|
| 30 |
+
2026-02-24 08:45:09,763 | INFO | [OK] 'model.fuse_layers' = False
|
| 31 |
+
2026-02-24 08:45:09,764 | INFO | [OK] 'model.post_fusion_schema' = ''
|
| 32 |
+
2026-02-24 08:45:09,764 | INFO | [OK] 'model.num_post_fusion_layers' = 1
|
| 33 |
+
2026-02-24 08:45:09,765 | INFO | [OK] 'model.num_rnn_layers' = 0
|
| 34 |
+
2026-02-24 08:45:09,765 | INFO | [OK] 'model.max_len' = 384
|
| 35 |
+
2026-02-24 08:45:09,765 | INFO | [OK] 'model.max_types' = 100
|
| 36 |
+
2026-02-24 08:45:09,766 | INFO | [OK] 'model.max_neg_type_ratio' = 2
|
| 37 |
+
2026-02-24 08:45:09,766 | INFO | [OK] 'model.words_splitter_type' = 'whitespace'
|
| 38 |
+
2026-02-24 08:45:09,767 | INFO | [OK] 'model.embed_ent_token' = True
|
| 39 |
+
2026-02-24 08:45:09,767 | INFO | [OK] 'model.class_token_index' = -1
|
| 40 |
+
2026-02-24 08:45:09,768 | INFO | [OK] 'model.ent_token' = '<<ENT>>'
|
| 41 |
+
2026-02-24 08:45:09,768 | INFO | [OK] 'model.sep_token' = '<<SEP>>'
|
| 42 |
+
2026-02-24 08:45:09,768 | INFO | [OK] 'model.token_loss_coef' = 1.0
|
| 43 |
+
2026-02-24 08:45:09,769 | INFO | [OK] 'model.span_loss_coef' = 1.0
|
| 44 |
+
2026-02-24 08:45:09,769 | INFO | [OK] 'model.encoder_config' = None
|
| 45 |
+
2026-02-24 08:45:09,770 | WARNING | [DEFAULT] 'model.labels_encoder_config' not set -- using default: None
|
| 46 |
+
2026-02-24 08:45:09,770 | WARNING | [DEFAULT] 'model.labels_decoder_config' not set -- using default: None
|
| 47 |
+
2026-02-24 08:45:09,771 | INFO | [OK] 'model._attn_implementation' = 'flash_attention_2'
|
| 48 |
+
2026-02-24 08:45:09,771 | INFO | [OK] 'model.vocab_size' = -1
|
| 49 |
+
2026-02-24 08:45:09,772 | INFO | [OK] 'data.root_dir' = 'gliner_logs/gliner-pii-token-multipii-v1-spam-v1'
|
| 50 |
+
2026-02-24 08:45:09,772 | INFO | [OK] 'data.train_data' = 'arthrod/gliner-flex-pii-ready-v5::train'
|
| 51 |
+
2026-02-24 08:45:09,772 | INFO | [OK] 'data.val_data_dir' = 'arthrod/gliner-flex-pii-ready-v5::eval'
|
| 52 |
+
2026-02-24 08:45:09,773 | INFO | [OK] 'training.prev_path' = 'urchade/gliner_multi_pii-v1'
|
| 53 |
+
2026-02-24 08:45:09,773 | INFO | [OK] 'training.num_steps' = 12000
|
| 54 |
+
2026-02-24 08:45:09,774 | INFO | [OK] 'training.scheduler_type' = 'cosine'
|
| 55 |
+
2026-02-24 08:45:09,774 | INFO | [OK] 'training.warmup_ratio' = 0.05
|
| 56 |
+
2026-02-24 08:45:09,775 | INFO | [OK] 'training.train_batch_size' = 32
|
| 57 |
+
2026-02-24 08:45:09,775 | INFO | [OK] 'training.eval_batch_size' = 64
|
| 58 |
+
2026-02-24 08:45:09,775 | INFO | [OK] 'training.gradient_accumulation_steps' = 1
|
| 59 |
+
2026-02-24 08:45:09,776 | INFO | [OK] 'training.max_grad_norm' = 1.0
|
| 60 |
+
2026-02-24 08:45:09,776 | INFO | [OK] 'training.optimizer' = 'adamw_torch'
|
| 61 |
+
2026-02-24 08:45:09,777 | INFO | [OK] 'training.lr_encoder' = 8e-06
|
| 62 |
+
2026-02-24 08:45:09,777 | INFO | [OK] 'training.lr_others' = 4e-05
|
| 63 |
+
2026-02-24 08:45:09,778 | INFO | [OK] 'training.weight_decay_encoder' = 0.01
|
| 64 |
+
2026-02-24 08:45:09,778 | INFO | [OK] 'training.weight_decay_other' = 0.01
|
| 65 |
+
2026-02-24 08:45:09,778 | INFO | [OK] 'training.loss_alpha' = 0.75
|
| 66 |
+
2026-02-24 08:45:09,779 | INFO | [OK] 'training.loss_gamma' = 2.0
|
| 67 |
+
2026-02-24 08:45:09,779 | INFO | [OK] 'training.loss_prob_margin' = 0.0
|
| 68 |
+
2026-02-24 08:45:09,780 | INFO | [OK] 'training.label_smoothing' = 0.0
|
| 69 |
+
2026-02-24 08:45:09,780 | INFO | [OK] 'training.loss_reduction' = 'sum'
|
| 70 |
+
2026-02-24 08:45:09,781 | INFO | [OK] 'training.negatives' = 2.0
|
| 71 |
+
2026-02-24 08:45:09,781 | INFO | [OK] 'training.masking' = 'none'
|
| 72 |
+
2026-02-24 08:45:09,781 | INFO | [OK] 'training.eval_every' = 500
|
| 73 |
+
2026-02-24 08:45:09,782 | INFO | [OK] 'training.save_total_limit' = 20
|
| 74 |
+
2026-02-24 08:45:09,782 | INFO | [OK] 'training.logging_steps' = 10
|
| 75 |
+
2026-02-24 08:45:09,783 | INFO | [OK] 'training.bf16' = True
|
| 76 |
+
2026-02-24 08:45:09,783 | INFO | [OK] 'training.fp16' = False
|
| 77 |
+
2026-02-24 08:45:09,783 | INFO | [OK] 'training.use_cpu' = False
|
| 78 |
+
2026-02-24 08:45:09,784 | INFO | [OK] 'training.dataloader_num_workers' = 18
|
| 79 |
+
2026-02-24 08:45:09,784 | INFO | [OK] 'training.dataloader_pin_memory' = True
|
| 80 |
+
2026-02-24 08:45:09,785 | INFO | [OK] 'training.dataloader_persistent_workers' = False
|
| 81 |
+
2026-02-24 08:45:09,785 | INFO | [OK] 'training.dataloader_prefetch_factor' = 2
|
| 82 |
+
2026-02-24 08:45:09,786 | INFO | [OK] 'training.freeze_components' = None
|
| 83 |
+
2026-02-24 08:45:09,786 | WARNING | [DEFAULT] 'training.gradient_checkpointing' not set -- using default: False
|
| 84 |
+
2026-02-24 08:45:09,786 | WARNING | [DEFAULT] 'training.eval_steps' not set -- using default: None
|
| 85 |
+
2026-02-24 08:45:09,787 | WARNING | [DEFAULT] 'training.eval_on_start' not set -- using default: False
|
| 86 |
+
2026-02-24 08:45:09,787 | INFO | [OK] 'training.compile_model' = False
|
| 87 |
+
2026-02-24 08:45:09,788 | INFO | [OK] 'lora.enabled' = False
|
| 88 |
+
2026-02-24 08:45:09,788 | INFO | [OK] 'lora.r' = 16
|
| 89 |
+
2026-02-24 08:45:09,789 | INFO | [OK] 'lora.lora_alpha' = 32
|
| 90 |
+
2026-02-24 08:45:09,789 | INFO | [OK] 'lora.lora_dropout' = 0.05
|
| 91 |
+
2026-02-24 08:45:09,789 | INFO | [OK] 'lora.bias' = 'none'
|
| 92 |
+
2026-02-24 08:45:09,790 | INFO | [OK] 'lora.target_modules' = ['all-linear']
|
| 93 |
+
2026-02-24 08:45:09,790 | INFO | [OK] 'lora.task_type' = 'TOKEN_CLS'
|
| 94 |
+
2026-02-24 08:45:09,791 | INFO | [OK] 'lora.modules_to_save' = None
|
| 95 |
+
2026-02-24 08:45:09,791 | WARNING | [DEFAULT] 'lora.init_lora_weights' not set -- using default: True
|
| 96 |
+
2026-02-24 08:45:09,792 | WARNING | [DEFAULT] 'lora.use_rslora' not set -- using default: False
|
| 97 |
+
2026-02-24 08:45:09,792 | WARNING | [DEFAULT] 'lora.fan_in_fan_out' not set -- using default: False
|
| 98 |
+
2026-02-24 08:45:09,793 | INFO | [OK] 'environment.push_to_hub' = True
|
| 99 |
+
2026-02-24 08:45:09,793 | INFO | [OK] 'environment.hub_model_id' = 'arthrod/gliner-pii-token-multipii-v1-spam-v1'
|
| 100 |
+
2026-02-24 08:45:09,793 | INFO | [OK] 'environment.hf_token' = None
|
| 101 |
+
2026-02-24 08:45:09,794 | INFO | [OK] 'environment.report_to' = 'wandb'
|
| 102 |
+
2026-02-24 08:45:09,794 | INFO | [OK] 'environment.wandb_project' = 'gliner-pii'
|
| 103 |
+
2026-02-24 08:45:09,795 | INFO | [OK] 'environment.wandb_entity' = None
|
| 104 |
+
2026-02-24 08:45:09,795 | INFO | [OK] 'environment.wandb_api_key' = None
|
| 105 |
+
2026-02-24 08:45:09,796 | INFO | [OK] 'environment.cuda_visible_devices' = '0'
|
| 106 |
+
2026-02-24 08:45:09,796 | INFO | [DATA] Training source is HF dataset repo 'arthrod/gliner-flex-pii-ready-v5' (split='train')
|
| 107 |
+
2026-02-24 08:45:09,797 | INFO | [DATA] Validation source is HF dataset repo 'arthrod/gliner-flex-pii-ready-v5' (split='eval')
|
| 108 |
+
2026-02-24 08:45:09,960 | INFO | [HF] Authenticated as 'arthrod'
|
| 109 |
+
2026-02-24 08:45:10,019 | INFO | [WANDB] Authenticated as 'arthrod'
|
| 110 |
+
2026-02-24 08:45:10,039 | INFO | Summary written to log file
|
| 111 |
+
2026-02-24 08:45:10,040 | INFO | Summary saved to runs/b-token-v1/summary_20260224T084509Z.txt
|
| 112 |
+
2026-02-24 08:45:10,043 | INFO | Resolved config saved to runs/b-token-v1/config.yaml
|
| 113 |
+
2026-02-24 08:45:10,043 | INFO | Preparing training run ...
|
| 114 |
+
2026-02-24 08:45:13,463 | INFO | Loading pretrained model from: urchade/gliner_multi_pii-v1
|
| 115 |
+
2026-02-24 08:45:26,843 | INFO | Model class: UniEncoderSpanGLiNER
|
| 116 |
+
2026-02-24 08:45:26,844 | INFO | [CONFIG] Overrode pretrained config with YAML values:
|
| 117 |
+
2026-02-24 08:45:26,844 | INFO | max_types: 25 → 100
|
| 118 |
+
2026-02-24 08:45:26,845 | INFO | max_neg_type_ratio: 3 → 2
|
| 119 |
+
2026-02-24 08:45:26,846 | INFO | dropout: 0.4 → 0.25
|
| 120 |
+
2026-02-24 08:45:26,846 | INFO | Loading training data from HF dataset 'arthrod/gliner-flex-pii-ready-v5' (split='train')
|
| 121 |
+
2026-02-24 08:47:25,793 | INFO | Training samples: 1111948
|
| 122 |
+
2026-02-24 08:47:25,794 | INFO | Loading validation data from HF dataset 'arthrod/gliner-flex-pii-ready-v5' (split='eval')
|
| 123 |
+
2026-02-24 08:48:30,030 | INFO | Validation samples: 526449
|
| 124 |
+
2026-02-24 08:48:30,032 | INFO | ============================================================
|
| 125 |
+
2026-02-24 08:48:30,032 | INFO | [DIAG] Model architecture summary
|
| 126 |
+
2026-02-24 08:48:30,033 | INFO | [DIAG] Class: UniEncoderSpanGLiNER
|
| 127 |
+
2026-02-24 08:48:30,033 | INFO | [DIAG] Total params: 288,949,504
|
| 128 |
+
2026-02-24 08:48:30,034 | INFO | [DIAG] Trainable params: 288,949,504 (100.00%)
|
| 129 |
+
2026-02-24 08:48:30,034 | INFO | [DIAG] Dataset sizes
|
| 130 |
+
2026-02-24 08:48:30,035 | INFO | [DIAG] Train: 1111948
|
| 131 |
+
2026-02-24 08:48:30,035 | INFO | [DIAG] Eval: 526449
|
| 132 |
+
2026-02-24 08:48:30,036 | INFO | [DIAG] Model config snapshot
|
| 133 |
+
2026-02-24 08:48:30,036 | INFO | [DIAG] return_dict: True
|
| 134 |
+
2026-02-24 08:48:30,037 | INFO | [DIAG] output_hidden_states: False
|
| 135 |
+
2026-02-24 08:48:30,038 | INFO | [DIAG] dtype: None
|
| 136 |
+
2026-02-24 08:48:30,038 | INFO | [DIAG] chunk_size_feed_forward: 0
|
| 137 |
+
2026-02-24 08:48:30,039 | INFO | [DIAG] is_encoder_decoder: False
|
| 138 |
+
2026-02-24 08:48:30,039 | INFO | [DIAG] architectures: None
|
| 139 |
+
2026-02-24 08:48:30,040 | INFO | [DIAG] id2label: {0: 'LABEL_0', 1: 'LABEL_1'}
|
| 140 |
+
2026-02-24 08:48:30,040 | INFO | [DIAG] label2id: {'LABEL_0': 0, 'LABEL_1': 1}
|
| 141 |
+
2026-02-24 08:48:30,041 | INFO | [DIAG] problem_type: None
|
| 142 |
+
2026-02-24 08:48:30,041 | INFO | [DIAG] transformers_version: None
|
| 143 |
+
2026-02-24 08:48:30,041 | INFO | [DIAG] size_sup: -1
|
| 144 |
+
2026-02-24 08:48:30,042 | INFO | [DIAG] shuffle_types: True
|
| 145 |
+
2026-02-24 08:48:30,042 | INFO | [DIAG] random_drop: False
|
| 146 |
+
2026-02-24 08:48:30,043 | INFO | [DIAG] lr_encoder: '5e-6'
|
| 147 |
+
2026-02-24 08:48:30,043 | INFO | [DIAG] lr_others: '1e-5'
|
| 148 |
+
2026-02-24 08:48:30,044 | INFO | [DIAG] num_steps: 30000
|
| 149 |
+
2026-02-24 08:48:30,044 | INFO | [DIAG] warmup_ratio: 5000
|
| 150 |
+
2026-02-24 08:48:30,045 | INFO | [DIAG] train_batch_size: 8
|
| 151 |
+
2026-02-24 08:48:30,045 | INFO | [DIAG] eval_every: 5000
|
| 152 |
+
2026-02-24 08:48:30,046 | INFO | [DIAG] encoder_config: DebertaV2Config {
|
| 153 |
+
"attention_probs_dropout_prob": 0.1,
|
| 154 |
+
"bos_token_id": null,
|
| 155 |
+
"eos_token_id": null,
|
| 156 |
+
"hidden_act": "gelu",
|
| 157 |
+
"hidden_dropout_prob": 0.1,
|
| 158 |
+
"hidden_size": 768,
|
| 159 |
+
"initializer_range": 0.02,
|
| 160 |
+
"intermediate_size": 3072,
|
| 161 |
+
"layer_norm_eps": 1e-07,
|
| 162 |
+
"legacy": true,
|
| 163 |
+
"max_position_embeddings": 512,
|
| 164 |
+
"max_relative_positions": -1,
|
| 165 |
+
"model_type": "deberta-v2",
|
| 166 |
+
"norm_rel_ebd": "layer_norm",
|
| 167 |
+
"num_attention_heads": 12,
|
| 168 |
+
"num_hidden_layers": 12,
|
| 169 |
+
"pad_token_id": 0,
|
| 170 |
+
"pooler_dropout": 0,
|
| 171 |
+
"pooler_hidden_act": "gelu",
|
| 172 |
+
"pooler_hidden_size": 768,
|
| 173 |
+
"pos_att_type": [
|
| 174 |
+
"p2c",
|
| 175 |
+
"c2p"
|
| 176 |
+
],
|
| 177 |
+
"position_biased_input": false,
|
| 178 |
+
"position_buckets": 256,
|
| 179 |
+
"relative_attention": true,
|
| 180 |
+
"share_att_key": true,
|
| 181 |
+
"tie_word_embeddings": true,
|
| 182 |
+
"transformers_version": "5.1.0",
|
| 183 |
+
"type_vocab_size": 0,
|
| 184 |
+
"vocab_size": 250105
|
| 185 |
+
}
|
| 186 |
+
|
| 187 |
+
2026-02-24 08:48:30,049 | INFO | [DIAG] model_name: 'microsoft/mdeberta-v3-base'
|
| 188 |
+
2026-02-24 08:48:30,049 | INFO | [DIAG] name: 'correct'
|
| 189 |
+
2026-02-24 08:48:30,050 | INFO | [DIAG] max_width: 12
|
| 190 |
+
2026-02-24 08:48:30,050 | INFO | [DIAG] hidden_size: 512
|
| 191 |
+
2026-02-24 08:48:30,051 | INFO | [DIAG] dropout: 0.25
|
| 192 |
+
2026-02-24 08:48:30,051 | INFO | [DIAG] fine_tune: True
|
| 193 |
+
2026-02-24 08:48:30,052 | INFO | [DIAG] subtoken_pooling: 'first'
|
| 194 |
+
2026-02-24 08:48:30,052 | INFO | [DIAG] span_mode: 'markerV0'
|
| 195 |
+
2026-02-24 08:48:30,053 | INFO | [DIAG] post_fusion_schema: ''
|
| 196 |
+
2026-02-24 08:48:30,053 | INFO | [DIAG] num_post_fusion_layers: 1
|
| 197 |
+
2026-02-24 08:48:30,054 | INFO | [DIAG] vocab_size: 250105
|
| 198 |
+
2026-02-24 08:48:30,054 | INFO | [DIAG] max_neg_type_ratio: 2
|
| 199 |
+
2026-02-24 08:48:30,055 | INFO | [DIAG] max_types: 100
|
| 200 |
+
2026-02-24 08:48:30,055 | INFO | [DIAG] max_len: 384
|
| 201 |
+
2026-02-24 08:48:30,055 | INFO | [DIAG] words_splitter_type: 'whitespace'
|
| 202 |
+
2026-02-24 08:48:30,056 | INFO | [DIAG] num_rnn_layers: 1
|
| 203 |
+
2026-02-24 08:48:30,056 | INFO | [DIAG] fuse_layers: False
|
| 204 |
+
2026-02-24 08:48:30,057 | INFO | [DIAG] class_token_index: 250103
|
| 205 |
+
2026-02-24 08:48:30,057 | INFO | [DIAG] embed_ent_token: True
|
| 206 |
+
2026-02-24 08:48:30,058 | INFO | [DIAG] ent_token: '<<ENT>>'
|
| 207 |
+
2026-02-24 08:48:30,058 | INFO | [DIAG] sep_token: '<<SEP>>'
|
| 208 |
+
2026-02-24 08:48:30,059 | INFO | [DIAG] token_loss_coef: 1.0
|
| 209 |
+
2026-02-24 08:48:30,059 | INFO | [DIAG] span_loss_coef: 1.0
|
| 210 |
+
2026-02-24 08:48:30,060 | INFO | [DIAG] represent_spans: False
|
| 211 |
+
2026-02-24 08:48:30,060 | INFO | [DIAG] neg_spans_ratio: 1.0
|
| 212 |
+
2026-02-24 08:48:30,061 | INFO | [DIAG] model_type: 'gliner_uni_encoder_span'
|
| 213 |
+
2026-02-24 08:48:30,061 | INFO | [DIAG] trust_remote_code: False
|
| 214 |
+
2026-02-24 08:48:30,061 | INFO | [DIAG] Training kwargs (non-dataset)
|
| 215 |
+
2026-02-24 08:48:30,062 | INFO | [DIAG] output_dir: 'runs/b-token-v1'
|
| 216 |
+
2026-02-24 08:48:30,062 | INFO | [DIAG] freeze_components: None
|
| 217 |
+
2026-02-24 08:48:30,063 | INFO | [DIAG] compile_model: False
|
| 218 |
+
2026-02-24 08:48:30,063 | INFO | [DIAG] resume_from_checkpoint: None
|
| 219 |
+
2026-02-24 08:48:30,064 | INFO | [DIAG] max_steps: 12000
|
| 220 |
+
2026-02-24 08:48:30,064 | INFO | [DIAG] lr_scheduler_type: 'cosine'
|
| 221 |
+
2026-02-24 08:48:30,064 | INFO | [DIAG] warmup_ratio: 0.05
|
| 222 |
+
2026-02-24 08:48:30,065 | INFO | [DIAG] per_device_train_batch_size: 32
|
| 223 |
+
2026-02-24 08:48:30,065 | INFO | [DIAG] per_device_eval_batch_size: 64
|
| 224 |
+
2026-02-24 08:48:30,066 | INFO | [DIAG] learning_rate: 8e-06
|
| 225 |
+
2026-02-24 08:48:30,066 | INFO | [DIAG] others_lr: 4e-05
|
| 226 |
+
2026-02-24 08:48:30,067 | INFO | [DIAG] weight_decay: 0.01
|
| 227 |
+
2026-02-24 08:48:30,067 | INFO | [DIAG] others_weight_decay: 0.01
|
| 228 |
+
2026-02-24 08:48:30,068 | INFO | [DIAG] max_grad_norm: 1.0
|
| 229 |
+
2026-02-24 08:48:30,068 | INFO | [DIAG] optim: 'adamw_torch'
|
| 230 |
+
2026-02-24 08:48:30,068 | INFO | [DIAG] focal_loss_alpha: 0.75
|
| 231 |
+
2026-02-24 08:48:30,069 | INFO | [DIAG] focal_loss_gamma: 2.0
|
| 232 |
+
2026-02-24 08:48:30,069 | INFO | [DIAG] focal_loss_prob_margin: 0.0
|
| 233 |
+
2026-02-24 08:48:30,070 | INFO | [DIAG] label_smoothing: 0.0
|
| 234 |
+
2026-02-24 08:48:30,070 | INFO | [DIAG] loss_reduction: 'sum'
|
| 235 |
+
2026-02-24 08:48:30,071 | INFO | [DIAG] negatives: 2.0
|
| 236 |
+
2026-02-24 08:48:30,071 | INFO | [DIAG] masking: 'none'
|
| 237 |
+
2026-02-24 08:48:30,071 | INFO | [DIAG] save_steps: 500
|
| 238 |
+
2026-02-24 08:48:30,072 | INFO | [DIAG] logging_steps: 10
|
| 239 |
+
2026-02-24 08:48:30,072 | INFO | [DIAG] save_total_limit: 20
|
| 240 |
+
2026-02-24 08:48:30,073 | INFO | [DIAG] eval_strategy: 'steps'
|
| 241 |
+
2026-02-24 08:48:30,073 | INFO | [DIAG] eval_steps: 500
|
| 242 |
+
2026-02-24 08:48:30,074 | INFO | [DIAG] eval_on_start: False
|
| 243 |
+
2026-02-24 08:48:30,074 | INFO | [DIAG] bf16: True
|
| 244 |
+
2026-02-24 08:48:30,074 | INFO | [DIAG] fp16: False
|
| 245 |
+
2026-02-24 08:48:30,075 | INFO | [DIAG] gradient_checkpointing: False
|
| 246 |
+
2026-02-24 08:48:30,075 | INFO | [DIAG] use_cpu: False
|
| 247 |
+
2026-02-24 08:48:30,076 | INFO | [DIAG] dataloader_num_workers: 18
|
| 248 |
+
2026-02-24 08:48:30,076 | INFO | [DIAG] dataloader_pin_memory: True
|
| 249 |
+
2026-02-24 08:48:30,077 | INFO | [DIAG] dataloader_persistent_workers: False
|
| 250 |
+
2026-02-24 08:48:30,077 | INFO | [DIAG] dataloader_prefetch_factor: 2
|
| 251 |
+
2026-02-24 08:48:30,077 | INFO | [DIAG] report_to: 'wandb'
|
| 252 |
+
2026-02-24 08:48:30,078 | INFO | [DIAG] run_name: 'b-gliner-pii-token-multipii-v1-spam-v1'
|
| 253 |
+
2026-02-24 08:48:30,078 | INFO | [DIAG] gradient_accumulation_steps: 1
|
| 254 |
+
2026-02-24 08:48:30,079 | INFO | [DIAG] seed: 42
|
| 255 |
+
2026-02-24 08:48:30,079 | INFO | [DIAG] remove_unused_columns: False
|
| 256 |
+
2026-02-24 08:48:30,080 | INFO | [DIAG] push_to_hub: True
|
| 257 |
+
2026-02-24 08:48:30,080 | INFO | [DIAG] hub_model_id: 'arthrod/gliner-pii-token-multipii-v1-spam-v1'
|
| 258 |
+
2026-02-24 08:48:30,081 | INFO | [DIAG] lora_config: None
|
| 259 |
+
2026-02-24 08:48:30,081 | INFO | ============================================================
|
| 260 |
+
2026-02-24 08:48:30,082 | INFO | Starting training ...
|