sbert
This is a sentence-transformers model finetuned from sentence-transformers/bert-base-nli-mean-tokens on the fin_modern_bert-pairs-sec-synthetic-v1 dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for retrieval.
Model Details
Model Description
- Model Type: Sentence Transformer
- Base model: sentence-transformers/bert-base-nli-mean-tokens
- Maximum Sequence Length: 512 tokens
- Output Dimensionality: 768 dimensions
- Similarity Function: Cosine Similarity
- Supported Modality: Text
- Training Dataset:
- fin_modern_bert-pairs-sec-synthetic-v1
- Language: en
- License: apache-2.0
Model Sources
Full Model Architecture
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
(1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)
Usage
Direct Usage (Sentence Transformers)
First install the Sentence Transformers library:
pip install -U sentence-transformers
Then you can load this model and run inference.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("sentence_transformers_model_id")
queries = [
"What was the percentage growth in third sector product sales for Aflac Japan in 2014, and how did this compare to the company's sales target?",
]
documents = [
'Sales of third sector products increased 6.1% in 2014 , compared with the same period in 2013 , achieving the high end of our 2% to 7% sales target.',
'Gross margin decreased 1 percentage point in 2024 when compared to 2023. Gross margin was impacted by lower sales volume and higher infrastructure costs which were partially offset by lower salary expense related to workforce reduction activities and expenses attributed to business exit activities.',
'As of December 31, 2015 and 2014 , the Company had aggregate unfunded commitments of $ 436.4 million and $ 187.9 million, respectively, of which $ 2.2 million and $ 5.6 million, respectively, is unfunded obligations recorded in relation to the acquisition of AREA Property Partners, L.P. (“AREA”).',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
Evaluation
Metrics
Information Retrieval
| Metric |
Value |
| cosine_accuracy@1 |
0.6376 |
| cosine_accuracy@3 |
0.7946 |
| cosine_accuracy@5 |
0.8436 |
| cosine_accuracy@10 |
0.8886 |
| cosine_precision@1 |
0.6376 |
| cosine_precision@3 |
0.2649 |
| cosine_precision@5 |
0.1687 |
| cosine_precision@10 |
0.0889 |
| cosine_recall@1 |
0.6376 |
| cosine_recall@3 |
0.7946 |
| cosine_recall@5 |
0.8436 |
| cosine_recall@10 |
0.8886 |
| cosine_ndcg@10 |
0.765 |
| cosine_mrr@10 |
0.7252 |
| cosine_map@100 |
0.7293 |
Training Details
Training Dataset
fin_modern_bert-pairs-sec-synthetic-v1
- Dataset: fin_modern_bert-pairs-sec-synthetic-v1
- Size: 44,709 training samples
- Columns:
query and positive
- Approximate statistics based on the first 100 samples:
|
query |
positive |
| type |
string |
string |
| modality |
text |
text |
| details |
- min: 16 tokens
- mean: 28.12 tokens
- max: 46 tokens
|
- min: 20 tokens
- mean: 63.47 tokens
- max: 171 tokens
|
- Samples:
| query |
positive |
Why are quarter-to-quarter comparisons of the company's revenues and operating results considered difficult or non-meaningful? |
Our revenues and operating results have varied significantly from quarter to quarter because our business experiences seasonal fluctuations, which reflect seasonal trends for the travel services offered by our websites. As a result, quarter-to-quarter comparisons of our revenues and operating results may not be meaningful. |
How does the company handle the amortization of DAC and DSI for interest-sensitive life, fixed annuities, and other investment contracts? |
For interest-sensitive life, fixed annuities and other investment contracts, DAC and DSI are amortized in proportion to the incidence of the total present value of gross profits, which includes both actual historical gross profits (“AGP”) and estimated future gross profits (“EGP”) expected to be earned over the estimated lives of the contracts. |
What are the specific risks associated with the company's strategy of making acquisitions and investments in businesses, software products, and technologies? |
The risks associated with such acquisitions include, among others, the difficulty of assimilating products, operations and personnel, inheriting liabilities such as intellectual property infringement claims, the failure to realize anticipated revenue and cost projections, the requirement to test and assimilate the internal control processes of the acquired business in accordance with the requirements of Section 404 of the Sarbanes-Oxley Act of 2002 and the diversion of management's time and attention. |
- Loss:
CachedMultipleNegativesRankingLoss with these parameters:{
"scale": 20.0,
"similarity_fct": "cos_sim",
"mini_batch_size": 4,
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
Evaluation Dataset
fin_modern_bert-pairs-sec-synthetic-v1
- Dataset: fin_modern_bert-pairs-sec-synthetic-v1
- Size: 9,581 evaluation samples
- Columns:
query and positive
- Approximate statistics based on the first 100 samples:
|
query |
positive |
| type |
string |
string |
| modality |
text |
text |
| details |
- min: 19 tokens
- mean: 27.96 tokens
- max: 41 tokens
|
- min: 16 tokens
- mean: 63.97 tokens
- max: 142 tokens
|
- Samples:
| query |
positive |
How could the proposed 2024 lawsuit settlement agreement between Visa and Mastercard impact the company's profitability? |
For example, in 2024 Visa and Mastercard proposed a lawsuit settlement agreement that would have, among other things, required reductions and caps on interchange fees and streamlined requirements for merchants who wish to impose a surcharge on credit transactions. While the settlement agreement was not approved by the court, Visa and Mastercard may ultimately agree or be subject to changes in policies or practices that result in greater surcharging generally or downward pressure on our merchant discount rates from decreases in competitor pricing in connection with reductions and caps on interchange fees. |
What are the specific dates and terms for the Indentures relating to the New 1.00% Notes and New 2.25% Notes? |
4.15(w) Indenture relating to New 1.00% Notes, dated as of November 6, 2006, between priceline.com Incorporated and American Stock Transfer and Trust Company, as Trustee. 4.16(w) Indenture relating to New 2.25% Notes, dated as of November 6, 2006, between priceline.com Incorporated and American Stock Transfer and Trust Company, as Trustee. |
What are the primary competitors in the external defibrillator market according to the company's report? |
Our competition in the external defibrillator market comes primarily from Physio Control Corporation (a subsidiary of Medtronic Inc.) and Zoll Medical Corporation. |
- Loss:
CachedMultipleNegativesRankingLoss with these parameters:{
"scale": 20.0,
"similarity_fct": "cos_sim",
"mini_batch_size": 4,
"gather_across_devices": false,
"directions": [
"query_to_doc"
],
"partition_mode": "joint",
"hardness_mode": null,
"hardness_strength": 0.0
}
Training Hyperparameters
Non-Default Hyperparameters
per_device_train_batch_size: 16
learning_rate: 2e-05
warmup_steps: 0.1
bf16: True
logging_nan_inf_filter: False
per_device_eval_batch_size: 16
seed: 777
data_seed: 777
prompts: {'query': '', 'positive': ''}
batch_sampler: no_duplicates
All Hyperparameters
Click to expand
per_device_train_batch_size: 16
num_train_epochs: 3
max_steps: -1
learning_rate: 2e-05
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0.1
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: True
fp16: False
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: False
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 16
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: False
hub_private_repo: None
hub_model_id: None
hub_strategy: every_save
hub_always_push: False
hub_revision: None
load_best_model_at_end: False
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 777
data_seed: 777
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
dataloader_multiprocessing_context: None
dataloader_in_order: True
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
local_rank: -1
prompts: {'query': '', 'positive': ''}
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}
warmup_ratio: None
Training Logs
Click to expand
| Epoch |
Step |
Training Loss |
Validation Loss |
finance-eval_cosine_ndcg@10 |
| -1 |
-1 |
- |
- |
0.2731 |
| 0.0179 |
50 |
1.0162 |
- |
- |
| 0.0358 |
100 |
0.9645 |
- |
- |
| 0.0537 |
150 |
0.8988 |
- |
- |
| 0.0716 |
200 |
0.9452 |
- |
- |
| 0.0894 |
250 |
0.9270 |
- |
- |
| 0.1073 |
300 |
0.7601 |
- |
- |
| 0.1252 |
350 |
0.7007 |
- |
- |
| 0.1431 |
400 |
0.6563 |
- |
- |
| 0.1610 |
450 |
0.5641 |
- |
- |
| 0.1789 |
500 |
0.4359 |
- |
- |
| 0.1968 |
550 |
0.4266 |
- |
- |
| 0.2147 |
600 |
0.3230 |
- |
- |
| 0.2326 |
650 |
0.2815 |
- |
- |
| 0.2504 |
700 |
0.2642 |
- |
- |
| 0.2683 |
750 |
0.2248 |
- |
- |
| 0.2862 |
800 |
0.2430 |
- |
- |
| 0.3041 |
850 |
0.2061 |
- |
- |
| 0.3220 |
900 |
0.1926 |
- |
- |
| 0.3399 |
950 |
0.1578 |
- |
- |
| 0.3578 |
1000 |
0.1848 |
- |
- |
| 0.3757 |
1050 |
0.1431 |
- |
- |
| 0.3936 |
1100 |
0.1361 |
- |
- |
| 0.4114 |
1150 |
0.1297 |
- |
- |
| 0.4293 |
1200 |
0.1537 |
- |
- |
| 0.4472 |
1250 |
0.1215 |
- |
- |
| 0.4651 |
1300 |
0.1196 |
- |
- |
| 0.4830 |
1350 |
0.1292 |
- |
- |
| 0.5009 |
1400 |
0.1322 |
- |
- |
| 0.5188 |
1450 |
0.1053 |
- |
- |
| 0.5367 |
1500 |
0.1070 |
- |
- |
| 0.5546 |
1550 |
0.0743 |
- |
- |
| 0.5725 |
1600 |
0.1000 |
- |
- |
| 0.5903 |
1650 |
0.1081 |
- |
- |
| 0.6082 |
1700 |
0.0882 |
- |
- |
| 0.6261 |
1750 |
0.1009 |
- |
- |
| 0.6440 |
1800 |
0.1053 |
- |
- |
| 0.6619 |
1850 |
0.0933 |
- |
- |
| 0.6798 |
1900 |
0.1083 |
- |
- |
| 0.6977 |
1950 |
0.0671 |
- |
- |
| 0.7156 |
2000 |
0.0882 |
- |
- |
| 0.7335 |
2050 |
0.0819 |
- |
- |
| 0.7513 |
2100 |
0.0829 |
- |
- |
| 0.7692 |
2150 |
0.0740 |
- |
- |
| 0.7871 |
2200 |
0.0873 |
- |
- |
| 0.8050 |
2250 |
0.0582 |
- |
- |
| 0.8229 |
2300 |
0.0753 |
- |
- |
| 0.8408 |
2350 |
0.0903 |
- |
- |
| 0.8587 |
2400 |
0.0739 |
- |
- |
| 0.8766 |
2450 |
0.0942 |
- |
- |
| 0.8945 |
2500 |
0.0955 |
- |
- |
| 0.9123 |
2550 |
0.0774 |
- |
- |
| 0.9302 |
2600 |
0.0710 |
- |
- |
| 0.9481 |
2650 |
0.0749 |
- |
- |
| 0.9660 |
2700 |
0.0740 |
- |
- |
| 0.9839 |
2750 |
0.0749 |
- |
- |
| 1.0 |
2795 |
- |
0.0554 |
0.7511 |
| 1.0018 |
2800 |
0.0693 |
- |
- |
| 1.0197 |
2850 |
0.0553 |
- |
- |
| 1.0376 |
2900 |
0.0575 |
- |
- |
| 1.0555 |
2950 |
0.0493 |
- |
- |
| 1.0733 |
3000 |
0.0537 |
- |
- |
| 1.0912 |
3050 |
0.0710 |
- |
- |
| 1.1091 |
3100 |
0.0586 |
- |
- |
| 1.1270 |
3150 |
0.0732 |
- |
- |
| 1.1449 |
3200 |
0.0729 |
- |
- |
| 1.1628 |
3250 |
0.0579 |
- |
- |
| 1.1807 |
3300 |
0.0702 |
- |
- |
| 1.1986 |
3350 |
0.0788 |
- |
- |
| 1.2165 |
3400 |
0.0765 |
- |
- |
| 1.2343 |
3450 |
0.0632 |
- |
- |
| 1.2522 |
3500 |
0.0506 |
- |
- |
| 1.2701 |
3550 |
0.0563 |
- |
- |
| 1.2880 |
3600 |
0.0696 |
- |
- |
| 1.3059 |
3650 |
0.0602 |
- |
- |
| 1.3238 |
3700 |
0.0790 |
- |
- |
| 1.3417 |
3750 |
0.0578 |
- |
- |
| 1.3596 |
3800 |
0.0800 |
- |
- |
| 1.3775 |
3850 |
0.0627 |
- |
- |
| 1.3953 |
3900 |
0.0711 |
- |
- |
| 1.4132 |
3950 |
0.0711 |
- |
- |
| 1.4311 |
4000 |
0.0557 |
- |
- |
| 1.4490 |
4050 |
0.0698 |
- |
- |
| 1.4669 |
4100 |
0.0663 |
- |
- |
| 1.4848 |
4150 |
0.0768 |
- |
- |
| 1.5027 |
4200 |
0.0678 |
- |
- |
| 1.5206 |
4250 |
0.0641 |
- |
- |
| 1.5385 |
4300 |
0.0543 |
- |
- |
| 1.5564 |
4350 |
0.0764 |
- |
- |
| 1.5742 |
4400 |
0.0594 |
- |
- |
| 1.5921 |
4450 |
0.0447 |
- |
- |
| 1.6100 |
4500 |
0.0630 |
- |
- |
| 1.6279 |
4550 |
0.0771 |
- |
- |
| 1.6458 |
4600 |
0.0478 |
- |
- |
| 1.6637 |
4650 |
0.0561 |
- |
- |
| 1.6816 |
4700 |
0.0628 |
- |
- |
| 1.6995 |
4750 |
0.0579 |
- |
- |
| 1.7174 |
4800 |
0.0631 |
- |
- |
| 1.7352 |
4850 |
0.0472 |
- |
- |
| 1.7531 |
4900 |
0.0505 |
- |
- |
| 1.7710 |
4950 |
0.0764 |
- |
- |
| 1.7889 |
5000 |
0.0549 |
- |
- |
| 1.8068 |
5050 |
0.0764 |
- |
- |
| 1.8247 |
5100 |
0.0782 |
- |
- |
| 1.8426 |
5150 |
0.0633 |
- |
- |
| 1.8605 |
5200 |
0.0638 |
- |
- |
| 1.8784 |
5250 |
0.0724 |
- |
- |
| 1.8962 |
5300 |
0.0688 |
- |
- |
| 1.9141 |
5350 |
0.0567 |
- |
- |
| 1.9320 |
5400 |
0.0688 |
- |
- |
| 1.9499 |
5450 |
0.0570 |
- |
- |
| 1.9678 |
5500 |
0.0666 |
- |
- |
| 1.9857 |
5550 |
0.0513 |
- |
- |
| 2.0 |
5590 |
- |
0.0487 |
0.7643 |
| 2.0036 |
5600 |
0.0745 |
- |
- |
| 2.0215 |
5650 |
0.0609 |
- |
- |
| 2.0394 |
5700 |
0.0765 |
- |
- |
| 2.0572 |
5750 |
0.0423 |
- |
- |
| 2.0751 |
5800 |
0.0551 |
- |
- |
| 2.0930 |
5850 |
0.0637 |
- |
- |
| 2.1109 |
5900 |
0.0695 |
- |
- |
| 2.1288 |
5950 |
0.0638 |
- |
- |
| 2.1467 |
6000 |
0.0629 |
- |
- |
| 2.1646 |
6050 |
0.0749 |
- |
- |
| 2.1825 |
6100 |
0.0485 |
- |
- |
| 2.2004 |
6150 |
0.0474 |
- |
- |
| 2.2182 |
6200 |
0.0466 |
- |
- |
| 2.2361 |
6250 |
0.0608 |
- |
- |
| 2.2540 |
6300 |
0.0563 |
- |
- |
| 2.2719 |
6350 |
0.0610 |
- |
- |
| 2.2898 |
6400 |
0.0573 |
- |
- |
| 2.3077 |
6450 |
0.0614 |
- |
- |
| 2.3256 |
6500 |
0.0700 |
- |
- |
| 2.3435 |
6550 |
0.0560 |
- |
- |
| 2.3614 |
6600 |
0.0888 |
- |
- |
| 2.3792 |
6650 |
0.0788 |
- |
- |
| 2.3971 |
6700 |
0.0622 |
- |
- |
| 2.4150 |
6750 |
0.0481 |
- |
- |
| 2.4329 |
6800 |
0.0696 |
- |
- |
| 2.4508 |
6850 |
0.0723 |
- |
- |
| 2.4687 |
6900 |
0.0492 |
- |
- |
| 2.4866 |
6950 |
0.0615 |
- |
- |
| 2.5045 |
7000 |
0.0651 |
- |
- |
| 2.5224 |
7050 |
0.0601 |
- |
- |
| 2.5403 |
7100 |
0.0484 |
- |
- |
| 2.5581 |
7150 |
0.0570 |
- |
- |
| 2.5760 |
7200 |
0.0516 |
- |
- |
| 2.5939 |
7250 |
0.0640 |
- |
- |
| 2.6118 |
7300 |
0.0590 |
- |
- |
| 2.6297 |
7350 |
0.0603 |
- |
- |
| 2.6476 |
7400 |
0.0678 |
- |
- |
| 2.6655 |
7450 |
0.0591 |
- |
- |
| 2.6834 |
7500 |
0.0515 |
- |
- |
| 2.7013 |
7550 |
0.0485 |
- |
- |
| 2.7191 |
7600 |
0.0602 |
- |
- |
| 2.7370 |
7650 |
0.0462 |
- |
- |
| 2.7549 |
7700 |
0.0608 |
- |
- |
| 2.7728 |
7750 |
0.0626 |
- |
- |
| 2.7907 |
7800 |
0.0652 |
- |
- |
| 2.8086 |
7850 |
0.0532 |
- |
- |
| 2.8265 |
7900 |
0.0660 |
- |
- |
| 2.8444 |
7950 |
0.0521 |
- |
- |
| 2.8623 |
8000 |
0.0566 |
- |
- |
| 2.8801 |
8050 |
0.0515 |
- |
- |
| 2.8980 |
8100 |
0.0639 |
- |
- |
| 2.9159 |
8150 |
0.0796 |
- |
- |
| 2.9338 |
8200 |
0.0559 |
- |
- |
| 2.9517 |
8250 |
0.0519 |
- |
- |
| 2.9696 |
8300 |
0.0710 |
- |
- |
| 2.9875 |
8350 |
0.0510 |
- |
- |
| 3.0 |
8385 |
- |
0.0483 |
0.7650 |
| -1 |
-1 |
- |
- |
0.7650 |
Training Time
Framework Versions
- Python: 3.13.16
- Sentence Transformers: 5.6.0
- Transformers: 5.22.1
- PyTorch: 2.14.1+cu130
- Accelerate: 1.14.0
- Datasets: 4.8.5
- Tokenizers: 0.23.3
Citation
BibTeX
Sentence Transformers
@inproceedings{reimers-2019-sentence-bert,
title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
author = "Reimers, Nils and Gurevych, Iryna",
booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
month = "11",
year = "2019",
publisher = "Association for Computational Linguistics",
url = "https://arxiv.org/abs/1908.10084",
}
CachedMultipleNegativesRankingLoss
@misc{gao2021scaling,
title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup},
author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
year={2021},
eprint={2101.06983},
archivePrefix={arXiv},
primaryClass={cs.LG}
}