tomaarsen/static-embedding-miriad

Static Embedding for MIRIAD Medical Retrieval

This is a sentence-transformers model trained on the retrieval and qa datasets. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

• Model Type: Sentence Transformer
• Maximum Sequence Length: inf tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Datasets:
  • retrieval
  • qa
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): StaticEmbedding({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("tomaarsen/static-embedding-miriad")
# Run inference
sentences = [
    'What instructions were given to patients regarding oral hygiene and care after the implant surgery?',
    "Patients were screened for study inclusion on the day of implant placement on a rolling admission basis. Medical history was reviewed prior to each implant procedure. Sociodemographic data were collected and included age, gender, race, and ethnicity. All implants were placed at posterior sites (premolar or molar) with the exception of one implant placed at an anterior (cuspid) location. A tooth was selected for evaluation as a control within the same quadrant as the implant site. The control tooth was the first available tooth mesial to the implant site and was separated from the implant by at least one natural tooth in order to avoid possible influence from the adjacent developing implant sulcus (34) .\n\n Probing depths, bleeding on probing, and supragingival plaque were recorded for the control tooth. Subgingival scaling was completed presurgically on the control tooth chosen to be sampled. During the implant surgery, flaps were reflected, and gingival thickness was measured from six different sites surrounding the implant with a Marquis probe (Hu-Friedy, Inc., Chicago, IL) (2). A transmucosal healing abutment was used for all patients. Implants were manufactured by Nobel Biocare (Yorba Linda, CA); root form, tapered implants were placed. Patients were instructed to avoid brushing at or near the surgical site for 1 week. All patients were prescribed and instructed to gently rinse with 0.12% chlorhexidine gluconate for 30 s twice per day for 1 week (2). Postoperative acetaminophen or hydrocodone with acetaminophen was advised for postoperative analgesia.\n\n Four-week GCF, PICF, and subgingival plaque sample collection. Medical history and exclusion criteria were reviewed for patients returning for the 4-week postoperative and sample collection visit. All patients were assessed for any signs of infection or failure of implant osseointegration at this appointment.\n\n Crevicular fluid samples, followed by subgingival plaque samples, were collected at the dental implant and control tooth sites on patients who continued to be eligible for study participation. Prior to crevicular fluid and subgingival plaque sample collection, supragingival plaque was removed around the implant abutment and control tooth by scaling. The implant and control tooth were isolated using cotton rolls and gently air dried. Crevicular fluid samples were then collected by inserting filter paper strips (Periopaper; Proflow, Amityville, NY), until slight resistance was felt, into the mesiobuccal and mesiolingual peri-implant and tooth sulci for 30 s. The two peri-implant crevicular fluid (PICF) samples for the implant were placed into a single microcentrifuge tube on ice beside the dentist's chair and frozen at Ϫ80°C within 10 min of collection. Likewise, the same procedure was followed for the two gingival crevicular fluid (GCF) samples taken from the control tooth.\n\n Peri-implant plaque samples were collected by inserting four separate sterile endodontic paper points (Henry Schein medium absorbent points; Melville, NY) into the mesial, distal, buccal, and lingual sites (i.e., circumferentially around the implant) for a period of 10 s each. Subgingival plaque samples collected from around the dental implant were pooled in a single microcentrifuge tube and frozen at Ϫ80°C within 10 min of collection. The same procedure was followed for the plaque samples taken from the control tooth. GCF, PICF, and subgingival plaque samples remained frozen at Ϫ80°C until analysis.\n\n Twelve-week GCF, PICF, and subgingival plaque sample collection. Twelve weeks after the initial implant placement, patients returned, and the same protocol was performed as described above for the 4-week sample collection appointment. All plaque, GCF, and PICF samples at both 4-week and 12-week visits were collected by the same investigator (P. G. Johnson).\n\n DNA extraction of plaque samples. DNA extraction was performed by the method of Martinez et al. (35) with slight modifications. Briefly, 200 l of ice-cold phosphate-buffered saline (PBS) was added to tubes containing the paper points and the tubes were vortexed for 15 min. Next, 750 l of lysis buffer and 20 mg/ml of lysozyme were added to the tubes, and the samples were then transferred into bead beating tubes containing 300 mg of disruption beads (0.1 mm; Research Products International Corp., Mount Prospect, IL). The samples were incubated for 15 min in a 37°C water bath.",
    '111 an earlier study, Boissonnault and Blaschak" found that only 11% of their subjects had rectus abdominis muscle separation below the umbilicus in their third trimester. The authors noted, however, that a higher incidence would have been found if the diastasis criteria had approached the normal linea alba width at this point rather than the 2 c m criterion suggested by -----+ I\n\n I(apandji3+ described the rectus abdomi~lis muscle as a powerful trunk flexor muscle operating by a lever system through the Iumbosacral and thoracolumbar joints. The rectus abdominis muscle\'s normal line of action is aligned vertically from the costal margin to the p u b i~.~.~~ Our results, however, show that by 30 weeks of gestation, the angle of insertion in the coronal and sagittal planes for rectus abdominis muscle had altered such that the muscle\'s line of action was deviated laterally and anteriorly, as shown in Figure 5 . A sinrplified force diagram for the rectus abdoniinis muscle at 30 weeks of gestation in the sagittal plane at the thoracolumbar and lumbosacral joints is show11 in Figure 6 . This diagram shows that the moment arm length, and therefore torque production of the rectus abdominis muscle about these joints in this plane, may be reduced at 30 weeks of gestation. The ability of the rectus abdominis muscle to flex the trunk is therefore possibly diminished. Whether the minimum change in angles of insertion at which reduced torque prodtrction of the rectus abdominis muscle will have a demonstrable affect on the muscle\'s functional capacity is unknown.\n\n -Umbilicus\n\n As we assumed that the adaptations of this representative abdominal muscle did not occur in isolation, we believe the ability to generate torque rnay be compromised across the entire muscle group. Hence, the second aim of this study was to examine the functional ability of the abdominal muscles, and the temporal relationship between functional ability and muscle adaptations, during pregnancy and into the postpartum period. Because there were few subjects in this study, which precluded the use of some statistical methods, we view this part of the study as a pilot investigation of abdominal muscle function during pregnancy and the postpartum period on a longitudinal basis.\n\n By 26 weeks of gestation, the ability to perform a curl-up type of abdominal exercise had diminished for all s u b jects. This decline continued to 38 weeks of gestation, at which time no subjects were successful in completi~lg a curl-up. Fast et a15 found that 22 out of 164 subjects at a mean of 38 weeks of gestation could successfully complete a hook-lying sit-up to a \\isually determined 40-degree angle from the horizontal. The higher percentage of successful subjects in the study by Fast e t ali lrldy be due to the use of a less extensive curl-up movement than that used in our study.\n\n We found the performance of curl-ups was improved postbirth. Five of the six subjects were successfill or moderately successful in performance of a curl-up at 8 weeks postbirth. This postbirth improvement supports the previous results of S p e n~e ,~ who found that 80% of her subjects could complete a similar curl-up exercise at 6 weeks postbirth.\n\n A comparison of the results for the two AMTs used in this study show that the AMT and curl-up results concur for test sessions during the pregnancy; however, these results are conflicting for the postbirth period. These apparent conflicts in AMT results also have been noted by Kendall and McCreary" for nongravid subjects. The AMT results showed that the ability to stabilize the pelvis against resistance while positioned supine generally remained compromised postbirth. In contrast, the abdonlinal muscles\' supine trunk flexion ability increased postbirth. The use of a curl-up as a f~lnctional test for the abdominal muscles during pregnancy, however, is questionable. During pregnancy, the uterus presents a physical obstruction to the close approximation of the thorax and pelvis, which is necessary to complete a curl-up. The ability to pelform the curl-up may be more related to the presence of this physical obstruction rather than to the functional ability of the muscles. In addition, supine trunk flexion may be assisted by the hip flexor~.:~5~"Thus, the performance of a curl-up in the postbirth period may not solely be a test of the abdominal muscles\' capabilities. In contrast, the MkfT is performed primarily by the abdominal muscles. 35 Potential assistance by the hip extensors acting to rotate the pelvis posteriorly is comprised due to the lack of a fixed distal attachment on the rotating limb.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.5613, 0.0316],
#         [0.5613, 1.0000, 0.0152],
#         [0.0316, 0.0152, 1.0000]])

Evaluation

Metrics

Information Retrieval

• Datasets: miriad-retrieval, miriad-qa, NanoClimateFEVER, NanoDBPedia, NanoFEVER, NanoFiQA2018, NanoHotpotQA, NanoMSMARCO, NanoNFCorpus, NanoNQ, NanoQuoraRetrieval, NanoSCIDOCS, NanoArguAna, NanoSciFact and NanoTouche2020
• Evaluated with InformationRetrievalEvaluator

Metric	miriad-retrieval	miriad-qa	NanoClimateFEVER	NanoDBPedia	NanoFEVER	NanoFiQA2018	NanoHotpotQA	NanoMSMARCO	NanoNFCorpus	NanoNQ	NanoQuoraRetrieval	NanoSCIDOCS	NanoArguAna	NanoSciFact	NanoTouche2020
cosine_accuracy@1	0.7447	0.9507	0.26	0.54	0.32	0.18	0.52	0.18	0.36	0.08	0.68	0.32	0.16	0.56	0.3878
cosine_accuracy@3	0.8697	0.9907	0.48	0.76	0.58	0.36	0.68	0.4	0.56	0.26	0.86	0.52	0.46	0.68	0.8571
cosine_accuracy@5	0.909	0.9956	0.56	0.84	0.64	0.44	0.82	0.46	0.56	0.36	0.94	0.62	0.58	0.8	0.9388
cosine_accuracy@10	0.9398	0.9989	0.64	0.9	0.82	0.52	0.88	0.6	0.66	0.48	0.98	0.74	0.72	0.84	0.9796
cosine_precision@1	0.7447	0.9507	0.26	0.54	0.32	0.18	0.52	0.18	0.36	0.08	0.68	0.32	0.16	0.56	0.3878
cosine_precision@3	0.2899	0.3304	0.1733	0.4667	0.2	0.1533	0.2933	0.1333	0.3467	0.0867	0.3333	0.24	0.1533	0.24	0.4762
cosine_precision@5	0.1818	0.1992	0.124	0.46	0.136	0.124	0.212	0.092	0.32	0.072	0.224	0.196	0.116	0.176	0.4857
cosine_precision@10	0.094	0.0999	0.08	0.408	0.086	0.08	0.126	0.06	0.276	0.05	0.12	0.142	0.072	0.094	0.4102
cosine_recall@1	0.7446	0.9504	0.14	0.0616	0.31	0.0689	0.26	0.18	0.043	0.08	0.6073	0.0677	0.16	0.525	0.0224
cosine_recall@3	0.8696	0.9906	0.225	0.1165	0.5533	0.2146	0.44	0.4	0.0743	0.25	0.8287	0.1477	0.46	0.65	0.0908
cosine_recall@5	0.9089	0.9955	0.2617	0.1694	0.6233	0.2851	0.53	0.46	0.09	0.33	0.8993	0.2027	0.58	0.78	0.1504
cosine_recall@10	0.9397	0.9989	0.3283	0.2777	0.7933	0.342	0.63	0.6	0.1251	0.46	0.9393	0.2917	0.72	0.83	0.2427
cosine_ndcg@10	0.845	0.978	0.2793	0.4817	0.5472	0.2467	0.5375	0.3731	0.3265	0.2576	0.8098	0.2755	0.4327	0.6867	0.4303
cosine_mrr@10	0.8143	0.971	0.3761	0.6587	0.479	0.2827	0.6352	0.3022	0.4612	0.1983	0.7813	0.4521	0.3413	0.6511	0.6247
cosine_map@100	0.8165	0.971	0.2286	0.3648	0.4728	0.1869	0.4598	0.3191	0.1513	0.2055	0.7637	0.2042	0.3525	0.6402	0.3128

Nano BEIR

• Dataset: NanoBEIR_mean
• Evaluated with NanoBEIREvaluator with these parameters:

  {
      "dataset_names": [
          "climatefever",
          "dbpedia",
          "fever",
          "fiqa2018",
          "hotpotqa",
          "msmarco",
          "nfcorpus",
          "nq",
          "quoraretrieval",
          "scidocs",
          "arguana",
          "scifact",
          "touche2020"
      ],
      "dataset_id": "sentence-transformers/NanoBEIR-en"
  }
  

Metric	Value
cosine_accuracy@1	0.3498
cosine_accuracy@3	0.5736
cosine_accuracy@5	0.6584
cosine_accuracy@10	0.7507
cosine_precision@1	0.3498
cosine_precision@3	0.2536
cosine_precision@5	0.2106
cosine_precision@10	0.1542
cosine_recall@1	0.1943
cosine_recall@3	0.3424
cosine_recall@5	0.4125
cosine_recall@10	0.5062
cosine_ndcg@10	0.4373
cosine_mrr@10	0.4803
cosine_map@100	0.3586

Training Details

Training Datasets

retrieval

• Dataset: retrieval at 596b9ab
• Size: 4,467,542 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 48 characters mean: 106.6 characters max: 317 characters	min: 2837 characters mean: 4576.82 characters max: 7387 characters

• Samples:

  anchor	positive
  What factors may contribute to increased pulmonary conduit durability in patients who undergo the Ross operation compared to those with right ventricular outflow tract obstruction?	I n 1966, Ross and Somerville 1 reported the first use of an aortic homograft to establish right ventricle-to-pulmonary artery continuity in a patient with tetralogy of Fallot and pulmonary atresia. Since that time, pulmonary position homografts have been used in a variety of right-sided congenital heart lesions. Actuarial 5-year homograft survivals for cryopreserved homografts are reported to range between 55% and 94%, with the shortest durability noted in patients less than 2 years of age. 4 Pulmonary position homografts also are used to replace pulmonary autografts explanted to repair left-sided outflow disease (the Ross operation). Several factors may be likely to favor increased pulmonary conduit durability in Ross patients compared with those with right ventricular outflow tract obstruction, including later age at operation (allowing for larger homografts), more normal pulmonary artery architecture, absence of severe right ventricular hypertrophy, and more natural positioning of ...
  How does MCAM expression in hMSC affect the growth and maintenance of hematopoietic progenitors?	After culture in a 3-dimensional hydrogel-based matrix, which constitutes hypoxic conditions, MCAM expression is lost. Concordantly, Tormin et al. demonstrated that MCAM is down-regulated under hypoxic conditions. 10 Furthermore, it was shown by others and our group that oxygen tension causes selective modification of hematopoietic cell and mesenchymal stromal cell interactions in co-culture systems as well as influence HSPC metabolism. [44] [45] [46] Thus, the observed differences between Sharma et al. and our data in HSPC supporting capacity of hMSC are likely due to the different culture conditions used. Further studies are required to clarify the influence of hypoxia in our model system. Altogether these findings provide further evidence for the importance of MCAM in supporting HSPC. Furthermore, previous reports have shown that MCAM is down-regulated in MSC after several passages as well as during aging and differentiation. 19, 47 Interestingly, MCAM overexpression in hMSC enhance...
  What is the relationship between Fanconi anemia and breast and ovarian cancer susceptibility genes?	( 31 ) , of which 5% -10 % may be caused by genetic factors ( 32 ) , up to half a million of these patients may be at risk of secondary hereditary neoplasms. The historic observation of twofold to fi vefold increased risks of cancers of the ovary, thyroid, and connective tissue after breast cancer ( 33 ) presaged the later syndromic association of these tumors with inherited mutations of BRCA1, BRCA2, PTEN, and p53 ( 16 ) . By far the largest cumulative risk of a secondary cancer in BRCA mutation carriers is associated with cancer in the contralateral breast, which may reach a risk of 29.5% at 10 years ( 34 ) . The Breast Cancer Linkage Consortium ( 35 , 36 ) also documented threefold to fi vefold increased risks of subsequent cancers of prostate, pancreas, gallbladder, stomach, skin (melanoma), and uterus in BRCA2 mutation carriers and twofold increased risks of prostate and pancreas cancer in BRCA1 mutation carriers; these results are based largely on self-reported family history inf...

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

qa

• Dataset: qa at 596b9ab
• Size: 4,467,542 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 48 characters mean: 106.6 characters max: 317 characters	min: 116 characters mean: 507.5 characters max: 1009 characters

• Samples:

  anchor	positive
  What factors may contribute to increased pulmonary conduit durability in patients who undergo the Ross operation compared to those with right ventricular outflow tract obstruction?	Several factors may contribute to increased pulmonary conduit durability in patients who undergo the Ross operation compared to those with right ventricular outflow tract obstruction. These factors include later age at operation (allowing for larger homografts), more normal pulmonary artery architecture, absence of severe right ventricular hypertrophy, and more natural positioning of the homograft. However, further systematic studies are needed to confirm these associations.
  How does MCAM expression in hMSC affect the growth and maintenance of hematopoietic progenitors?	MCAM expression in hMSC has been shown to support the growth of hematopoietic progenitors. It enhances the adhesion and migration of HSPC, potentially through direct cell-cell interactions. However, the putative interaction partner of MCAM on HSPC remains unknown. Additionally, MCAM expression in hMSC does not seem to regulate the expression or secretion of SDF-1, a key factor in HSPC homing and maintenance.
  What is the relationship between Fanconi anemia and breast and ovarian cancer susceptibility genes?	Fanconi anemia is a rare, autosomal recessive syndrome characterized by chromosomal instability, cancer susceptibility, and hypersensitivity to DNA cross-linking agents. It has been found that all known Fanconi anemia proteins cooperate with breast and/or ovarian cancer susceptibility gene products (BRCA1 and BRCA2) in a pathway required for cellular resistance to DNA cross-linking agents. This pathway, known as the "Fanconi anemia-BRCA pathway," is a DNA damage-activated signaling pathway that controls DNA repair. Methylation of one of the Fanconi anemia genes, FANCF, can lead to the inactivation of this pathway in breast and ovarian cancer, suggesting its importance in human carcinogenesis.

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Evaluation Datasets

retrieval

• Dataset: retrieval at 596b9ab

• Size: 10,000 evaluation samples

• Columns: anchor and positive

• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 40 characters mean: 106.65 characters max: 227 characters	min: 2841 characters mean: 4564.28 characters max: 6653 characters

• Samples:

  anchor	positive
  What are some hereditary cancer syndromes that can result in various forms of cancer?	Hereditary Cancer Syndromes, including Hereditary Breast and Ovarian Cancer (HBOC) and Lynch Syndrome (LS), can result in various forms of cancer due to germline mutations in cancer predisposition genes. While the major contributory genes for these syndromes have been identified and well-studied (BRCA1/ BRCA2 for HBOC and MSH2/MSH6/MLH1/PMS2/ EPCAM for LS), there remains a large percentage of associated cancer cases that are negative for germline mutations in these genes, including 80% of women with a personal or family history of breast cancer who are negative for BRCA1/2 mutations [1] . Similarly, between 30 and 50% of families fulfill stringent criteria for LS and test negative for germline mismatch repair gene mutations [2] . Adding complexity to these disorders is the significant overlap in the spectrum of cancers observed between various hereditary cancer syndromes, including many cancer susceptibility syndromes. Some that contribute to elevated breast cancer risk include Li-Frau...
  How do MAK-4 and MAK-5 exert their antioxidant properties?	Hybrid F1 mice were injected with urethane (300 mg/kg) at 8 days of age. A group was then put on a MAK-supplemented diet, another group was fed a standard pellet diet. At 36 weeks of age the mice were sacrificed and the livers examined for the presence of tumors mouse (Panel A) and for the number of nodules per mouse (Panel B) (* p < 0.05, ** P < 0.001). Statistical analysis was performed by Two Way ANOVA Test followed by Post Hoc Bonferroni analysis.

  We than measured the influence of the MAK-4+5 combination on the expression of the three liver-specific connexins (cx26, cx32, and cx43). The level of cx26 expression was similar in all the groups of mice treated with the MAK-supplemented diet and in the control (Figure 4, Panel A) . A significant, time-dependent increase in cx32 was observed in the liver of all the groups of MAK treated mice compared to the normal diet-fed controls. Cx32 expression increased 2-fold after 1 week of treatment, and 3-to 4-fold at 3 months (Figure 4, Pane... | | What are the primary indications for a decompressive craniectomy, and what role does neurocritical care play in determining the suitability of a patient for this procedure? | Decompressive craniectomy is a valid neurosurgical strategy now a day as an alternative to control an elevated intracranial pressure (ICP) and controlling the risk of uncal and/or subfalcine herniation, in refractory cases to the postural, ventilator, and pharmacological measures to control it. The neurocritical care and the ICP monitorization are key determinants to identify and postulate the inclusion criteria to consider a patient as candidate to this procedure, as it is always considered a rescue surgical technique. Head trauma and ischemic or hemorrhagic cerebrovascular disease with progressive deterioration due to mass effect are some of the cases that may require a decompressive craniectomy with its different variants. However, this procedure per se can have complications described in the postcraniectomy syndrome and may occur in short, medium, or even long term.

  1,2 The paradoxical herniation is a condition in which there is a deviation of the midline with mass effect, even t... |

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

qa

• Dataset: qa at 596b9ab
• Size: 10,000 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 40 characters mean: 106.65 characters max: 227 characters	min: 189 characters mean: 492.49 characters max: 915 characters

• Samples:

  anchor	positive
  What are some hereditary cancer syndromes that can result in various forms of cancer?	Hereditary cancer syndromes, such as Hereditary Breast and Ovarian Cancer (HBOC) and Lynch Syndrome (LS), can result in various forms of cancer due to germline mutations in cancer predisposition genes. These syndromes are associated with an increased risk of developing specific types of cancer.
  How do MAK-4 and MAK-5 exert their antioxidant properties?	MAK-4 and MAK-5 have been shown to have antioxidant properties both in vitro and in vivo. These preparations contain multiple antioxidants such as alpha-tocopherol, beta-carotene, ascorbate, bioflavonoid, catechin, polyphenols, riboflavin, and tannic acid. These antioxidants are known to scavenge free radicals and reactive oxygen species (ROS) such as superoxide, hydroxyl, and peroxyl radicals, as well as hydrogen peroxide. In the present study, the antioxidant properties of MAK-4 and MAK-5 were confirmed in mice, with higher oxygen radical absorbance capacity (ORAC) values observed in mice fed the MAK-supplemented diet. Additionally, the activity of liver enzymes GPX, GST, and QR, which are involved in detoxification processes, were upregulated in the MAK-fed mice. This suggests that MAK-4 and MAK-5 may protect against carcinogenesis by reducing oxidative stress and enhancing detoxification processes.
  What are the primary indications for a decompressive craniectomy, and what role does neurocritical care play in determining the suitability of a patient for this procedure?	The primary indications for a decompressive craniectomy include refractory intracranial pressure (ICP) and progressive neurological deterioration due to mass effect from conditions like head trauma, or ischemic or hemorrhagic cerebrovascular disease. Neurocritical care and ICP monitoring are essential in identifying suitable candidates for the procedure, as it is considered a rescue surgical technique. These measures help to assess the patient's condition and determine the need for decompressive craniectomy in cases of elevated ICP.

• Loss: MatryoshkaLoss with these parameters:

  {
      "loss": "MultipleNegativesRankingLoss",
      "matryoshka_dims": [
          1024,
          512,
          256,
          128,
          64,
          32
      ],
      "matryoshka_weights": [
          1,
          1,
          1,
          1,
          1,
          1
      ],
      "n_dims_per_step": -1
  }
  

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 2048
• num_train_epochs: 1
• learning_rate: 0.1
• lr_scheduler_type: cosine
• warmup_steps: 0.1
• bf16: True
• disable_tqdm: True
• project: static-embedding-miriad
• per_device_eval_batch_size: 2048
• push_to_hub: True
• hub_model_id: tomaarsen/static-embedding-miriad
• dataloader_num_workers: 4
• batch_sampler: no_duplicates

All Hyperparameters

Click to expand

• per_device_train_batch_size: 2048
• num_train_epochs: 1
• max_steps: -1
• learning_rate: 0.1
• lr_scheduler_type: cosine
• lr_scheduler_kwargs: None
• warmup_steps: 0.1
• optim: adamw_torch_fused
• optim_args: None
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• optim_target_modules: None
• gradient_accumulation_steps: 1
• average_tokens_across_devices: True
• max_grad_norm: 1.0
• label_smoothing_factor: 0.0
• bf16: True
• fp16: False
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• use_liger_kernel: False
• liger_kernel_config: None
• use_cache: False
• neftune_noise_alpha: None
• torch_empty_cache_steps: None
• auto_find_batch_size: False
• log_on_each_node: True
• logging_nan_inf_filter: True
• include_num_input_tokens_seen: no
• log_level: passive
• log_level_replica: warning
• disable_tqdm: True
• project: static-embedding-miriad
• trackio_space_id: trackio
• per_device_eval_batch_size: 2048
• prediction_loss_only: True
• eval_on_start: False
• eval_do_concat_batches: True
• eval_use_gather_object: False
• eval_accumulation_steps: None
• include_for_metrics: []
• batch_eval_metrics: False
• save_only_model: False
• save_on_each_node: False
• enable_jit_checkpoint: False
• push_to_hub: True
• hub_private_repo: None
• hub_model_id: tomaarsen/static-embedding-miriad
• hub_strategy: every_save
• hub_always_push: False
• hub_revision: None
• load_best_model_at_end: False
• ignore_data_skip: False
• restore_callback_states_from_checkpoint: False
• full_determinism: False
• seed: 42
• data_seed: None
• use_cpu: False
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• parallelism_config: None
• dataloader_drop_last: False
• dataloader_num_workers: 4
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• dataloader_prefetch_factor: None
• remove_unused_columns: True
• label_names: None
• train_sampling_strategy: random
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• ddp_backend: None
• ddp_timeout: 1800
• fsdp: []
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• deepspeed: None
• debug: []
• skip_memory_metrics: True
• do_predict: False
• resume_from_checkpoint: None
• warmup_ratio: None
• local_rank: -1
• prompts: None
• batch_sampler: no_duplicates
• multi_dataset_batch_sampler: proportional
• router_mapping: {}
• learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	retrieval loss	qa loss	miriad-retrieval_cosine_ndcg@10	miriad-qa_cosine_ndcg@10	NanoClimateFEVER_cosine_ndcg@10	NanoDBPedia_cosine_ndcg@10	NanoFEVER_cosine_ndcg@10	NanoFiQA2018_cosine_ndcg@10	NanoHotpotQA_cosine_ndcg@10	NanoMSMARCO_cosine_ndcg@10	NanoNFCorpus_cosine_ndcg@10	NanoNQ_cosine_ndcg@10	NanoQuoraRetrieval_cosine_ndcg@10	NanoSCIDOCS_cosine_ndcg@10	NanoArguAna_cosine_ndcg@10	NanoSciFact_cosine_ndcg@10	NanoTouche2020_cosine_ndcg@10	NanoBEIR_mean_cosine_ndcg@10
-1	-1	-	-	-	0.2548	0.6615	0.0580	0.4458	0.2880	0.1166	0.3887	0.3165	0.2063	0.1524	0.6621	0.1922	0.1742	0.2818	0.1905	0.2672
0.0002	1	88.9344	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0101	44	54.9491	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0202	88	28.4558	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0302	132	17.8002	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0403	176	15.3586	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0504	220	12.8462	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0605	264	12.1462	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0706	308	11.5985	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0807	352	9.5926	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.0907	396	10.6146	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1001	437	-	12.9915	6.8537	0.8086	0.9687	0.2717	0.5033	0.5820	0.2109	0.5732	0.3447	0.3106	0.2636	0.8015	0.2629	0.4169	0.6740	0.4580	0.4364
0.1008	440	9.7729	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1109	484	11.1276	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1210	528	9.7008	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1311	572	8.7597	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1412	616	9.8124	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1512	660	10.3953	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1613	704	9.2735	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1714	748	10.0010	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1815	792	9.4746	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.1916	836	8.9106	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2003	874	-	11.2877	6.3690	0.8238	0.9714	0.2792	0.4938	0.5800	0.2240	0.5468	0.3623	0.2972	0.2451	0.8102	0.2654	0.4023	0.6827	0.4258	0.4319
0.2016	880	9.6910	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2117	924	9.2948	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2218	968	9.7736	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2319	1012	8.7777	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2420	1056	8.4413	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2521	1100	9.5213	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2621	1144	9.1040	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2722	1188	10.3161	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2823	1232	8.7639	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.2924	1276	9.3876	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3004	1311	-	10.9829	6.1913	0.8297	0.9739	0.2700	0.4865	0.5691	0.2161	0.5384	0.3666	0.3089	0.2401	0.7887	0.2730	0.4411	0.6882	0.4371	0.4326
0.3025	1320	8.8995	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3126	1364	9.0478	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3226	1408	8.5948	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3327	1452	8.8627	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3428	1496	9.2912	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3529	1540	9.7053	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3630	1584	9.2486	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3731	1628	8.2048	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3831	1672	9.6167	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.3932	1716	8.5938	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4005	1748	-	10.6257	6.0514	0.8330	0.9756	0.2725	0.4925	0.5606	0.2392	0.5463	0.3573	0.3227	0.2231	0.8210	0.2841	0.4319	0.6993	0.4377	0.4376
0.4033	1760	8.8808	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4134	1804	9.3018	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4235	1848	8.7846	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4335	1892	8.1371	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4436	1936	8.2877	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4537	1980	9.3938	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4638	2024	8.6855	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4739	2068	8.2660	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4840	2112	9.0818	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.4940	2156	7.9626	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5007	2185	-	10.4397	5.9879	0.8383	0.9765	0.2770	0.4979	0.5653	0.2378	0.5270	0.3603	0.3260	0.2585	0.8116	0.2784	0.4022	0.6924	0.4391	0.4364
0.5041	2200	9.3393	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5142	2244	7.9270	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5243	2288	9.1986	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5344	2332	9.0107	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5445	2376	8.4710	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5545	2420	8.1332	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5646	2464	8.8726	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5747	2508	8.6585	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5848	2552	8.7556	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.5949	2596	8.3934	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6008	2622	-	10.3170	5.8990	0.8401	0.9776	0.2795	0.4799	0.5475	0.2382	0.5466	0.3699	0.3209	0.2455	0.8191	0.2774	0.4308	0.6933	0.4334	0.4371
0.6049	2640	8.0685	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6150	2684	8.6580	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6251	2728	8.8270	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6352	2772	9.0009	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6453	2816	8.8914	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6554	2860	9.0320	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6654	2904	8.4398	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6755	2948	8.6450	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6856	2992	8.1186	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.6957	3036	8.4620	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7010	3059	-	10.1960	5.8606	0.8429	0.9772	0.2805	0.4859	0.5531	0.2361	0.5324	0.3710	0.3295	0.2678	0.8182	0.2720	0.4327	0.6734	0.4356	0.4376
0.7058	3080	8.5030	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7159	3124	8.4862	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7259	3168	8.2113	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7360	3212	8.9945	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7461	3256	8.6438	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7562	3300	8.4359	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7663	3344	8.1664	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7764	3388	8.0845	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7864	3432	8.0391	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.7965	3476	8.4185	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8011	3496	-	10.0932	5.8304	0.8449	0.9779	0.2784	0.4850	0.5439	0.2390	0.5346	0.3799	0.3289	0.2564	0.8161	0.2769	0.4246	0.6922	0.4330	0.4376
0.8066	3520	8.5722	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8167	3564	8.7177	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8268	3608	8.0759	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8368	3652	9.1715	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8469	3696	8.0562	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8570	3740	8.7197	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8671	3784	9.3027	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8772	3828	8.3458	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8873	3872	8.4659	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.8973	3916	9.3818	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9012	3933	-	10.0359	5.8242	0.8452	0.9782	0.2798	0.4814	0.5472	0.2472	0.5371	0.3731	0.3275	0.2576	0.8098	0.2738	0.4261	0.6867	0.4301	0.4367
0.9074	3960	8.7067	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9175	4004	9.8819	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9276	4048	8.3104	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9377	4092	8.1972	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9478	4136	8.5456	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9578	4180	8.8344	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9679	4224	9.1987	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9780	4268	8.4421	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9881	4312	8.5238	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
0.9982	4356	8.5983	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-	-
1.0	4364	-	10.0224	5.8227	0.8450	0.9780	0.2793	0.4817	0.5472	0.2467	0.5375	0.3731	0.3265	0.2576	0.8098	0.2755	0.4327	0.6867	0.4303	0.4373
-1	-1	-	-	-	0.8450	0.9780	0.2793	0.4817	0.5472	0.2467	0.5375	0.3731	0.3265	0.2576	0.8098	0.2755	0.4327	0.6867	0.4303	0.4373

Training Time

• Training: 55.3 minutes

Framework Versions

• Python: 3.11.6
• Sentence Transformers: 5.5.0.dev0
• Transformers: 5.5.0
• PyTorch: 2.10.0+cu128
• Accelerate: 1.13.0.dev0
• Datasets: 4.8.4
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

@misc{li2023generaltextembeddingsmultistage,
      title={Towards General Text Embeddings with Multi-stage Contrastive Learning},
      author={Zehan Li and Xin Zhang and Yanzhao Zhang and Dingkun Long and Pengjun Xie and Meishan Zhang},
      year={2023},
      eprint={2308.03281},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2308.03281},
}