Image-Text-to-Text
PEFT
Safetensors
lora
qwen2_5_vl
video
video-language-model
lab-supervision
wet-lab
protocol-monitoring
error-detection
spatial-grounding
Instructions to use cong-lab/labos-vlm-32b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use cong-lab/labos-vlm-32b with PEFT:
from peft import PeftModel from transformers import AutoModelForCausalLM base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-VL-32B-Instruct") model = PeftModel.from_pretrained(base_model, "cong-lab/labos-vlm-32b") - Notebooks
- Google Colab
- Kaggle
Update model card
Browse files
README.md
CHANGED
|
@@ -20,7 +20,7 @@ tags:
|
|
| 20 |
|
| 21 |
LabOS-VLM-32B is a PEFT LoRA adapter for `Qwen/Qwen2.5-VL-32B-Instruct`, fine-tuned on wet-lab supervision tasks from the LabOS JoVE and FineBio datasets and validated on the LSV benchmark. These tasks include protocol monitoring/step prediction, error detection, spatial grounding, protocol generation, and general VQA with first-person and third-person views.
|
| 22 |
|
| 23 |
-
The adapter is intended for research on laboratory video-language assistants. It works best with the JSON-style monitoring and benchmark prompts used by the LabOS datasets, such as the public LSV benchmark at [`cong-lab/lsv`](https://huggingface.co/datasets/cong-lab/lsv). Note, expected performance may drop
|
| 24 |
|
| 25 |
## Adapter Details
|
| 26 |
|
|
|
|
| 20 |
|
| 21 |
LabOS-VLM-32B is a PEFT LoRA adapter for `Qwen/Qwen2.5-VL-32B-Instruct`, fine-tuned on wet-lab supervision tasks from the LabOS JoVE and FineBio datasets and validated on the LSV benchmark. These tasks include protocol monitoring/step prediction, error detection, spatial grounding, protocol generation, and general VQA with first-person and third-person views.
|
| 22 |
|
| 23 |
+
The adapter is intended for research on laboratory video-language assistants. It works best with the JSON-style monitoring and benchmark prompts used by the LabOS datasets, such as the public LSV benchmark at [`cong-lab/lsv`](https://huggingface.co/datasets/cong-lab/lsv). Note, the expected performance may drop when evaluating on different modalities, or prompts, as is typically expected with SFT.
|
| 24 |
|
| 25 |
## Adapter Details
|
| 26 |
|