Any-to-Any
Transformers
ONNX
Safetensors
English
Chinese
multimodal
audio
video
speech
streaming
full-duplex
long-video
custom-code
Instructions to use inclusionAI/Realtime-Venus with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use inclusionAI/Realtime-Venus with Transformers:
# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("inclusionAI/Realtime-Venus", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Use the canonical paper PDF link
Browse files- README.md +5 -5
- README_zh.md +5 -5
README.md
CHANGED
|
@@ -30,13 +30,13 @@ tags:
|
|
| 30 |
<a href="https://realtime-venus.github.io/"><img src="https://img.shields.io/badge/Project_Page-4c9aff.svg?logo=googlechrome&logoColor=white" alt="Project Page"></a>
|
| 31 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/Hugging_Face-Realtime--Venus-FFD21E.svg?logo=huggingface&logoColor=000" alt="Realtime-Venus on Hugging Face"></a>
|
| 32 |
<a href="https://www.modelscope.cn/models/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/ModelScope-Realtime--Venus-624AFF.svg?logo=modelscope&logoColor=white" alt="Realtime-Venus on ModelScope"></a>
|
| 33 |
-
<a href="https://arxiv.org/
|
| 34 |
<a href="https://github.com/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/GitHub-Realtime--Venus-181717.svg?logo=github&logoColor=white" alt="GitHub"></a>
|
| 35 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-0b7285.svg?logo=apache&logoColor=white" alt="Apache License 2.0"></a>
|
| 36 |
</p>
|
| 37 |
|
| 38 |
<p align="center">
|
| 39 |
-
<a href="https://arxiv.org/
|
| 40 |
<img src="https://arxiv.org/html/2609.13814v1/case.png" alt="Example proactive, delegated, and full-duplex interactions with Realtime-Venus" width="100%">
|
| 41 |
</a>
|
| 42 |
</p>
|
|
@@ -97,11 +97,11 @@ integrations live in the
|
|
| 97 |
## 4. 📊 Evaluation
|
| 98 |
|
| 99 |
All values are reported in the
|
| 100 |
-
[Realtime-Venus technical report](https://arxiv.org/
|
| 101 |
|
| 102 |
-
<p align="center"><img src="assets/paper-understanding.svg" width="100%" alt="Radar charts comparing video understanding for Omni and audio understanding for Audio" /><br /><sub>Figure 1. Video and audio understanding results from the <a href="https://arxiv.org/
|
| 103 |
|
| 104 |
-
<p align="center"><img src="assets/paper-duplex.svg" width="100%" alt="Full-duplex benchmark comparisons for interruption handling and continuation under different types of overlapping speech" /><br /><sub>Figure 2. Full-duplex interaction results from the <a href="https://arxiv.org/
|
| 105 |
|
| 106 |
## 5. 🗂️ Repository Layout
|
| 107 |
|
|
|
|
| 30 |
<a href="https://realtime-venus.github.io/"><img src="https://img.shields.io/badge/Project_Page-4c9aff.svg?logo=googlechrome&logoColor=white" alt="Project Page"></a>
|
| 31 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/Hugging_Face-Realtime--Venus-FFD21E.svg?logo=huggingface&logoColor=000" alt="Realtime-Venus on Hugging Face"></a>
|
| 32 |
<a href="https://www.modelscope.cn/models/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/ModelScope-Realtime--Venus-624AFF.svg?logo=modelscope&logoColor=white" alt="Realtime-Venus on ModelScope"></a>
|
| 33 |
+
<a href="https://arxiv.org/pdf/2609.13814"><img src="https://img.shields.io/badge/arXiv-2609.13814-b31b1b.svg?logo=arxiv&logoColor=white" alt="arXiv"></a>
|
| 34 |
<a href="https://github.com/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/GitHub-Realtime--Venus-181717.svg?logo=github&logoColor=white" alt="GitHub"></a>
|
| 35 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-0b7285.svg?logo=apache&logoColor=white" alt="Apache License 2.0"></a>
|
| 36 |
</p>
|
| 37 |
|
| 38 |
<p align="center">
|
| 39 |
+
<a href="https://arxiv.org/pdf/2609.13814">
|
| 40 |
<img src="https://arxiv.org/html/2609.13814v1/case.png" alt="Example proactive, delegated, and full-duplex interactions with Realtime-Venus" width="100%">
|
| 41 |
</a>
|
| 42 |
</p>
|
|
|
|
| 97 |
## 4. 📊 Evaluation
|
| 98 |
|
| 99 |
All values are reported in the
|
| 100 |
+
[Realtime-Venus technical report](https://arxiv.org/pdf/2609.13814).
|
| 101 |
|
| 102 |
+
<p align="center"><img src="assets/paper-understanding.svg" width="100%" alt="Radar charts comparing video understanding for Omni and audio understanding for Audio" /><br /><sub>Figure 1. Video and audio understanding results from the <a href="https://arxiv.org/pdf/2609.13814">paper</a>.</sub></p>
|
| 103 |
|
| 104 |
+
<p align="center"><img src="assets/paper-duplex.svg" width="100%" alt="Full-duplex benchmark comparisons for interruption handling and continuation under different types of overlapping speech" /><br /><sub>Figure 2. Full-duplex interaction results from the <a href="https://arxiv.org/pdf/2609.13814">paper</a>.</sub></p>
|
| 105 |
|
| 106 |
## 5. 🗂️ Repository Layout
|
| 107 |
|
README_zh.md
CHANGED
|
@@ -12,13 +12,13 @@
|
|
| 12 |
<a href="https://realtime-venus.github.io/"><img src="https://img.shields.io/badge/Project_Page-4c9aff.svg?logo=googlechrome&logoColor=white" alt="项目主页"></a>
|
| 13 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/Hugging_Face-Realtime--Venus-FFD21E.svg?logo=huggingface&logoColor=000" alt="Hugging Face 上的 Realtime-Venus"></a>
|
| 14 |
<a href="https://www.modelscope.cn/models/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/ModelScope-Realtime--Venus-624AFF.svg?logo=modelscope&logoColor=white" alt="ModelScope 上的 Realtime-Venus"></a>
|
| 15 |
-
<a href="https://arxiv.org/
|
| 16 |
<a href="https://github.com/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/GitHub-Realtime--Venus-181717.svg?logo=github&logoColor=white" alt="GitHub 代码仓库"></a>
|
| 17 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-0b7285.svg?logo=apache&logoColor=white" alt="Apache License 2.0"></a>
|
| 18 |
</p>
|
| 19 |
|
| 20 |
<p align="center">
|
| 21 |
-
<a href="https://arxiv.org/
|
| 22 |
<img src="https://arxiv.org/html/2609.13814v1/case.png" alt="Realtime-Venus 的主动交互、异步任务委派与全双工交互示例" width="100%">
|
| 23 |
</a>
|
| 24 |
</p>
|
|
@@ -58,11 +58,11 @@
|
|
| 58 |
|
| 59 |
## 4. 📊 评测
|
| 60 |
|
| 61 |
-
以下结果均引自 [Realtime-Venus 技术报告](https://arxiv.org/
|
| 62 |
|
| 63 |
-
<p align="center"><img src="assets/paper-understanding.svg" width="100%" alt="比较 Omni 视频理解能力与 Audio 音频理解能力的雷达图" /><br /><sub>图 1. <a href="https://arxiv.org/
|
| 64 |
|
| 65 |
-
<p align="center"><img src="assets/paper-duplex.svg" width="100%" alt="不同语音重叠场景下打断处理与继续发言能力的全双工基准对比" /><br /><sub>图 2. <a href="https://arxiv.org/
|
| 66 |
|
| 67 |
## 5. 🗂️ 仓库结构
|
| 68 |
|
|
|
|
| 12 |
<a href="https://realtime-venus.github.io/"><img src="https://img.shields.io/badge/Project_Page-4c9aff.svg?logo=googlechrome&logoColor=white" alt="项目主页"></a>
|
| 13 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/Hugging_Face-Realtime--Venus-FFD21E.svg?logo=huggingface&logoColor=000" alt="Hugging Face 上的 Realtime-Venus"></a>
|
| 14 |
<a href="https://www.modelscope.cn/models/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/ModelScope-Realtime--Venus-624AFF.svg?logo=modelscope&logoColor=white" alt="ModelScope 上的 Realtime-Venus"></a>
|
| 15 |
+
<a href="https://arxiv.org/pdf/2609.13814"><img src="https://img.shields.io/badge/arXiv-2609.13814-b31b1b.svg?logo=arxiv&logoColor=white" alt="arXiv 论文"></a>
|
| 16 |
<a href="https://github.com/inclusionAI/Realtime-Venus"><img src="https://img.shields.io/badge/GitHub-Realtime--Venus-181717.svg?logo=github&logoColor=white" alt="GitHub 代码仓库"></a>
|
| 17 |
<a href="https://huggingface.co/inclusionAI/Realtime-Venus/blob/main/LICENSE"><img src="https://img.shields.io/badge/License-Apache_2.0-0b7285.svg?logo=apache&logoColor=white" alt="Apache License 2.0"></a>
|
| 18 |
</p>
|
| 19 |
|
| 20 |
<p align="center">
|
| 21 |
+
<a href="https://arxiv.org/pdf/2609.13814">
|
| 22 |
<img src="https://arxiv.org/html/2609.13814v1/case.png" alt="Realtime-Venus 的主动交互、异步任务委派与全双工交互示例" width="100%">
|
| 23 |
</a>
|
| 24 |
</p>
|
|
|
|
| 58 |
|
| 59 |
## 4. 📊 评测
|
| 60 |
|
| 61 |
+
以下结果均引自 [Realtime-Venus 技术报告](https://arxiv.org/pdf/2609.13814)。
|
| 62 |
|
| 63 |
+
<p align="center"><img src="assets/paper-understanding.svg" width="100%" alt="比较 Omni 视频理解能力与 Audio 音频理解能力的雷达图" /><br /><sub>图 1. <a href="https://arxiv.org/pdf/2609.13814">论文</a>中的视频和音频理解评测结果。</sub></p>
|
| 64 |
|
| 65 |
+
<p align="center"><img src="assets/paper-duplex.svg" width="100%" alt="不同语音重叠场景下打断处理与继续发言能力的全双工基准对比" /><br /><sub>图 2. <a href="https://arxiv.org/pdf/2609.13814">论文</a>中的全双工交互评测结果。</sub></p>
|
| 66 |
|
| 67 |
## 5. 🗂️ 仓库结构
|
| 68 |
|