Instructions to use Aratako/gemma-4-E4B-it-audio-encoder with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Aratako/gemma-4-E4B-it-audio-encoder with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="Aratako/gemma-4-E4B-it-audio-encoder", trust_remote_code=True)# Load model directly from transformers import AutoProcessor, AutoModel processor = AutoProcessor.from_pretrained("Aratako/gemma-4-E4B-it-audio-encoder", trust_remote_code=True) model = AutoModel.from_pretrained("Aratako/gemma-4-E4B-it-audio-encoder", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Update gemma4_audio_encoder.py
Browse files- gemma4_audio_encoder.py +2 -2
gemma4_audio_encoder.py
CHANGED
|
@@ -16,7 +16,7 @@ class Gemma4AudioEncoder(PreTrainedModel):
|
|
| 16 |
"""
|
| 17 |
Args:
|
| 18 |
input_features: Audio mel-spectrogram features.
|
| 19 |
-
input_features_mask: Attention mask for audio features.
|
| 20 |
project: If True, project to LLM embedding space (2560-dim).
|
| 21 |
If False, return audio tower output (1536-dim).
|
| 22 |
|
|
@@ -24,7 +24,7 @@ class Gemma4AudioEncoder(PreTrainedModel):
|
|
| 24 |
If project=True: (projected_features, attention_mask)
|
| 25 |
If project=False: (encoder_features, attention_mask)
|
| 26 |
"""
|
| 27 |
-
output = self.audio_tower(input_features,
|
| 28 |
if project:
|
| 29 |
projected = self.embed_audio(inputs_embeds=output.last_hidden_state)
|
| 30 |
return projected, output.attention_mask
|
|
|
|
| 16 |
"""
|
| 17 |
Args:
|
| 18 |
input_features: Audio mel-spectrogram features.
|
| 19 |
+
input_features_mask: Attention mask for audio features (True = valid, False = padding).
|
| 20 |
project: If True, project to LLM embedding space (2560-dim).
|
| 21 |
If False, return audio tower output (1536-dim).
|
| 22 |
|
|
|
|
| 24 |
If project=True: (projected_features, attention_mask)
|
| 25 |
If project=False: (encoder_features, attention_mask)
|
| 26 |
"""
|
| 27 |
+
output = self.audio_tower(input_features, input_features_mask)
|
| 28 |
if project:
|
| 29 |
projected = self.embed_audio(inputs_embeds=output.last_hidden_state)
|
| 30 |
return projected, output.attention_mask
|