Instructions to use NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3")
model = AutoModelForCausalLM.from_pretrained("NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3

SGLang

How to use NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 with Docker Model Runner:
```
docker model run hf.co/NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Disclaimer:

This model is experimental, do not expect everything to work.

This model uses the Alpaca prompting format(or just directly download the SillyTavern instruct preset here)

Beeg noromaid on steroids. Suitable for RP, ERP.

This time based on Mixtral Instruct, seems to do wonders!

This model was trained for 8h(v1) + 8h(v2) + 12h(v3) on customized modified datasets, focusing on RP, uncensoring, and a modified version of the Alpaca prompting (that was already used in LimaRP), which should be at the same conversational level as ChatLM or Llama2-Chat without adding any additional special tokens.

If you wanna have more infos about this model(and v1 + v2) you can check out my blog post

Recommended settings - Settings 1

Credits:

Undi
IkariDev

Description

This repo contains FP16 files of Noromaid-v0.1-mixtral-8x7b-Instruct-v3.

FP16 - by IkariDev and Undi

GGUF - by IkariDev and Undi

Ratings:

Note: We have permission of all users to upload their ratings, we DONT screenshot random reviews without asking if we can put them here!

No ratings yet!

If you want your rating to be here, send us a message over on DC and we'll put up a screenshot of it here. DC name is "ikaridev" and "undi".

Custom format:

### Instruction:
{system prompt}

### Input:
{input}

### Response:
{reply}

Datasets used:

Aesir 1 and 2 (MinervaAI / Gryphe)
LimaRP-20231109 (Lemonilia)
ToxicDPO-NoWarning (unalignment orga repo + Undi)
No-robots-ShareGPT (Doctor-Shotgun)

Others

Undi: If you want to support me, you can here.

IkariDev: Visit my retro/neocities style website please kek

Downloads last month: 82

Model tree for NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3

Merges

10 models

Quantizations

5 models

NeverSleep
/

Noromaid-v0.1-mixtral-8x7b-Instruct-v3