Instructions to use Motif-Technologies/Motif-3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Motif-Technologies/Motif-3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Motif-Technologies/Motif-3", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModel
model = AutoModel.from_pretrained("Motif-Technologies/Motif-3", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Motif-Technologies/Motif-3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Motif-Technologies/Motif-3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Motif-Technologies/Motif-3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Motif-Technologies/Motif-3

SGLang

How to use Motif-Technologies/Motif-3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Motif-Technologies/Motif-3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Motif-Technologies/Motif-3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Motif-Technologies/Motif-3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Motif-Technologies/Motif-3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Motif-Technologies/Motif-3 with Docker Model Runner:
```
docker model run hf.co/Motif-Technologies/Motif-3
```

Custom vLLM merge request to main vLLM?

by pathosethoslogos - opened 4 days ago

Discussion

pathosethoslogos

4 days ago

Hi, I just saw this thread: https://huggingface.co/Motif-Technologies/Motif-3-NVFP4/discussions/1

If you get your code merged, they automatically build dozens of different Docker image configurations for various hardware. It would be amazing if you can get this passed. You can check some of their Docker images on Docker Hub under vllm/vllm-openai.

This would definitely help getting it mainstream.

iamwyldecat

Motif Technologies org 4 days ago

@pathosethoslogos
Thanks for the information 😀
We don't have any plans to merge upstream vLLM at the moment, but we'll definitely consider it.

GadflyII

3 days ago

•

edited 3 days ago

Do you mind If I submit the PR?

JH-Motif

Motif Technologies org 3 days ago

@GadflyII Of course not! We’d be more than happy. We always appreciate anything that helps make our model more accessible to more people. 😊

GadflyII

3 days ago

Great, I will get started on that, and again, Great work on your v3 model. It is really good.

TaehyunKimMotif

Motif Technologies org 3 days ago

@GadflyII

Thank you very much for your continued interest and contributions.

The images in our model card, as well as the public Motif vLLM branch, were developed based on the vLLM 0.20 release. Because of this, attempting to open a PR directly against the current main branch would likely involve a significant number of conflicts and substantial code changes.

We have another branch that we have been working on based on vLLM 0.26, and we would like to share it with you. There are still some parts of the code that are not yet suitable for upstreaming to main and some areas that need further cleanup, but we hope this branch will be helpful when preparing your PR.
(https://github.com/MotifTechnologies/vllm-motif3-v0.26.0)

Once again, we sincerely appreciate your interest and contributions. If there is anything else we can help with, or if you have any questions, please feel free to reach out at any time.

Thank you again.

pathosethoslogos

3 days ago

Great, I will get started on that, and again, Great work on your v3 model. It is really good.

Amazing!

Can you link your merge request?

pathosethoslogos

3 days ago

We have another branch that we have been working on based on vLLM 0.26, and we would like to share it with you. There are still some parts of the code that are not yet suitable for upstreaming to main and some areas that need further cleanup, but we hope this branch will be helpful when preparing your PR.
(https://github.com/MotifTechnologies/vllm-motif3-v0.26.0)

Also amazing

Once this gets pushed, 100% people will find some strengths of this model over the V4 Flash 0731, and will preach + spread super fast.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment