Instructions to use sapienzanlp/Minerva-7B-instruct-v1.0 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sapienzanlp/Minerva-7B-instruct-v1.0 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="sapienzanlp/Minerva-7B-instruct-v1.0")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("sapienzanlp/Minerva-7B-instruct-v1.0")
model = AutoModelForCausalLM.from_pretrained("sapienzanlp/Minerva-7B-instruct-v1.0", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use sapienzanlp/Minerva-7B-instruct-v1.0 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sapienzanlp/Minerva-7B-instruct-v1.0"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapienzanlp/Minerva-7B-instruct-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/sapienzanlp/Minerva-7B-instruct-v1.0

SGLang

How to use sapienzanlp/Minerva-7B-instruct-v1.0 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sapienzanlp/Minerva-7B-instruct-v1.0" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapienzanlp/Minerva-7B-instruct-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sapienzanlp/Minerva-7B-instruct-v1.0" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sapienzanlp/Minerva-7B-instruct-v1.0",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use sapienzanlp/Minerva-7B-instruct-v1.0 with Docker Model Runner:
```
docker model run hf.co/sapienzanlp/Minerva-7B-instruct-v1.0
```

../llama.cpp/convert_hf_to_gguf.py . --outfile minerva.gguff --outtype f16 : The BPE pre-tokenizer was not recognized!

by Raphy10-Collab - opened Jan 14, 2025

Discussion

Raphy10-Collab

Jan 14, 2025

I downloaded Minerva-7B-instruct-v1.0 :

(.venv) raphy@raohy:~/whisper.cpp/models$ cat download-minerva.py 
from huggingface_hub import snapshot_download
model_id="sapienzanlp/Minerva-7B-instruct-v1.0"
snapshot_download(repo_id=model_id, local_dir="minerva",
              local_dir_use_symlinks=False, revision="main")

And tried to convert it to gguff

(.venv) raphy@raohy:~/whisper.cpp/models/minerva$ ../llama.cpp/convert_hf_to_gguf.py . --outfile minerva.gguff  --outtype f16

But got this error :

INFO:hf-to-gguf:gguf: rms norm epsilon = 1e-05
INFO:hf-to-gguf:gguf: file type = 1
INFO:hf-to-gguf:Set model tokenizer
WARNING:hf-to-gguf:

WARNING:hf-to-gguf:**************************************************************************************
WARNING:hf-to-gguf:** WARNING: The BPE pre-tokenizer was not recognized!
WARNING:hf-to-gguf:**          There are 2 possible reasons for this:
WARNING:hf-to-gguf:**          - the model has not been added to convert_hf_to_gguf_update.py yet
WARNING:hf-to-gguf:**          - the pre-tokenization config has changed upstream
WARNING:hf-to-gguf:**          Check your model files and convert_hf_to_gguf_update.py and update them accordingly.
WARNING:hf-to-gguf:** ref:     https://github.com/ggerganov/llama.cpp/pull/6920
WARNING:hf-to-gguf:**
WARNING:hf-to-gguf:** chkhsh:  68fa7e0a33050885cc10a2acfa4df354042188f0afa03b809f7a71c4cde6e373
WARNING:hf-to-gguf:**************************************************************************************
WARNING:hf-to-gguf:

Traceback (most recent call last):
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 1566, in set_vocab
    self._set_vocab_sentencepiece()
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 789, in _set_vocab_sentencepiece
    tokens, scores, toktypes = self._create_vocab_sentencepiece()
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 806, in _create_vocab_sentencepiece
    raise FileNotFoundError(f"File not found: {tokenizer_path}")
FileNotFoundError: File not found: tokenizer.model

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 1569, in set_vocab
    self._set_vocab_llama_hf()
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 881, in _set_vocab_llama_hf
    vocab = gguf.LlamaHfVocab(self.dir_model)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/gguf-py/gguf/vocab.py", line 390, in __init__
    raise FileNotFoundError('Cannot find Llama BPE tokenizer')
FileNotFoundError: Cannot find Llama BPE tokenizer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 5077, in <module>
    main()
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 5071, in main
    model_instance.write()
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 440, in write
    self.prepare_metadata(vocab_only=False)
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 433, in prepare_metadata
    self.set_vocab()
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 1572, in set_vocab
    self._set_vocab_gpt2()
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 725, in _set_vocab_gpt2
    tokens, toktypes, tokpre = self.get_vocab_base()
                               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 526, in get_vocab_base
    tokpre = self.get_vocab_base_pre(tokenizer)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/raphy/whisper.cpp/models/minerva/../llama.cpp/convert_hf_to_gguf.py", line 713, in get_vocab_base_pre
    raise NotImplementedError("BPE pre-tokenizer was not recognized - update get_vocab_base_pre()")
NotImplementedError: BPE pre-tokenizer was not recognized - update get_vocab_base_pre()

How to make it work?

iperbole

Sapienza NLP, Sapienza University of Rome org Jan 20, 2025

Hi! as first try to use the latest version of llama.cpp, if you are using an old version. Let us know if it solve the problem.

Raphy10-Collab

Jan 20, 2025

Hi!

I git cloned, compiled and built, the latest version of llama.cpp and used the just compiled binary files. But the problem persists :

raphy@raohy:~$ git clone --recurse-submodules https://github.com/ggerganov/llama.cpp
Cloning into 'llama.cpp'..

raphy@raohy:~/llama.cpp$ cmake -B builddir
-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 14.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.43.0")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- ccache found, compilation results will be cached. Disable with GGML_CCACHE=OFF.
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native
-- Configuring done (0.7s)
-- Generating done (0.1s)
-- Build files have been written to: /home/raphy/llama.cpp/builddir


raphy@raohy:~/llama.cpp$ cmake --build builddir/ --config Release
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml.c.o
[  1%] Building C object ggml/src/CMakeFiles/ggml-base.dir/ggml-alloc.c.o

[ 99%] Building CXX object pocs/vdot/CMakeFiles/llama-q8dot.dir/q8dot.cpp.o
[100%] Linking CXX executable ../../bin/llama-q8dot
[100%] Built target llama-q8dot



(.venv) raphy@raohy:~/whisper.cpp/models/models--sapienzanlp--Minerva-7B-instruct-v1.0$ ../../../llama.cpp/convert_hf_to_gguf.py . --outfile minerva.gguff  --outtype f16
INFO:hf-to-gguf:Loading model: 
Traceback (most recent call last):
  File "/home/raphy/whisper.cpp/models/models--sapienzanlp--Minerva-7B-instruct-v1.0/../../../llama.cpp/convert_hf_to_gguf.py", line 5140, in <module>
    main()
  File "/home/raphy/whisper.cpp/models/models--sapienzanlp--Minerva-7B-instruct-v1.0/../../../llama.cpp/convert_hf_to_gguf.py", line 5108, in main
    hparams = Model.load_hparams(dir_model)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/raphy/whisper.cpp/models/models--sapienzanlp--Minerva-7B-instruct-v1.0/../../../llama.cpp/convert_hf_to_gguf.py", line 468, in load_hparams
    with open(dir_model / "config.json", "r", encoding="utf-8") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'config.json'

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment