openbmb

/

MiniCPM-o-2_6-gguf

like 119

Follow

OpenBMB 4.39k

I get an error message as well running this on Ollama. I'm not desperate though, I hope this can be solved in the coming weeks. 🙂

same problem

Hello, it is based on llama.cpp and can use gguf to inference, then my changes have not been merged into the official yet, and you may need to wait until the merge.

I run this command:
ollama run hf.co/openbmb/MiniCPM-o-2_6-gguf:Q4_K_M

and has this error:
Error: llama runner process has terminated: exit status 2

Specs:

• CUDA: v12
• GPU T4 Tesla
• Running in SageMaker instance (AWS)
• Python 3.11.11

Model has been downloaded but failed to run. I tried ollama list and confirmed that model has been downloaded:
hf.co/openbmb/MiniCPM-o-2_6-gguf:Q4_K_M 577cda2e3886 5.7 GB 21 seconds ago

I also tried running on llama-cpp-python but it also has a lot of errors.

Please help. I am desperate.
Thank you very much.

same for me

Same here

ollama run hf.co/openbmb/MiniCPM-o-2_6-gguf:latest
python --version
Python 3.12.7

system_profiler SPDisplaysDataType
Graphics/Displays:

Apple M4 Max:

  Chipset Model: Apple M4 Max
  Type: GPU
  Bus: Built-In
  Total Number of Cores: 40
  Vendor: Apple (0x106b)
  Metal Support: Metal 3

I download the MiniCPM-o_2.6 Q_8 version (almost 8GB), and create with ollama create, it works but when I asking some question, the model answers other things, I assume there is no context for the model

Same thing here! I downloaded MiniCPM-o-2_6 from the official Ollama site and it seems to work correctly in Windows CLI. However this is a Q4 model and its complete name (as listed by Ollama) is: ZimaBlueAI/MiniCpm-o-2_6:latest. Unfortunately there are no other quantization models in the "ZimaBlueAI" directory, q4 is the only one and it comes with out the projector-model from the original hf directory (hf.co/openbmb/MiniCPM-o-2_6-gguf). This is the place where I downloaded the Q6 and the Q8, but with the same result. It seems the 'exit-status 2' failure has something to do with the projector-model, witch came with both, Q6 and Q8, but not with the ZimaBlue-Q4 version. I have tried to run the Q6 model by manipulating its 'manifest-file' in the local ollama directory of my system, by excluding the projector-model. Voila! This time it started without error message. However the model wasn't capable to answer any question, instead started to go wild and run in loops etc...

still facing the same error , did anyone figured it out please make a post on r/LocalLLaMA i

Our model may not yet support direct deployment with Ollama, we will push forward as soon as possible

Thank you for keeping us updated!!

same error I got, What should be the solution?

https://ollama.com/openbmb/minicpm-o2.6

You can directly use our adapted ollama model. Run ollama with one line of code.

ollama run openbmb/minicpm-o2.6