Instructions to use NyayaLabs98/nyaya-3b-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NyayaLabs98/nyaya-3b-v3 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NyayaLabs98/nyaya-3b-v3")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NyayaLabs98/nyaya-3b-v3")
model = AutoModelForCausalLM.from_pretrained("NyayaLabs98/nyaya-3b-v3", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use NyayaLabs98/nyaya-3b-v3 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NyayaLabs98/nyaya-3b-v3"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NyayaLabs98/nyaya-3b-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NyayaLabs98/nyaya-3b-v3

SGLang

How to use NyayaLabs98/nyaya-3b-v3 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NyayaLabs98/nyaya-3b-v3" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NyayaLabs98/nyaya-3b-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NyayaLabs98/nyaya-3b-v3" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NyayaLabs98/nyaya-3b-v3",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use NyayaLabs98/nyaya-3b-v3 with Docker Model Runner:
```
docker model run hf.co/NyayaLabs98/nyaya-3b-v3
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Nyaya-3B — the model component of the Nyaya legal guidance system

⚖️ Not legal advice. Nyaya provides legal information. The practice of law in India is reserved to advocates enrolled under the Advocates Act, 1961. Consult a licensed advocate for anything consequential. Free legal aid is available through NALSA / DLSA (Legal Services Authorities Act, 1987).

📋 Non-commercial licence. The base model Qwen/Qwen2.5-3B-Instruct is released under the Qwen Research License — not Apache-2.0. The 3B is one of the Qwen2.5 sizes carrying the restricted licence, and these merged weights inherit it: research / non-commercial use only.

What this is, stated plainly

This is the model that reads retrieved statute sections and writes a cited answer. On the project's own benchmark its accuracy is statistically tied with the base model it was fine-tuned from. The system's measured gains come from retrieval, not from these weights.

It is published because a single download that works — model, prompt format and citation style already aligned with the Nyaya retriever — is more useful than assembling the pieces yourself. It is not published as an improvement over Qwen/Qwen2.5-3B-Instruct, because it is not one.

Use it with NyayaLabs98/nyaya-statute-db and the retriever from the repository. Used bare, without retrieval, it behaves close to the base model.

Evaluation

Nyaya-Eval-v1, 409 gradeable questions, paired comparison, 10,000-round bootstrap. Same retriever, same questions — only the weights differ.

	fact recall	citation accuracy
`Qwen2.5-3B-Instruct` + RAG	34.3%	52.8%
Nyaya-3B-v3 + RAG	32.9%	50.3%

95% CI on the paired difference spans zero → statistically indistinguishable.

Where the accuracy actually comes from

retrieval outcome	n	fact recall
gold statute retrieved	94	63.2%
gold statute missed	43	17.1%

That 46-point gap is why this project's effort moved to retrieval. Adding a cross-encoder reranker put the correct section in the top result for 58.5% of questions, up from 45.8% — validated on records never used for tuning.

Fine-tuning attempts, for the record

	fact recall	vs base
base	34.3%	—
v3 (this model, RAFT)	32.9%	tied
v5 (grounded citation data)	24.0%	worse, CI [−13.5, −7.2]
v6 (v5 + answer-style fix)	23.4%	worse, CI [−14.0, −7.8]

None beat base. v5 and v6 regressed because training on short templated targets shortened the answers (173 → 90 → 57 words), and shorter answers carry fewer of the facts being scored.

Honest status

No external benchmark comparison has been run. No claim is made against any other legal model.
No human evaluation has been passed.
The project's earlier benchmark (Eval-v0) scored its own gold answers at 10.7%, so any accuracy figure derived from it is meaningless — including numbers previously shown on this card.
Coverage is limited to the 16 acts in the statute DB.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained("NyayaLabs98/nyaya-3b-v3")
model = AutoModelForCausalLM.from_pretrained("NyayaLabs98/nyaya-3b-v3", device_map="auto")

system = (
    "You are Nyaya, an Indian legal information model. You provide accurate, "
    "plain-language legal guidance for Indian citizens, cite specific sections of "
    "current law (BNS/BNSS/BSA and other acts in force), clearly state uncertainty, "
    "and recommend consulting a licensed advocate for anything consequential. "
    "You provide legal information, not legal advice."
)
# Prepend retrieved statute passages to the user turn — that is where the
# accuracy comes from. See the repo's nyaya.retrieval.build_rag_prompt.
messages = [
    {"role": "system", "content": system},
    {"role": "user", "content": "Police FIR nahi likh rahi, kya karu?"},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
out = model.generate(inputs, max_new_tokens=512)
print(tok.decode(out[0][inputs.shape[1]:], skip_special_tokens=True))

Licence & attribution

Research / non-commercial only. Merged LoRA derivative of Qwen/Qwen2.5-3B-Instruct (qwen-research). The training/eval code is Apache-2.0; that licence does not extend to these weights.

Statutory text is Government of India material, public domain under Section 52(1)(q) of the Copyright Act, 1957.

Code, evaluation harness, and the full record of what did and did not work: github.com/JitendraJha98/nyaya-model

Downloads last month: 596

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for NyayaLabs98/nyaya-3b-v3

Base model

Qwen/Qwen2.5-3B

Finetuned

Qwen/Qwen2.5-3B-Instruct

Finetuned

(1509)

this model

Quantizations

1 model