Instructions to use SungJoo/llama2-7b-sft-detox with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use SungJoo/llama2-7b-sft-detox with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="SungJoo/llama2-7b-sft-detox")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SungJoo/llama2-7b-sft-detox")
model = AutoModelForCausalLM.from_pretrained("SungJoo/llama2-7b-sft-detox", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use SungJoo/llama2-7b-sft-detox with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "SungJoo/llama2-7b-sft-detox"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SungJoo/llama2-7b-sft-detox",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/SungJoo/llama2-7b-sft-detox

SGLang

How to use SungJoo/llama2-7b-sft-detox with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "SungJoo/llama2-7b-sft-detox" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SungJoo/llama2-7b-sft-detox",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "SungJoo/llama2-7b-sft-detox" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "SungJoo/llama2-7b-sft-detox",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use SungJoo/llama2-7b-sft-detox with Docker Model Runner:
```
docker model run hf.co/SungJoo/llama2-7b-sft-detox
```

Model Card for SungJoo/llama2-7b-sft-detox

This model is an instruction-tuned version of meta-llama/Llama-2-7b-hf, fine-tuned to reduce toxicity in Large Language Models (LLMs).

Model Details

Model Description

This is an instruction-tuned model based on the LLaMA-2-7b architecture. It has been fine-tuned using a comprehensive instruction dataset specifically designed for detoxification of LLMs.

Developed by: Sungjoo Byun (Grace Byun)
Model type: Auto-regressive language model
Language(s) (NLP): English
License: Apache License 2.0
Finetuned from: meta-llama/Llama-2-7b-hf

Model Sources

Repository: TBD
Paper: TBD

Uses

This model is intended to be used for generating less toxic language in various applications, including chatbots and other NLP systems.

Bias, Risks, and Limitations

While this model aims to reduce toxicity, it may still generate biased or harmful content. Users should apply this model with caution and review outputs for sensitive applications.

How to Get Started with the Model

Use the code below to get started with the model:

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("SungJoo/llama2-7b-sft-detox")
model = AutoModelForCausalLM.from_pretrained("SungJoo/llama2-7b-sft-detox")

Training Details

Training Data

The model was fine-tuned using a dataset specifically created to detoxify LLMs. This dataset will be publicly available soon.

Training Procedure

The model was trained using full fine-tuning with the following hyperparameters:

Hyperparameter	Value
Batch size	128
Learning rate	2e-5
Epochs	3
Max length	512
Weight decay	0

The training was conducted on 4 A100 80GB GPUs.

Objective

The main objective of this research is to reduce toxicity in LLMs by applying instruction tuning and Direct Preference Optimization (DPO). This version has been tuned with instruction tuning only. A comprehensive instruction and DPO dataset was constructed for this purpose, which will be released in the future.

Model	LLaMA-2-base		Finetuned LLaMA-2		DPO LLaMA-2
Category	>=0.5 (%)	Count	>=0.5 (%)	Count	>=0.5 (%)	Count
TOXICITY	4.46	4,438	3.61	3,593	2.39	2,377
			(-0.85)	(-845)	(-1.22)	(-1,216)
SEVERE_TOXICITY	0.08	77	0.07	70	0.03	31
			(-0.01)	(-7)	(-0.04)	(-39)
IDENTITY_ATTACK	0.79	788	0.42	413	0.28	274
			(-0.37)	(-375)	(-0.14)	(-139)
INSULT	1.97	1,961	1.60	1,588	0.90	892
			(-0.37)	(-373)	(-0.70)	(-696)
PROFANITY	2.10	2,086	1.76	1,753	1.04	1,030
			(-0.34)	(-333)	(-0.72)	(-723)
THREAT	1.43	1,424	0.92	919	0.76	754
			(-0.51)	(-505)	(-0.16)	(-165)
Comparison of LLaMA-2-base, Finetuned LLaMA-2, and DPO LLaMA-2 across various categories. Reductions in blue indicate comparisons between the base model and the fine-tuned model, while text in green represents comparisons between the fine-tuned model and the DPO model.

The table above shows the effectiveness of this model in reducing bias, measured using the RealToxicityPrompt dataset and the Perspective API.

Contact

For any questions or issues, please contact byunsj@snu.ac.kr.

Downloads last month: 6

Safetensors

Model size

7B params

Tensor type

F32