Instructions to use togethercomputer/RedPajama-INCITE-7B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use togethercomputer/RedPajama-INCITE-7B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="togethercomputer/RedPajama-INCITE-7B-Instruct")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct")
model = AutoModelForCausalLM.from_pretrained("togethercomputer/RedPajama-INCITE-7B-Instruct")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use togethercomputer/RedPajama-INCITE-7B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "togethercomputer/RedPajama-INCITE-7B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "togethercomputer/RedPajama-INCITE-7B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/togethercomputer/RedPajama-INCITE-7B-Instruct

SGLang

How to use togethercomputer/RedPajama-INCITE-7B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "togethercomputer/RedPajama-INCITE-7B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "togethercomputer/RedPajama-INCITE-7B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "togethercomputer/RedPajama-INCITE-7B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "togethercomputer/RedPajama-INCITE-7B-Instruct",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use togethercomputer/RedPajama-INCITE-7B-Instruct with Docker Model Runner:
```
docker model run hf.co/togethercomputer/RedPajama-INCITE-7B-Instruct
```

Prohibited misuse

by Aspie96 - opened Jul 4, 2023

Discussion

Aspie96

Jul 4, 2023

RedPajama-INCITE-7B-Instruct is designed for language modeling. Misuse of the model, such as using it to engage in illegal or unethical activities, is strictly prohibited and goes against the principles of the project.

Prohibited by whom?

What law, contract or other legal documents is prohibiting unethical activities? The license sure isn't.

Those activities can be strongly recommended against, vilified, discouraged, shunned and frowned upon, but I think describing them as "prohibited" is potentially misleading and could lead to misunderstandings about the licensing and legal status of the model.

amaliak changed discussion status to closed Jul 8, 2023

Aspie96

Jul 8, 2023

@amaliak I think the question was legitimate. May I ask why it was closed with no answer?

vipul

Together org Jul 8, 2023

The license is Apache 2.0. Open source AI licensing is an active area of work and this is the best current view on how to release open models while limiting potential misuses of them.

Aspie96

Jul 8, 2023

while limiting potential misuses of them

My question is what is that limits them. It doesn't appear to be limited in any way.

someone13574

Jul 8, 2023

The licence is Apache 2.0.

Exactly, and Apache 2.0 doesn’t prohibit that kind of use, meaning it is either not prohibited or there is a direct contradiction regarding how the model can be used, thus this question.

OllieStanley

Jul 8, 2023

Seems that the "prohibited" part of the readme is not part of the license and therefore from a legal POV can be ignored in favour of the terms of the license.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment