Instructions to use intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound")
model = AutoModelForCausalLM.from_pretrained("intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound

SGLang

How to use intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound with Docker Model Runner:
```
docker model run hf.co/intel-ai/GLM-5.1-MXFP4-Mixed-CT-AutoRound
```

INC4AI commited on 29 days ago

Commit

eac9ff8

verified ·

1 Parent(s): d2828a8

Add files using upload-large-folder tool

Browse files

This view is limited to 50 files because it contains too many changes. See raw diff

Files changed (50) hide show

.gitattributes +2 -0
model-00001-of-00282.safetensors +3 -0
model-00017-of-00282.safetensors +3 -0
model-00018-of-00282.safetensors +3 -0
model-00022-of-00282.safetensors +3 -0
model-00023-of-00282.safetensors +3 -0
model-00025-of-00282.safetensors +3 -0
model-00028-of-00282.safetensors +3 -0
model-00032-of-00282.safetensors +3 -0
model-00037-of-00282.safetensors +3 -0
model-00039-of-00282.safetensors +3 -0
model-00040-of-00282.safetensors +3 -0
model-00062-of-00282.safetensors +3 -0
model-00065-of-00282.safetensors +3 -0
model-00066-of-00282.safetensors +3 -0
model-00067-of-00282.safetensors +3 -0
model-00069-of-00282.safetensors +3 -0
model-00073-of-00282.safetensors +3 -0
model-00078-of-00282.safetensors +3 -0
model-00080-of-00282.safetensors +3 -0
model-00082-of-00282.safetensors +3 -0
model-00086-of-00282.safetensors +3 -0
model-00092-of-00282.safetensors +3 -0
model-00093-of-00282.safetensors +3 -0
model-00095-of-00282.safetensors +3 -0
model-00097-of-00282.safetensors +3 -0
model-00102-of-00282.safetensors +3 -0
model-00104-of-00282.safetensors +3 -0
model-00111-of-00282.safetensors +3 -0
model-00117-of-00282.safetensors +3 -0
model-00121-of-00282.safetensors +3 -0
model-00128-of-00282.safetensors +3 -0
model-00129-of-00282.safetensors +3 -0
model-00135-of-00282.safetensors +3 -0
model-00136-of-00282.safetensors +3 -0
model-00137-of-00282.safetensors +3 -0
model-00141-of-00282.safetensors +3 -0
model-00143-of-00282.safetensors +3 -0
model-00145-of-00282.safetensors +3 -0
model-00152-of-00282.safetensors +3 -0
model-00157-of-00282.safetensors +3 -0
model-00167-of-00282.safetensors +3 -0
model-00173-of-00282.safetensors +3 -0
model-00177-of-00282.safetensors +3 -0
model-00180-of-00282.safetensors +3 -0
model-00183-of-00282.safetensors +3 -0
model-00184-of-00282.safetensors +3 -0
model-00187-of-00282.safetensors +3 -0
model-00188-of-00282.safetensors +3 -0
model-00191-of-00282.safetensors +3 -0

.gitattributes CHANGED Viewed

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text

 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+model.safetensors.index.json filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

model-00001-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:99cc1089f5635e12efb865bfb66dcbe5a42c3a3ba38ffc4bacc7b55f10893fec
+size 5055602840

model-00017-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:895ea34c3edcac202449019b879468fa4d76faff4af9bee340928f5d9896ccf6
+size 1423889896

model-00018-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2b635ba5481da522f583f17548c4a05b7dfcd674c4ca69d6e2c2b628872d714f
+size 1423889880

model-00022-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ea2a9be72e4f0cfe920d3b77b9611eb82d34e23f0f3e5b740a3146f503c63d5e
+size 1423889840

model-00023-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:55572b4fe363113009ea365c2dfd968ca50ed67109f5ab110da0a4e3c47b6aa3
+size 1423889472

model-00025-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0670a2ab7377ae9522f07a017431a6cc5e4f3bd23c7563e8859a880187f9f8e4
+size 1423889888

model-00028-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5aa21064d8898093d5ff2b9e30f6b051940a3a8efdba85377976224816a0a4ec
+size 1423889896

model-00032-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a3ccbaf57f80e971ac99047e95a093dadb08f6146a216130024457b9de1c7cb3
+size 1423889896

model-00037-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:80c04d9a928cd328bc983f424fc5f61ead06490ca3d49133e49a9dee8759b031
+size 1423889744

model-00039-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6a080e55fdb2fca2ad53fd3f10c58e60e8cbb231c0d1417cb854d9b65913b138
+size 1444742200

model-00040-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6b8e73c1eb574f909575714e63094ee0c7040cc74b23e587cecefa4f7013c076
+size 1423889888

model-00062-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:406b0dcd9a7bba917405b1bac76f01802f686e2ca9ee6ca1f6192e24cd4e0ba3
+size 1423889888

model-00065-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c5b3219027a305ae7b2640d4bd8533d77dc4c468291acaa5b56bb1f4c86f4ee8
+size 1423889888

model-00066-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1ea5d36d2505d5647566c8a399052ba5698a8a7f756e5eae6036a6ae3978c08f
+size 1423889888

model-00067-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c3e9e9a79f5ac38599c9dca8d10be190f956d73d05c7dc1102104d62a724f6a9
+size 1423889584

model-00069-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1310647e20ea21a051712b436df98e46ea8c037c42a34012a83c071cb5b9f5a6
+size 1423889896

model-00073-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f66e856b6c123bc415595543ab073332803aa4aed5d89692682e9a0619562eb2
+size 1423889880

model-00078-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:97c87395c237efe15c83ebb5d1b4b7443a51b2d7d0616e007eb795f834249d04
+size 1423889192

model-00080-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:52ada44d3a3be2a1b2bb236d1a8a1f4dc949996ef31277169e18f203546518b0
+size 1423889888

model-00082-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ef86c774f0a28493396ac2e7db5fcdb6ee756d3641108f5d88227d2800c4120
+size 1423889480

model-00086-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6699e798092bae6a70a001ee1f677af79a3218abd850333e9ace3f5dbdde659f
+size 1541274272

model-00092-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ab51f672445ff0f1492888c78aecf901c11f1c63b06e5e2768f6a4088ec59dc4
+size 1423889880

model-00093-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a99a8d901035efb11bf28b2466dcb50258155ce0cbd922a5679deb5cd6873d9d
+size 1423889504

model-00095-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:75ce8e21c6eb951490e30dec21f2151ffe452dec9519ca89d31d273fd640a31e
+size 1423889888

model-00097-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b2477cff7d4577a3ac31777c4da52e05b812bc8e12bcf6b4f38ed6e0ea23166b
+size 1541274264

model-00102-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e235ee3535ad1ab3a7c98949f0d64457ee885891a764626ee3d9c0e8b404a217
+size 1423889888

model-00104-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c6068da96f8f319ea1c92098db435bce4c75c4e8e768c239ad71d66a398e56f6
+size 1423889520

model-00111-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:007b605b49e90eef2b37cfd768f3180846da4c5b43d77f9ff2482cea1eb5ba2b
+size 1423889696

model-00117-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2f90341d015306602783bfa3a65a8f9d635fceb411824c9cb01c109ca29e1253
+size 1423889472

model-00121-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5dea3319b49714231287f4476600ab88deabacaacc6af522536d6dcc09da0b81
+size 1423889888

model-00128-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:755d6c41a0297f1e35f33f83b92122f5e7e6e81074b4b223c2be17f8fd831ec6
+size 1423889896

model-00129-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:685229e8c9a4077a13a9227d56bfd107d56cbacf252a6ae5268cda74fcf9a0e6
+size 1423889864

model-00135-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9d5438b2565f85235ca5ed319694b75234bf9fc9ddddc2e204035c860a26a7db
+size 1423889888

model-00136-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b307ddacd89385321bb5f5607a1e1f12b29a0b2034a0d30ace08ef8d069ec94e
+size 1423889888

model-00137-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ab2c6125f5fee84c592994dd30053093643b9036f969582183aa323f16cd8f31
+size 1423889608

model-00141-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:615aae7e1b493428519b35d8e71c4808f6840cd1c542eb2121598f460a785ce4
+size 1423889472

model-00143-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8aee2d12ddddb98dd6f6e136a46439921d00e2d5467bd021cb69fb5e0656f77c
+size 1423889880

model-00145-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9efe1e91d5b8b648b5abbe912a63f0bfcd447f484ff19dd321701d1ecda43c38
+size 1541274280

model-00152-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3f0f763e7c625c47b60cb286636752f57b0e93498950ce3cb11da5af2a0278d6
+size 1423889496

model-00157-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0a565c1892ce57f201d78b96a38fbd18622735c5d033b4e2a2ac5690e3fef627
+size 1423889464

model-00167-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e18a24ad9f2d1e426b18bd574aff5812b53bbbe0a34651e8ab526641ffacda6b
+size 1423293320

model-00173-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fb860c2b055dd5280a6018e808c7f1050444adba272004ce3751f2bfd4beac30
+size 1423889880

model-00177-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ad0906d7f1f9a8a77e3d12f53fb476bfade24fe979e9fb9f1344843b2d9e91e8
+size 1423889832

model-00180-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2776d290fde0bd44c74abddfd67dd8885d68abfb0424a1c6c8be1ad2ad769f3f
+size 1423889888

model-00183-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fd302bf7262d7c73e2ef763478231ca2ecc123b20fef6ef9846d126df6ee1710
+size 1423889896

model-00184-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:56432a37d830fc6da2504c50d60f4d7f85fab84d4745c42e1383b0884d9cc081
+size 1423889888

model-00187-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9fccf1341b6610f1a126c290822ed25e2e287befdc9e9bd005c73a97fe1b7b4d
+size 1423889896

model-00188-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5a143d04ce51c18ed6795c0ac5d98cff2771619330031977229379faa3468996
+size 1423889864

model-00191-of-00282.safetensors ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ec74f8781487f9fb795cbc1dc9d6070b89c008f1f19cef1c0fe6088917fb42ca
+size 1423889888