Instructions to use Kwaipilot/KAT-Dev-72B-Exp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Kwaipilot/KAT-Dev-72B-Exp with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="Kwaipilot/KAT-Dev-72B-Exp") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("Kwaipilot/KAT-Dev-72B-Exp") model = AutoModelForCausalLM.from_pretrained("Kwaipilot/KAT-Dev-72B-Exp", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Inference
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use Kwaipilot/KAT-Dev-72B-Exp with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "Kwaipilot/KAT-Dev-72B-Exp" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kwaipilot/KAT-Dev-72B-Exp", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/Kwaipilot/KAT-Dev-72B-Exp
- SGLang
How to use Kwaipilot/KAT-Dev-72B-Exp with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "Kwaipilot/KAT-Dev-72B-Exp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kwaipilot/KAT-Dev-72B-Exp", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "Kwaipilot/KAT-Dev-72B-Exp" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "Kwaipilot/KAT-Dev-72B-Exp", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use Kwaipilot/KAT-Dev-72B-Exp with Docker Model Runner:
docker model run hf.co/Kwaipilot/KAT-Dev-72B-Exp
| agent: | |
| model: | |
| api_base: http://127.0.0.1:61467/v1 | |
| api_key: sk-vllm-dummy-key | |
| max_input_tokens: 85000 | |
| name: hosted_vllm/Qwen3-1.7B | |
| per_instance_call_limit: 150 | |
| per_instance_cost_limit: 0 | |
| total_cost_limit: 0 | |
| temperature: 0.6 | |
| templates: | |
| system_template: |- | |
| You are a helpful assistant that can interact with a computer to solve tasks. | |
| <IMPORTANT> | |
| * If user provides a path, you should NOT assume it's relative to the current working directory. Instead, you should explore the file system to find the file before working on it. | |
| </IMPORTANT> | |
| # Tools | |
| You may call one or more functions to assist with the user query. | |
| You are provided with function signatures within <tools></tools> XML tags. | |
| For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags. | |
| You have access to the following functions: | |
| <tools> | |
| {"name": "bash", "description": "runs the given command directly in bash", "parameters": {"type": "object", "properties": {"command": {"type": "string", "description": "The bash command to execute."}}, "required": ["command"]}} | |
| {"name": "str_replace_editor", "description": "Custom editing tool for viewing, creating and editing files * State is persistent across command calls and discussions with the user * If `path` is a file, `view` displays the result of applying `cat -n`. If `path` is a directory, `view` lists non-hidden files and directories up to 2 levels deep * The `create` command cannot be used if the specified `path` already exists as a file * If a `command` generates a long output, it will be truncated and marked with `<response clipped>` * The `undo_edit` command will revert the last edit made to the file at `path`\nNotes for using the `str_replace` command: * The `old_str` parameter should match EXACTLY one or more consecutive lines from the original file. Be mindful of whitespaces! * If the `old_str` parameter is not unique in the file, the replacement will not be performed. Make sure to include enough context in `old_str` to make it unique * The `new_str` parameter should contain the edited lines that should replace the `old_str`\n", "parameters": {"type": "object", "properties": {"command": {"type": "string", "description": "The commands to run. Allowed options are: `view`, `create`, `str_replace`, `insert`, `undo_edit`.", "enum": ["view", "create", "str_replace", "insert", "undo_edit"]}, "path": {"type": "string", "description": "Absolute path to file or directory, e.g. `/testbed/file.py` or `/testbed`."}, "file_text": {"type": "string", "description": "Required parameter of `create` command, with the content of the file to be created."}, "old_str": {"type": "string", "description": "Required parameter of `str_replace` command containing the string in `path` to replace."}, "new_str": {"type": "string", "description": "Optional parameter of `str_replace` command containing the new string (if not given, no string will be added). Required parameter of `insert` command containing the string to insert."}, "insert_line": {"type": "integer", "description": "Required parameter of `insert` command. The `new_str` will be inserted AFTER the line `insert_line` of `path`."}, "view_range": {"type": "array", "description": "Optional parameter of `view` command when `path` points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. [11, 12] will show lines 11 and 12. Indexing at 1 to start. Setting `[start_line, -1]` shows all lines from `start_line` to the end of the file.", "items": {"type": "integer"}}}, "required": ["command", "path"]}} | |
| {"name": "submit", "description": "submits the current file", "parameters": {"type": "object", "properties": {}, "required": []}} | |
| </tools> | |
| If you choose to call a function ONLY reply in the following format with NO suffix: | |
| <tool_call> | |
| <function=example_function_name> | |
| <parameter=example_parameter_1> | |
| value_1 | |
| </parameter> | |
| <parameter=example_parameter_2> | |
| This is the value for the second parameter | |
| that can span | |
| multiple lines | |
| </parameter> | |
| </function> | |
| </tool_call> | |
| <IMPORTANT> | |
| Reminder: | |
| - Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags | |
| - Required parameters MUST be specified | |
| - You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after | |
| - If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls | |
| - Only call one function at a time | |
| - Always provide reasoning for your function call in natural language BEFORE the function call (not after) | |
| </IMPORTANT> | |
| instance_template: |- | |
| <uploaded_files> | |
| {{working_dir}} | |
| </uploaded_files> | |
| I've uploaded a python code repository in the directory {{working_dir}}. Consider the following PR description: | |
| <pr_description> | |
| {{problem_statement}} | |
| </pr_description> | |
| Can you help me implement the necessary changes to the repository so that the requirements specified in the <pr_description> are met? | |
| I've already taken care of all changes to any of the test files described in the <pr_description>. This means you DON'T have to modify the testing logic or any of the tests in any way! | |
| Your task is to make the minimal changes to non-tests files in the {{working_dir}} directory to ensure the <pr_description> is satisfied. | |
| Follow these steps to resolve the issue: | |
| 1. As a first step, it might be a good idea to find and read code relevant to the <pr_description> | |
| 2. Create a script to reproduce the error and execute it with `python <filename.py>` using the bash tool, to confirm the error | |
| 3. Edit the source code of the repo to resolve the issue | |
| 4. Rerun your reproduce script and confirm that the error is fixed! | |
| 5. Think about edgecases and make sure your fix handles them as well | |
| Your thinking should be thorough and so it's fine if it's very long. | |
| next_step_template: |- | |
| <tool_response> | |
| {{observation}} | |
| </tool_response> | |
| next_step_no_output_template: |- | |
| Your command ran successfully and did not produce any output. | |
| max_observation_length: 85000 | |
| tools: | |
| bundles: | |
| - path: tools/registry | |
| - path: tools/edit_anthropic | |
| - path: tools/submit | |
| env_variables: | |
| USE_FILEMAP: 'true' | |
| enable_bash_tool: true | |
| parse_function: | |
| type: xml_function_calling | |
| str_replace_editor: | |
| arguments: | |
| - name: view_range | |
| argument_format: "--view_range {{value}}" | |
| execution_timeout: 400 | |
| history_processors: | |
| - type: last_n_observations | |
| n: 100 | |