1920345c3b
* Add files via upload * fix unit test * fix crashes for --reasoning-format=none * Patch buggy official MiniMax-M2 chat template * add upstream minja fix: https://github.com/ochafik/minja/pull/7 * Fix <think> token not generated * add test copied from https://github.com/ggml-org/llama.cpp/pull/16946 * cleanup * Hopes to fix the compilation error on CI * Delete chat template patching since it’s fixed by upstream Minja * Remove undeeded Minimax-M2 template patch https://github.com/ochafik/minja/pull/7#issuecomment-3480356100 * Add proper handling of optional parameters with test merged tests from: https://github.com/ggml-org/llama.cpp/pull/16946/commits/23d4bb75c485c12ac89f81c424dc03c87a640e8c * Fix making all tool parameters optional * Move xml tool parser to separate file * cleanup & add tests for GLM4.5 * add streaming tests & enhancement & cleanups Add streaming test for both GLM 4.5 and minimax-m2. Cleanup for preserved_tokens. Cleanup for grammar rule name. Enhance the parser's stability. * cleanup & add support for Kimi-K2 Qwen3-Coder Apriel-1.5 Xiaomi-MiMo * apply suggestions from reviewers * fix a misuse for data.grammar_lazy * fix grammar when tool have no argument * Fix `no triggers set for lazy grammar!` for GLM4.5/4.6. Insert additional stops for Kimi-K2 * update chat.cpp * fix grammar for GLM 4.5/4.6 * Try fix Jinja template for GLM * Try fix GLM-4.6.jinja * Update common/chat-parser-xml-toolcall.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update tests/test-chat.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * improve chat template for GLM, rename Kimi-K2 template to Kimi-K2-Thinking * Improve Kimi-K2 chat template * Fix unit test * Fix "Invalid tool call arguments passed" in a rare case. In a rare case, the model may emit a raw string that begins with a valid JSON string. This commit adds unit tests to cover that scenario and fixes the regression introduced during the Kimi-K2 adaptation. --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> |
||
|---|---|---|
| .. | ||
| Apertus-8B-Instruct.jinja | ||
| ByteDance-Seed-OSS.jinja | ||
| CohereForAI-c4ai-command-r-plus-tool_use.jinja | ||
| CohereForAI-c4ai-command-r7b-12-2024-tool_use.jinja | ||
| deepseek-ai-DeepSeek-R1-Distill-Llama-8B.jinja | ||
| deepseek-ai-DeepSeek-R1-Distill-Qwen-32B.jinja | ||
| deepseek-ai-DeepSeek-V3.1.jinja | ||
| fireworks-ai-llama-3-firefunction-v2.jinja | ||
| GLM-4.6.jinja | ||
| google-gemma-2-2b-it.jinja | ||
| ibm-granite-granite-3.3-2B-Instruct.jinja | ||
| Kimi-K2-Instruct.jinja | ||
| Kimi-K2-Thinking.jinja | ||
| llama-cpp-deepseek-r1.jinja | ||
| llama-cpp-lfm2.jinja | ||
| llama-cpp-rwkv-world.jinja | ||
| meetkai-functionary-medium-v3.1.jinja | ||
| meetkai-functionary-medium-v3.2.jinja | ||
| meta-llama-Llama-3.1-8B-Instruct.jinja | ||
| meta-llama-Llama-3.2-3B-Instruct.jinja | ||
| meta-llama-Llama-3.3-70B-Instruct.jinja | ||
| microsoft-Phi-3.5-mini-instruct.jinja | ||
| MiMo-VL.jinja | ||
| MiniMax-M2.jinja | ||
| Mistral-Small-3.2-24B-Instruct-2506.jinja | ||
| mistralai-Mistral-Nemo-Instruct-2407.jinja | ||
| moonshotai-Kimi-K2.jinja | ||
| NousResearch-Hermes-2-Pro-Llama-3-8B-tool_use.jinja | ||
| NousResearch-Hermes-3-Llama-3.1-8B-tool_use.jinja | ||
| NVIDIA-Nemotron-Nano-v2.jinja | ||
| openai-gpt-oss-120b.jinja | ||
| Qwen-Qwen2.5-7B-Instruct.jinja | ||
| Qwen-Qwen3-0.6B.jinja | ||
| Qwen-QwQ-32B.jinja | ||
| Qwen3-Coder.jinja | ||
| README.md | ||
| unsloth-Apriel-1.5.jinja | ||
| unsloth-mistral-Devstral-Small-2507.jinja | ||
These templates can be updated with the following commands:
./scripts/get_chat_template.py CohereForAI/c4ai-command-r-plus tool_use > models/templates/CohereForAI-c4ai-command-r-plus-tool_use.jinja
./scripts/get_chat_template.py CohereForAI/c4ai-command-r7b-12-2024 default > models/templates/CohereForAI-c4ai-command-r7b-12-2024-default.jinja
./scripts/get_chat_template.py CohereForAI/c4ai-command-r7b-12-2024 rag > models/templates/CohereForAI-c4ai-command-r7b-12-2024-rag.jinja
./scripts/get_chat_template.py CohereForAI/c4ai-command-r7b-12-2024 tool_use > models/templates/CohereForAI-c4ai-command-r7b-12-2024-tool_use.jinja
./scripts/get_chat_template.py deepseek-ai/DeepSeek-R1-Distill-Llama-8B > models/templates/deepseek-ai-DeepSeek-R1-Distill-Llama-8B.jinja
./scripts/get_chat_template.py deepseek-ai/DeepSeek-R1-Distill-Qwen-32B > models/templates/deepseek-ai-DeepSeek-R1-Distill-Qwen-32B.jinja
./scripts/get_chat_template.py fireworks-ai/llama-3-firefunction-v2 > models/templates/fireworks-ai-llama-3-firefunction-v2.jinja
./scripts/get_chat_template.py google/gemma-2-2b-it > models/templates/google-gemma-2-2b-it.jinja
./scripts/get_chat_template.py meetkai/functionary-medium-v3.1 > models/templates/meetkai-functionary-medium-v3.1.jinja
./scripts/get_chat_template.py meetkai/functionary-medium-v3.2 > models/templates/meetkai-functionary-medium-v3.2.jinja
./scripts/get_chat_template.py meta-llama/Llama-3.1-8B-Instruct > models/templates/meta-llama-Llama-3.1-8B-Instruct.jinja
./scripts/get_chat_template.py meta-llama/Llama-3.2-3B-Instruct > models/templates/meta-llama-Llama-3.2-3B-Instruct.jinja
./scripts/get_chat_template.py meta-llama/Llama-3.3-70B-Instruct > models/templates/meta-llama-Llama-3.3-70B-Instruct.jinja
./scripts/get_chat_template.py microsoft/Phi-3.5-mini-instruct > models/templates/microsoft-Phi-3.5-mini-instruct.jinja
./scripts/get_chat_template.py mistralai/Mistral-Nemo-Instruct-2407 > models/templates/mistralai-Mistral-Nemo-Instruct-2407.jinja
./scripts/get_chat_template.py NousResearch/Hermes-2-Pro-Llama-3-8B tool_use > models/templates/NousResearch-Hermes-2-Pro-Llama-3-8B-tool_use.jinja
./scripts/get_chat_template.py NousResearch/Hermes-3-Llama-3.1-8B tool_use > models/templates/NousResearch-Hermes-3-Llama-3.1-8B-tool_use.jinja
./scripts/get_chat_template.py Qwen/Qwen2.5-7B-Instruct > models/templates/Qwen-Qwen2.5-7B-Instruct.jinja
./scripts/get_chat_template.py Qwen/QwQ-32B > models/templates/Qwen-QwQ-32B.jinja
./scripts/get_chat_template.py Qwen/Qwen3-0.6B > models/templates/Qwen-Qwen3-0.6B.jinja
./scripts/get_chat_template.py zai-org/GLM-4.5 > models/templates/zai-org-GLM-4.5.jinja
./scripts/get_chat_template.py deepseek-ai/DeepSeek-V3.1 > models/templates/deepseek-ai-DeepSeek-V3.1.jinja