thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

1:3072-alt1.20240603

Browse source 
- Update to b3072 (2024-06-03).
- The version scheme now matches the upstream build number more closely,
  instead of using the commit date.
- Build with libcurl and OpenBLAS support.
This commit is contained in:
Vitaly Chikunov 2024-06-03 08:53:03 +03:00
parent 6070579270
commit 956398b7fe
1 changed files with 14 additions and 6 deletions
Download patch file Download diff file Expand all files Collapse all files

20
.gear/llama.cpp.spec
View file

@ -4,9 +4,10 @@
%set_verify_elf_method strict %set_verify_elf_method strict
Name: llama.cpp Name: llama.cpp
Version: 20240527 Version: 3072
Release: alt1 Release: alt1.20240603
Summary: Inference of LLaMA model in pure C/C++ Epoch: 1
Summary: LLM inference in C/C++
License: MIT License: MIT
Group: Sciences/Computer science Group: Sciences/Computer science
Url: https://github.com/ggerganov/llama.cpp Url: https://github.com/ggerganov/llama.cpp
@ -34,7 +35,7 @@ BuildRequires: tinyllamas-gguf
}} }}
%description %description
Plain C/C++ implementation (of inference of LLaMA model) without Plain C/C++ implementation (of inference of many LLM models) without
dependencies. AVX, AVX2 and AVX512 support for x86 architectures. dependencies. AVX, AVX2 and AVX512 support for x86 architectures.
Mixed F16/F32 precision. 1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and Mixed F16/F32 precision. 1.5-bit, 2-bit, 3-bit, 4-bit, 5-bit, 6-bit, and
8-bit integer quantization for faster inference and reduced memory use. 8-bit integer quantization for faster inference and reduced memory use.
@ -47,12 +48,13 @@ Supported models:
Baichuan 1 & 2 + derivations, Aquila 1 & 2, Starcoder models, Refact, Baichuan 1 & 2 + derivations, Aquila 1 & 2, Starcoder models, Refact,
Persimmon 8B, MPT, Bloom, Yi models, StableLM models, Deepseek models, Persimmon 8B, MPT, Bloom, Yi models, StableLM models, Deepseek models,
Qwen models, PLaMo-13B, Phi models, GPT-2, Orion 14B, InternLM2, Qwen models, PLaMo-13B, Phi models, GPT-2, Orion 14B, InternLM2,
CodeShell, Gemma CodeShell, Gemma, Mamba, Grok-1, Xverse, Command-R models, SEA-LION,
GritLM-7B + GritLM-8x7B, OLMo, GPT-NeoX + Pythia
Multimodal models: Multimodal models:
LLaVA 1.5 models, BakLLaVA, Obsidian, ShareGPT4V, MobileVLM 1.7B/3B LLaVA 1.5 models, BakLLaVA, Obsidian, ShareGPT4V, MobileVLM 1.7B/3B
models, Yi-VL models, Yi-VL, Mini CPM, Moondream, Bunny
NOTE 1: You will need to: NOTE 1: You will need to:
@ -150,6 +152,12 @@ llama-main -m %_datadir/tinyllamas/stories260K.gguf -p "Once upon a time" -s 55
%_datadir/%name %_datadir/%name
%changelog %changelog
* Mon Jun 03 2024 Vitaly Chikunov <vt@altlinux.org> 1:3072-alt1.20240603
- Update to b3072 (2024-06-03).
- The version scheme now matches the upstream build number more closely,
instead of using the commit date.
- Build with libcurl and OpenBLAS support.
* Tue May 28 2024 Vitaly Chikunov <vt@altlinux.org> 20240527-alt1 * Tue May 28 2024 Vitaly Chikunov <vt@altlinux.org> 20240527-alt1
- Update to b3012 (2024-05-27). - Update to b3012 (2024-05-27).