thek0tyara/llama-cpp-turboquant
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions 4

model : mtmd : make input norm optional in LFM2-VL (#18594)

Browse source 
Upcoming LFM2-VL releases will have configurable input norm.
See https://github.com/huggingface/transformers/pull/43087 for details.
This commit is contained in:
Tarek Dakhran 2026-01-04 18:50:02 +01:00 • committed by GitHub
parent 908a9e5a1e
commit 4974bf53cf
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
2 changed files with 17 additions and 4 deletions
Download patch file Download diff file Expand all files Collapse all files

13
tools/mtmd/models/siglip.cpp
View file

@ -50,10 +50,15 @@ ggml_cgraph * clip_graph_siglip::build() {
const int scale_factor = model.hparams.n_merge;
cur = build_patch_merge_permute(cur, scale_factor);
// projection
cur = ggml_norm(ctx0, cur, 1e-5); // default nn.LayerNorm
cur = ggml_mul(ctx0, cur, model.mm_input_norm_w);
cur = ggml_add(ctx0, cur, model.mm_input_norm_b);
// projection, in LFM2-VL input norm is optional
if (model.mm_input_norm_w) {
cur = ggml_norm(ctx0, cur, 1e-5); // default nn.LayerNorm
cur = ggml_mul(ctx0, cur, model.mm_input_norm_w);
}
if (model.mm_input_norm_b) {
cur = ggml_add(ctx0, cur, model.mm_input_norm_b);
}
cur = build_ffn(cur,
model.mm_1_w, model.mm_1_b,