+ mkdir /usr/src/.npm-global
+ npm config set prefix /usr/src/.npm-global
+ npm install -g @aikidosec/safe-chain
added 57 packages in 6s
5 packages are looking for funding
run `npm fund` for details
+ PATH=/usr/src/.npm-global/bin:/usr/bin:/bin:/usr/local/bin
+ cd /usr/src/build
+ rm tools/server/public/index.html.gz
+ cd tools/server/webui
+ aikido-npm ci --ignore-scripts
added 547 packages, and audited 548 packages in 26s
218 packages are looking for funding
run `npm fund` for details
7 vulnerabilities (1 low, 1 moderate, 5 high)
To address all issues, run:
npm audit fix
Run `npm audit` for details.
+ aikido-npm audit --audit-level=critical fix
added 12 packages, removed 1 package, changed 8 packages, and audited 559 packages in 27s
227 packages are looking for funding
run `npm fund` for details
# npm audit report
cookie <0.7.0
cookie accepts cookie name, path, and domain with out of bounds characters - https://github.com/advisories/GHSA-pxg6-pf52-xh8x
fix available via `npm audit fix --force`
Will install @sveltejs/kit@0.0.30, which is a breaking change
node_modules/cookie
@sveltejs/kit >=1.0.0-next.0
Depends on vulnerable versions of cookie
node_modules/@sveltejs/kit
@sveltejs/adapter-static >=1.0.0-next.0
Depends on vulnerable versions of @sveltejs/kit
node_modules/@sveltejs/adapter-static
runed >=0.32.0
Depends on vulnerable versions of @sveltejs/kit
node_modules/bits-ui/node_modules/runed
bits-ui >=2.11.8
Depends on vulnerable versions of runed
Depends on vulnerable versions of svelte-toolbelt
node_modules/bits-ui
svelte-toolbelt >=0.10.6
Depends on vulnerable versions of runed
node_modules/bits-ui/node_modules/svelte-toolbelt
6 low severity vulnerabilities
To address issues that do not require attention, run:
npm audit fix
To address all issues (including breaking changes), run:
npm audit fix --force
ℹ Safe-chain: Some package versions were suppressed due to minimum age requirement.
To disable this check, use: --safe-chain-skip-minimum-package-age
+ du -sh node_modules
407M node_modules
+ npm run build
> webui@1.0.0 build
> vite build && ./scripts/post-build.sh
▲ [WARNING] Cannot find base config file "./.svelte-kit/tsconfig.json" [tsconfig.json]
tsconfig.json:2:12:
2 │ "extends": "./.svelte-kit/tsconfig.json",
╵ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
│
▲ Failed to load preset:
│ "file:/usr/src/build/tools/server/webui/node_modules/@storybook/builder-vite/dist/preset.js"
│
■ Error [ERR_MODULE_NOT_FOUND]: Cannot find module
│ './node_modules/@storybook/builder-vite/dist/preset.js' imported from
│ ./node_modules/storybook/dist/_node-chunks/chunk-OXJURHEN.js
│ at finalizeResolution (node:internal/modules/esm/resolve:274:11)
│ at moduleResolve (node:internal/modules/esm/resolve:859:10)
│ at defaultResolve (node:internal/modules/esm/resolve:983:11)
│ at nextResolve (node:internal/modules/esm/hooks:748:28)
│ at o
│ (file://./node_modules/@tailwindcss/node/dist/esm-cache.loader.mjs:1:69)
│ at nextResolve (node:internal/modules/esm/hooks:748:28)
│ at Hooks.resolve (node:internal/modules/esm/hooks:240:30)
│ at handleMessage (node:internal/modules/esm/worker:199:24)
│ at Immediate.checkForMessages
│ (node:internal/modules/esm/worker:141:28)
│ at process.processImmediate (node:internal/timers:485:21)
vite v7.2.2 building ssr environment for production...
transforming...
DEPRECATION WARNING [import]: Sass @import rules are deprecated and will be removed in Dart Sass 3.0.0.
More info and automated migrator: https://sass-lang.com/d/import
╷
17 │ @import 'katex/src/styles/katex.scss';
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
╵
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [import]: Sass @import rules are deprecated and will be removed in Dart Sass 3.0.0.
More info and automated migrator: https://sass-lang.com/d/import
╷
2 │ @import "./fonts.scss";
│ ^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 2:9 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.append instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
9 │ $src: append($src, url('#{$font-folder}/KaTeX_#{$family}-#{$family-suffix}.woff2') format('woff2'), comma);
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/fonts.scss 9:15 generate-src()
node_modules/katex/src/styles/fonts.scss 42:11 font-face()
node_modules/katex/src/styles/fonts.scss 52:1 @import
node_modules/katex/src/styles/katex.scss 2:9 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.length instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
344 │ @for $from from 1 through length($sizes) {
│ ^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 344:35 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.length instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
345 │ @for $to from 1 through length($sizes) {
│ ^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 345:37 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.nth instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
348 │ font-size: calc((nth($sizes, $to) / nth($sizes, $from)) * 1em);
│ ^^^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 348:38 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.nth instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
348 │ font-size: calc((nth($sizes, $to) / nth($sizes, $from)) * 1em);
│ ^^^^^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 348:57 @import
src/styles/katex-custom.scss 17:9 root stylesheet
node_modules/@sveltejs/kit/src/runtime/client/client.js (333:26): "fork" is not exported by "node_modules/svelte/src/index-server.js", imported by "node_modules/@sveltejs/kit/src/runtime/client/client.js".
"default" is imported from external module "highlight.js" but never used in "src/lib/components/app/misc/SyntaxHighlightedCode.svelte".
✓ 4621 modules transformed.
Export "getJsonHeaders" of module "src/lib/utils/api-headers.ts" was reexported through module "src/lib/utils/index.ts" while both modules are dependencies of each other and will end up in different chunks by current Rollup settings. This scenario is not well supported at the moment as it will produce a circular dependency between chunks and will likely lead to broken execution order.
Either change the import in "src/lib/services/models.ts" to point directly to the exporting module or reconfigure "output.manualChunks" to ensure these modules end up in the same chunk.
Export "getJsonHeaders" of module "src/lib/utils/api-headers.ts" was reexported through module "src/lib/utils/index.ts" while both modules are dependencies of each other and will end up in different chunks by current Rollup settings. This scenario is not well supported at the moment as it will produce a circular dependency between chunks and will likely lead to broken execution order.
Either change the import in "src/lib/services/chat.ts" to point directly to the exporting module or reconfigure "output.manualChunks" to ensure these modules end up in the same chunk.
rendering chunks...
│
▲ Failed to load preset:
│ "file:/usr/src/build/tools/server/webui/node_modules/@storybook/builder-vite/dist/preset.js"
│
■ Error [ERR_MODULE_NOT_FOUND]: Cannot find module
│ './node_modules/@storybook/builder-vite/dist/preset.js' imported from
│ ./node_modules/storybook/dist/_node-chunks/chunk-OXJURHEN.js
│ at finalizeResolution (node:internal/modules/esm/resolve:274:11)
│ at moduleResolve (node:internal/modules/esm/resolve:859:10)
│ at defaultResolve (node:internal/modules/esm/resolve:983:11)
│ at nextResolve (node:internal/modules/esm/hooks:748:28)
│ at o
│ (file://./node_modules/@tailwindcss/node/dist/esm-cache.loader.mjs:1:69)
│ at nextResolve (node:internal/modules/esm/hooks:748:28)
│ at Hooks.resolve (node:internal/modules/esm/hooks:240:30)
│ at handleMessage (node:internal/modules/esm/worker:199:24)
│ at Immediate.checkForMessages
│ (node:internal/modules/esm/worker:141:28)
│ at process.processImmediate (node:internal/timers:485:21)
vite v7.2.2 building client environment for production...
transforming...
DEPRECATION WARNING [import]: Sass @import rules are deprecated and will be removed in Dart Sass 3.0.0.
More info and automated migrator: https://sass-lang.com/d/import
╷
17 │ @import 'katex/src/styles/katex.scss';
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
╵
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [import]: Sass @import rules are deprecated and will be removed in Dart Sass 3.0.0.
More info and automated migrator: https://sass-lang.com/d/import
╷
2 │ @import "./fonts.scss";
│ ^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 2:9 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.append instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
9 │ $src: append($src, url('#{$font-folder}/KaTeX_#{$family}-#{$family-suffix}.woff2') format('woff2'), comma);
│ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/fonts.scss 9:15 generate-src()
node_modules/katex/src/styles/fonts.scss 42:11 font-face()
node_modules/katex/src/styles/fonts.scss 52:1 @import
node_modules/katex/src/styles/katex.scss 2:9 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.length instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
344 │ @for $from from 1 through length($sizes) {
│ ^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 344:35 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.length instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
345 │ @for $to from 1 through length($sizes) {
│ ^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 345:37 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.nth instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
348 │ font-size: calc((nth($sizes, $to) / nth($sizes, $from)) * 1em);
│ ^^^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 348:38 @import
src/styles/katex-custom.scss 17:9 root stylesheet
DEPRECATION WARNING [global-builtin]: Global built-in functions are deprecated and will be removed in Dart Sass 3.0.0.
Use list.nth instead.
More info and automated migrator: https://sass-lang.com/d/import
╷
348 │ font-size: calc((nth($sizes, $to) / nth($sizes, $from)) * 1em);
│ ^^^^^^^^^^^^^^^^^^
╵
node_modules/katex/src/styles/katex.scss 348:57 @import
src/styles/katex-custom.scss 17:9 root stylesheet
node_modules/@sveltejs/kit/src/runtime/client/client.js (332:26): "fork" is not exported by "node_modules/svelte/src/index-client.js", imported by "node_modules/@sveltejs/kit/src/runtime/client/client.js".
✓ 5356 modules transformed.
rendering chunks...
computing gzip size...
.svelte-kit/output/client/_app/version.json 0.03 kB │ gzip: 0.05 kB
.svelte-kit/output/client/.vite/manifest.json 0.33 kB │ gzip: 0.19 kB
.svelte-kit/output/client/_app/immutable/assets/style.sWIT1xAp.css 482.64 kB │ gzip: 286.19 kB
.svelte-kit/output/client/_app/immutable/bundle.DxpKjihs.js 3,777.47 kB │ gzip: 1,118.20 kB
(!) Some chunks are larger than 3072 kB after minification. Consider:
- Using dynamic import() to code-split the application
- Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
- Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
✓ built in 11.56s
.svelte-kit/output/server/.vite/manifest.json 5.75 kB
.svelte-kit/output/server/_app/immutable/assets/style.BvrYRhgP.css 482.35 kB
.svelte-kit/output/server/chunks/environment.js 0.07 kB
.svelte-kit/output/server/chunks/api-key-validation.js 0.14 kB
.svelte-kit/output/server/entries/pages/_page.ts.js 0.17 kB
.svelte-kit/output/server/entries/pages/chat/_id_/_page.ts.js 0.18 kB
.svelte-kit/output/server/chunks/server.js 0.22 kB
.svelte-kit/output/server/chunks/browser-only.js 0.31 kB
.svelte-kit/output/server/internal.js 0.35 kB
.svelte-kit/output/server/chunks/utils.js 0.63 kB
.svelte-kit/output/server/entries/pages/_page.svelte.js 0.98 kB
.svelte-kit/output/server/entries/pages/chat/_id_/_page.svelte.js 1.00 kB
.svelte-kit/output/server/chunks/label.js 2.21 kB
.svelte-kit/output/server/chunks/exports.js 3.06 kB
.svelte-kit/output/server/chunks/supported-file-types.js 4.32 kB
.svelte-kit/output/server/chunks/internal.js 6.02 kB
.svelte-kit/output/server/chunks/shared.js 7.48 kB
.svelte-kit/output/server/remote-entry.js 8.29 kB
.svelte-kit/output/server/entries/pages/_error.svelte.js 8.43 kB
.svelte-kit/output/server/chunks/index.js 21.41 kB
.svelte-kit/output/server/entries/pages/_layout.svelte.js 34.19 kB
.svelte-kit/output/server/index.js 56.23 kB
.svelte-kit/output/server/chunks/SyntaxHighlightedCode.svelte_svelte_type_style_lang.js 86.69 kB
.svelte-kit/output/server/chunks/DialogConfirmation.js 109.90 kB
.svelte-kit/output/server/chunks/ServerLoadingSplash.js 217.27 kB
✓ built in 21.04s
Run npm run preview to preview your production build locally.
> Using @sveltejs/adapter-static
Overwriting ../public/index.html with fallback page. Consider using a different name for the fallback.
Wrote site to "../public"
✔ done
✓ Inlined favicon.svg as base64 data URL
✓ Created index.html.gz
* graph : avoid branches between embedding and token inputs
* models : make deepstack graphs (e.g. Qwen3 VL) have constant topology
* ci : enable -DGGML_SCHED_NO_REALLOC=ON for server CI
* cont : pad token embeddings to n_embd_inp
This commit modifies all the utility scripts to use an optional
BUILD_DIR variable/argument to specify the build directory.
The motivation for this is that Commit
3d55846a5c ("model-conversion : add
BUILD_DIR variable to run-converted-model scripts") introduced this
variable to the causal and embeddings scripts, but I missed the scripts
in the utils directory.
* Boilerplate for q5_Kx8 REPACK on ARM and fallback
Signed-off-by: Alberto Cabrera <alberto.cabrera@liquid.ai>
* Implements make_block_q5_Kx8 by extending make_block_q4_Kx8
Signed-off-by: Alberto Cabrera <alberto.cabrera@liquid.ai>
* q5_K repack gemm and gemv generics
* Gemm and Gemv ARM implementations (i8mm)
* Improved qh manipulation looking at non-repack vec_dot implementation
* Full unroll
* Apply Q5_K Gemv vand and vshl optimizations to gemm. Improve comments.
Signed-off-by: Alberto Cabrera <alberto.cabrera@liquid.ai>
* Fix wrong fallback definitions of Q5_K
Signed-off-by: Alberto Cabrera <alberto.cabrera@liquid.ai>
* Fixed comments. Reverted unnecessary formatting
Signed-off-by: Alberto Cabrera <alberto.cabrera@liquid.ai>
* Fixed typo in generic definitions
* Switching AND + Shift with Shift Insert. Better op interleaving.
* Vectorize + unroll the block scales
* Apply gemm optimizations to gemv
* Improve bias calculation
---------
Signed-off-by: Alberto Cabrera <alberto.cabrera@liquid.ai>
* mla : pass V as a view of K to the FA op
* cuda : adjust mla logic to new layout
* kv-cache : fix rope shift
* tests : remove comment
* cuda : fix reusable_cutoff
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
---------
Co-authored-by: Johannes Gäßler <johannesg@5d6.de>
* opencl: add `copy_to_contiguous` and utilize mm kernels
* opencl: only copy to cont for f32 and f16 tensors
* opencl: use cont mm for fallback when dst is large
* opencl: use nb local to copy-to-cont
* opencl: use local offset as well
* Move `task_result_state::update_chat_msg` to match with header
* Move `server_task_result_cmpl_partial::to_json_anthropic()` to match with header
---------
Co-authored-by: openingnow <>
* Add Ministral3ForCausalLM architeture
This adds support for newer architectres like Devstral-2
* removed blank line found after function decorator
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
---------
Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com>
* vulkan: Remove transfer_ctx, do everything in compute_ctx.
We had a bug where a set_tensor_async (using transfer_ctx) didn't get
submitted before the graph_compute (using compute_ctx) that came after
it. To avoid this sort of issue, just do everything in compute_ctx.
Remove transfer_cmd_pool, which was already unused.
* fix crash with perf logger
* from previous PR
* Make instruction(system) as first message
* Convert [input_message] (text/image/file)
* Rename convert_responses_to_chatcmpl(body) -> response_body
* Initial tool call support
* Erase instructions field from chatcmpl body
* Feed reasoning texts to chat template
* Use std::vector instead of opaque json array
* Make output_item.added events consistent
* Move `server_task_result_cmpl_partial::update` from header to source
* Match ID of output_item.added and .done events
* Add function_call only if there is no "fc_" prefix
* Add function call output at non-streaming API
* Test if ID is persistent
* Add doc
* Fix style - use trailing comma
* Rewrite state management
* catch up with upstream/master
* Fix style - "type" is the first item of SSE data
* Explicitly check "instructions" from response_body
* Make lambdas static
* Check if reasoning content exists
* Add `oai_resp_id` to task_result_state(also initialized at ctor), server_task_result_cmpl_partial, and server_task_result_cmpl_final
* Reject `input_file` since it is not supported by chatcmpl
* Add "fc_" prefix to non-straming function call id as coderabbit pointed out
---------
Co-authored-by: openingnow <>
Change ggml_vk_mul_mat_vec_id_q_f16 to loop over the batch dimension and
update the indexing calculations in get_offsets.
Mat-vec is faster than mat-mat for small values of n. We don't get the same
reuse of the weights as in the non-ID path, but with this the cost is linear
in n rather than n>1 being far slower than n==1.
I've had issues loading models with llama-server:
[44039] E gguf_init_from_file: failed to open GGUF file 'mistral-7b-v0.1.Q8_0.gguf'
and I was sure it could access the file. Seems like --models-dir and
--models-presets dont interact like I thought they would but I salvaged
this snippet that helps troubleshooting
[44039] E gguf_init_from_file: failed to open GGUF file 'mistral-7b-v0.1.Q8_0.gguf' (errno No such file or directory)
* CUDA: Replace `init_offsets` with iterators in argsort
This is a QOL improvement, saving us the cost of materializing the
iterator
* Remove unnecessary include from top-k.cu
* convert : use n_groups instead of hardcoded values in reshape
This commit modifies the conversion script for NemotronHModel to use
the 'n_groups' hyperparameter, and allow Python to calculate the the
last dimension, using -1, when reshaping the 'mixer.norm.weight' tensor.
* use self.n_group instead of self.hparams["n_groups"]
This commit adds a BUILD_DIR variable to the scripts used for running
converted models.
The motivation for this is that currently the `build` directory is
hardcoded and it can be useful to specify a different build directory,
with builds for different configurations.
