mirror of
https://github.com/ggml-org/llama.cpp.git
synced 2026-11-11 06:12:03 +01:00
ModernBERT but without `head.norm` so will currently fail to convert and run any other ModernBERT models, PRs with `head.norm` support welcome! * constants and tensor mappings for modern bert support, model not supported yet but working on getting conversion to work for encoder only * conversion now working, hf -> gguf * working on support, now working on building graph * some cleanup * cleanup * continuing * correct tensor shape for qkv * fixed tensor mappings and working on buildin graph * tensor debugging now works -> (llama-eval-callback), instead of simulated gate split with views, GEGLU is now used which does exactly this * cleanup * cleanup * cleanup * more cleanup * ubatch issues, the assert for checking equal seqs in llama-graph.cpp when building attention keeps failing, setting ubatch size to 1 when running llama-embedding with --ubatch-size 1 makes it work, but needs to be looked into more * added cls token per previous modern bert attempt, still working on checking out the rest * fixed pre tokenizer and still working through previous pr * working through previous attemp, implimented more accurate conversion per previous attempt, added local sliding window attention that alternates every third layer * fixed pre tokenizer * working on swa with local and global alternating attention * some cleanup and now fails on build attn * starting to work, and some cleanup, currently failing on last layer construction in graph build * alternating rope implemented and modern bert graph build succeeds * fixed asser for equal ubatch seq * cleanup * added mask check in vocab * fixed alternating rope, the hparams.rope_freq_base_train and hparams.rope_freq_base_train_swa were the same and i set them to correct values * reuse variable * removed repeat * standard swa method can be used instead of a new enum being LLAMA_SWA_TYPE_LOCAL * correct swa layer indexing, is supposed to be 0, 3, 6 ... instead of 1, 4, 7 ... * more modular hparam setting * replaced attn out norm with ffn_norm and cosine similarity between hf embds and llama.cpp embds went way up, from 0.05 to 0.24, replaced the cacheless kv with swa todo per the previous conversion * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update convert_hf_to_gguf_update.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-vocab.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update convert_hf_to_gguf.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update convert_hf_to_gguf.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/tensor_mapping.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-graph.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-arch.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * removed redundant hparam set * enums for model sizes * conversion for modern-bert model supported rather than just granite-small * Update src/llama-model.cpp Co-authored-by: Gabe Goodhart <ghart@us.ibm.com> * Update src/llama-model.cpp Co-authored-by: Gabe Goodhart <ghart@us.ibm.com> * fixed ordering of enum for freq_base_swa * fixed where I added residual, now gives much much better embeddings~ * readded cacheless logic * removing whitespace * conversion now working for swa pattern - dense every n layers * modern bert put into seperate src file * removing whitespace * fixed whitespace and newline errors in editorconfig job * Update convert_hf_to_gguf.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * better naming convention, n_swa_pattern -> swa_period * reusing sliding_window_pattern key rather than making new dense_every_n_layers key, and adding writing and reading support * fixing pyright type-check fail * Update convert_hf_to_gguf.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/gguf_writer.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-hparams.h Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model-saver.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/models/modern-bert.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/models/modern-bert.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/models/modern-bert.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update gguf-py/gguf/gguf_writer.py Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/models/modern-bert.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/models/modern-bert.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model-loader.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model-loader.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model-loader.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * added descriptions in llama-model * fixed tensor mappings for conversion * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * Update src/llama-model.cpp Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> * mapping name for size * nits * unused --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@scala.com> Co-authored-by: Gabe Goodhart <ghart@us.ibm.com>
157 lines
4.6 KiB
CMake
157 lines
4.6 KiB
CMake
llama_add_compile_flags()
|
|
|
|
#
|
|
# libraries
|
|
#
|
|
|
|
# llama
|
|
|
|
add_library(llama
|
|
../include/llama.h
|
|
llama.cpp
|
|
llama-adapter.cpp
|
|
llama-arch.cpp
|
|
llama-batch.cpp
|
|
llama-chat.cpp
|
|
llama-context.cpp
|
|
llama-cparams.cpp
|
|
llama-grammar.cpp
|
|
llama-graph.cpp
|
|
llama-hparams.cpp
|
|
llama-impl.cpp
|
|
llama-io.cpp
|
|
llama-kv-cache.cpp
|
|
llama-kv-cache-iswa.cpp
|
|
llama-memory.cpp
|
|
llama-memory-hybrid.cpp
|
|
llama-memory-recurrent.cpp
|
|
llama-mmap.cpp
|
|
llama-model-loader.cpp
|
|
llama-model-saver.cpp
|
|
llama-model.cpp
|
|
llama-quant.cpp
|
|
llama-sampling.cpp
|
|
llama-vocab.cpp
|
|
unicode-data.cpp
|
|
unicode.cpp
|
|
unicode.h
|
|
models/afmoe.cpp
|
|
models/apertus.cpp
|
|
models/arcee.cpp
|
|
models/arctic.cpp
|
|
models/arwkv7.cpp
|
|
models/baichuan.cpp
|
|
models/bailingmoe.cpp
|
|
models/bailingmoe2.cpp
|
|
models/bert.cpp
|
|
models/bitnet.cpp
|
|
models/bloom.cpp
|
|
models/chameleon.cpp
|
|
models/chatglm.cpp
|
|
models/codeshell.cpp
|
|
models/cogvlm.cpp
|
|
models/cohere2-iswa.cpp
|
|
models/command-r.cpp
|
|
models/dbrx.cpp
|
|
models/deci.cpp
|
|
models/deepseek.cpp
|
|
models/deepseek2.cpp
|
|
models/dots1.cpp
|
|
models/dream.cpp
|
|
models/ernie4-5-moe.cpp
|
|
models/ernie4-5.cpp
|
|
models/exaone.cpp
|
|
models/exaone4.cpp
|
|
models/falcon-h1.cpp
|
|
models/falcon.cpp
|
|
models/gemma-embedding.cpp
|
|
models/gemma.cpp
|
|
models/gemma2-iswa.cpp
|
|
models/gemma3.cpp
|
|
models/gemma3n-iswa.cpp
|
|
models/glm4-moe.cpp
|
|
models/glm4.cpp
|
|
models/gpt2.cpp
|
|
models/gptneox.cpp
|
|
models/granite-hybrid.cpp
|
|
models/granite.cpp
|
|
models/grok.cpp
|
|
models/grovemoe.cpp
|
|
models/hunyuan-dense.cpp
|
|
models/hunyuan-moe.cpp
|
|
models/internlm2.cpp
|
|
models/jais.cpp
|
|
models/jamba.cpp
|
|
models/lfm2.cpp
|
|
models/llada-moe.cpp
|
|
models/llada.cpp
|
|
models/llama-iswa.cpp
|
|
models/llama.cpp
|
|
models/mamba.cpp
|
|
models/minicpm3.cpp
|
|
models/minimax-m2.cpp
|
|
models/modern-bert.cpp
|
|
models/mpt.cpp
|
|
models/nemotron-h.cpp
|
|
models/nemotron.cpp
|
|
models/neo-bert.cpp
|
|
models/olmo.cpp
|
|
models/olmo2.cpp
|
|
models/olmoe.cpp
|
|
models/openai-moe-iswa.cpp
|
|
models/openelm.cpp
|
|
models/orion.cpp
|
|
models/pangu-embedded.cpp
|
|
models/phi2.cpp
|
|
models/phi3.cpp
|
|
models/plamo.cpp
|
|
models/plamo2.cpp
|
|
models/plm.cpp
|
|
models/qwen.cpp
|
|
models/qwen2.cpp
|
|
models/qwen2moe.cpp
|
|
models/qwen2vl.cpp
|
|
models/qwen3.cpp
|
|
models/qwen3vl.cpp
|
|
models/qwen3vl-moe.cpp
|
|
models/qwen3moe.cpp
|
|
models/qwen3next.cpp
|
|
models/refact.cpp
|
|
models/rnd1.cpp
|
|
models/rwkv6-base.cpp
|
|
models/rwkv6.cpp
|
|
models/rwkv6qwen2.cpp
|
|
models/rwkv7-base.cpp
|
|
models/rwkv7.cpp
|
|
models/seed-oss.cpp
|
|
models/smallthinker.cpp
|
|
models/smollm3.cpp
|
|
models/stablelm.cpp
|
|
models/starcoder.cpp
|
|
models/starcoder2.cpp
|
|
models/t5-dec.cpp
|
|
models/t5-enc.cpp
|
|
models/wavtokenizer-dec.cpp
|
|
models/xverse.cpp
|
|
models/mistral3.cpp
|
|
models/graph-context-mamba.cpp
|
|
)
|
|
|
|
set_target_properties(llama PROPERTIES
|
|
VERSION ${LLAMA_INSTALL_VERSION}
|
|
SOVERSION 0
|
|
MACHO_CURRENT_VERSION 0 # keep macOS linker from seeing oversized version number
|
|
)
|
|
|
|
target_include_directories(llama PRIVATE .)
|
|
target_include_directories(llama PUBLIC ../include)
|
|
target_compile_features (llama PRIVATE cxx_std_17) # don't bump
|
|
|
|
target_link_libraries(llama PUBLIC ggml)
|
|
|
|
if (BUILD_SHARED_LIBS)
|
|
set_target_properties(llama PROPERTIES POSITION_INDEPENDENT_CODE ON)
|
|
target_compile_definitions(llama PRIVATE LLAMA_BUILD)
|
|
target_compile_definitions(llama PUBLIC LLAMA_SHARED)
|
|
endif()
|