mozempk/ggml-org-llama-cpp-mirror
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-08-29 10:42:18 +02:00
Code Issues Packages Projects Releases Wiki Activity
2,732 Commits 737 Branches 7,226 Tags
3fe0596c1817a6114ffffb6dbfd6c36ca7815dc7
Commit Graph
2 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
jiezGitHubGeorgi Gerganovz5269887
 		1966eb2615 quantize : add '--keep-split' to quantize model into shards (#6688) 
* Implement '--keep-split' to quantize model into several shards

* Add test script

* Update examples/quantize/quantize.cpp

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>

* Split model correctly even if tensor id is out-of-order

* Update llama_model_quantize_params

* Fix preci failures

---------

Co-authored-by: z5269887 <z5269887@unsw.edu.au>
Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2024-04-25 13:29:35 +03:00
jiezGitHubz5269887
 		91c736015b llama : add gguf_remove_key + remove split meta during quantize (#6591) 
* Remove split metadata when quantize model shards

* Find metadata key by enum

* Correct loop range for gguf_remove_key and code format

* Free kv memory

---------

Co-authored-by: z5269887 <z5269887@unsw.edu.au>
 2024-04-12 13:45:06 +03:00