ggml-org-llama-cpp-mirror

mozempk/ggml-org-llama-cpp-mirror

Fork 0

mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-02 12:42:19 +02:00

Commit Graph

Author	SHA1	Message	Date
JohnandGitHub	d34633d8db	clip : support more quantization types (#4846 ) Uses ggml functions instead of hardcoded names and adds support to quantize into the modern Q-K variants. This is just the bare minimum to get k-types working - a more refined choice of types would be needed to get best quality on low quantizations. I ran a few tests, it doesn't break anything I could notice and a Q6_K ViT works almost as well as Q8_0 but 3 times the inference speed.	2024-01-10 15:37:09 +02:00
JohnandGitHub	33c9892af5	llava : ShareGPT4V compatibility (vision encoder only loading) (#4172 ) * ShareGPT4 compatibility (vision encoder only loading) Load only a CLIP vision encoder (as supplied by ShareGPT finetunes) Corrects the argument parsing for --img_mean and --img_std (which were previously not parsed but attempted to access) Defines defaults for img_mean and img_std which are equal to the llava 1.5 CLIP encoder, so you do not have to provide them * Update convert-image-encoder-to-gguf.py	2023-11-30 23:11:14 +01:00
JohnandGitHub	11173c92d6	py : Falcon HF compatibility (#4104 ) Falcon HF compatibility	2023-11-17 17:24:30 +02:00

Author

SHA1

Message

Date

JohnandGitHub

d34633d8db

clip : support more quantization types (#4846 )

Uses ggml functions instead of hardcoded names and adds support to quantize into the modern Q-K variants.
This is just the bare minimum to get k-types working - a more refined choice of types would be needed to get best quality on low quantizations.

I ran a few tests, it doesn't break anything I could notice and a Q6_K ViT works almost as well as Q8_0 but 3 times the inference speed.

2024-01-10 15:37:09 +02:00

JohnandGitHub

33c9892af5

llava : ShareGPT4V compatibility (vision encoder only loading) (#4172 )

* ShareGPT4 compatibility (vision encoder only loading)

Load only a CLIP vision encoder (as supplied by ShareGPT finetunes)
Corrects the argument parsing for --img_mean and --img_std (which were previously not parsed but attempted to access)
Defines defaults for img_mean and img_std which are equal to the llava 1.5 CLIP encoder, so you do not have to provide them

* Update convert-image-encoder-to-gguf.py

2023-11-30 23:11:14 +01:00

JohnandGitHub

11173c92d6

py : Falcon HF compatibility (#4104 )

Falcon HF compatibility

2023-11-17 17:24:30 +02:00