mozempk/ggml-org-llama-cpp-mirror
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-09-03 21:22:20 +02:00
Code Issues Packages Projects Releases Wiki Activity
573 Commits 758 Branches 7,317 Tags
07e9ace0f9da424d82e75df969642522880feb92
Commit Graph
5 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Johannes GäßlerandGitHub eb363627fd cuda : deduplicated dequantization code (#1453) 2023-05-14 21:53:23 +03:00
Johannes GäßlerGitHubGeorgi Gerganov
 		905d87b70a ggml : GPU-accelerated token generation (#1412) 
* CUDA kernel for q4_0 dequant. + mat. vec. mult.

* Added q4_1 via template

* Added missing __syncthreads();

* --gpu_layers -> --gpu-layers

* Shorter dequantize_mul_mat_vec line

* q5_0 dequantize_mul_mat kernel

* More readable dequantize_mul_mat_vec logic

* dequantize_mul_mat_vec kernels for q5_1, q8_0, f16

* llama : offload "output" tensor to GPU too + coding style fixes

---------

Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
 2023-05-13 16:38:36 +03:00
Johannes GäßlerandGitHub 773ee249fb CLI args use - instead of _, backwards compatible (#1416) 2023-05-12 14:34:55 +00:00
Johannes GäßlerandGitHub 1f48b0abcf Documented CUDA reproducibility, added warning (#1346) 2023-05-08 02:42:01 +02:00
Johannes GäßlerandGitHub 92a6e13a31 Add Manjaro CUDA include and lib dirs to Makefile (#1212) 2023-04-28 15:40:32 +02:00