mozempk/ggml-org-llama-cpp-mirror
1
0
Fork 0
mirror of https://github.com/ggml-org/llama.cpp.git synced 2026-10-31 08:07:54 +01:00
Code Issues Packages Projects Releases Wiki Activity
3,121 Commits 936 Branches 8,272 Tags
1f0dabda8d5c131f9d4632aa41de74317cdd61fb
Commit Graph
4 Commits
Author SHA1 Message Date
Johannes Gäßler 1f0dabda8d CUDA: use tensor cores for MMQ (#7676) 
* CUDA: int8 tensor cores for MMQ (legacy quants)

* fix out-of-bounds writes

* __builtin_assume -> GGML_CUDA_ASSUME

* fix writeback returning too early
 2024-06-10 11:45:13 +02:00
Johannes Gäßler 42b53d192f CUDA: revise q8_1 data layout for mul_mat_q (#7824) 2024-06-09 09:42:25 +02:00
Johannes Gäßler 7d1a378b8f CUDA: refactor mmq, dmmv, mmvq (#7716) 
* CUDA: refactor mmq, dmmv, mmvq

* fix out-of-bounds write

* struct for qk, qr, qi

* fix cmake build

* mmq_type_traits
 2024-06-05 16:53:00 +02:00
slaren ae1f211ce2 cuda : refactor into multiple files (#6269) 2024-03-25 13:50:23 +01:00