CPP

42x faster prompt lookup drafting in llama.cpp
13 points | 2 comments

Llama.cpp Under the Hood
4 points | 0 comments

Transformers now runs llama.cpp quants
4 points | 0 comments

Custom Models in Oh My Pi: vLLM, Llama.cpp, SGLang and More
2 points | 0 comments

Llama.cpp banned him for accidentally tagging in a fork PR
1 points | 2 comments

Show HN: Llama.cpp fork with 2-4x multiGPU speed for MoE models bigger than VRAM
1 points | 2 comments

Using Llama-cpp-Python grammars to generate JSON
1 points | 0 comments