This inference engine runs LLMs 4x faster than llama.cpp: (Qwen3.5-9B at 92 tokens/s on 16GB Mac) I ran Qwen3.5 9B on a base M5 MacBook Pro using three local inference engines, one at a time. → llama.cpp reached 22.0 tokens/s → MLX reached 25.1 tokens/s → Uzu reached 92.1

Name: This inference engine runs LLMs 4x faster than llama.cpp: (Qwen3.5-9B at 92 tokens/s on 16GB Mac) I ran Qwen3.5 9B on a base M5 MacBook Pro using three local inference engines, one at a time. → llama.cpp reached 22.0 tokens/s → MLX reached 25.1 tokens/s → Uzu reached 92.1
Channel: Akshay
Description: This inference engine runs LLMs 4x faster than llama.cpp: (Qwen3.5-9B at 92 tokens/s on 16GB Mac) I ran Qwen3.5 9B on a base M5 MacBook Pro using three local inference engines, one at a time. → llama.cpp reached 22.0 tokens/s → MLX reached 25.1 tokens/s → Uzu reached 92.1

@Akshay

1 grabs

Download

This inference engine runs LLMs 4x faster than llama.cpp: (Qwen3.5-9B at 92 tokens/s on 16GB Mac) I ran Qwen3.5 9B on a base M5 MacBook Pro using three local inference engines, one at a time. → llama.cpp reached 22.0 tokens/s → MLX reached 25.1 tokens/s → Uzu reached 92.1

No comments yet