For many models the performance of llama.cpp on Mac is 20-40% lower than MLX. Di... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

dust42 68 days ago | parent | context | favorite | on: DeepSeek V4 – almost on the frontier

For many models the performance of llama.cpp on Mac is 20-40% lower than MLX. Did you try MLX? At least on HF there are MLX 2-bit quants. Unfortunately I have only 64GB, so I can't test it.

antirez 68 days ago [–]

I'm not using llama.cpp there, it's my inference engine that is DeepSeek v4 specific. The goal is to optimize it as much as possible.

oveja 63 days ago | [–]

That's cool!

I knew the name sounded familiar, thank you for SDS!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact