Best GPU for AI and Local LLMs

Short answer

Best, in this table, means the largest model (by Hub parameter count) whose Q4_K_M estimate leaves at least 10 percent of the card free. Ties break toward the lower estimate, then the page slug. There is no street price in the sort.

Memory sizes are the configurations named in the first column, copied from the manufacturer pages linked on each row. The excerpts are stored in this repo. Where a family page lists 2 sizes, the row says which one it uses.

Apple's row is unified memory. The pool is shared with the operating system, so fit and headroom use 75 percent of the 48 GB figure, which is 36 GB. The memory column still shows the 48 GB printed on the spec page.

H100 SXM is the 80 GB column on NVIDIA's table. That page does not label the 80 GB cell with a memory type, so the type cell says not stated.