This GPU can be Hacked to Unlock Secret Memory

Summary by Krater

Linus Tech Tips explores whether old Nvidia CMP 170HX mining GPUs can be repurposed for running large language models locally, testing stock performance against unlocked memory bandwidth and PCIe configurations.

From the video

Answers: Can you use old Nvidia mining GPUs like the CMP 170HX for AI and running LLMs?

• Nvidia CMP 170HX mining GPU
• LLM inference with llama.cpp
• PCIe and memory bandwidth unlocking on mining GPUs
• Comparing CMP 170HX against RTX 3090 Ti for AI workloads

What it concludes

• Unlocking the CMP 170HX unlocks floating point performance, hidden memory, and PCIe interface speeds, turning it from e-waste into a viable machine for running small language models.
• Running LLMs on dual CMP 170HX cards offers massive VRAM capacity for large models, though at slower compute speeds compared to consumer gaming cards like the RTX 3090 Ti.
• Buying used CMP 170HX mining cards carries a high risk of hardware failure due to manufacturing defects or degradation from previous mining operations.
• Applying hardware modifications such as soldering capacitors can unlock full x16 PCIe functionality on the CMP 170HX, though performance still depends heavily on bandwidth limitations.