you need at least an RTX 6000 pro, maybe 2 to run local models on that level. probably only worth it if you plan on doing other workloads like finetuning or generating a lot of synthetic data