It took me about three hours total to set up a local model. I already have a GPU and I have fiber for the download. llama.cpp is not difficult to compile and has many backends. It can run parts of the model on different backends, like in the common case that the GPU doesn't have enough VRAM for everything. There are many step-by-step guides available.
Takes even less depending on your system. LM Studio or Lemonade and you are set up in minutes and now they can even tell you what models will fit with the memory you have.
And it would be in seconds if models weren’t that large and slow-ish to download! LM studio is such a noob friendly experience, pretty neat first experience!
At least for the most part, if you are downloading from huggingface, you should be able to saturate your connection. I know I usually can pretty easily even with a 5gig connection at home.
PS: This whole thread reminded me of several managers I've worked with who were pathologically unable to estimate... anything, be it driving time or development effort.
They always focused on the "minimal aspect", ignoring everything before and after. Walking to the car park. Standing in line at the machine. Paying at the machine. Getting out of the car park in the car, surprisingly long during busy times. Driving through traffic. Any delays that could -- and regularly do -- occur. Finding parking. Actually parking. Walking from the car park. Etc.
