Another Piece Added to the Homelab

I’m still planning the details of the photo archive pipeline, so it’s not yet clear that I’ll need more hardware to handle the workload. I’m running Ollama on g11cd, making use of the Nvidia GeForce RTX 3060 12GB VRAM. Additionally, I have Open WebUI running on one node of my PVE cluster. That’s given me the opportunity to play around with some locally running LLMs. I am able to easily run deepseek-r1:8b, llama3.1:8b, qwen3-v1:8b, qwen3.5:9b, and even phi4:14b and qwen2.5-coder:14b without exceeding the VRAM. However, now that I’ve dipped my toe in the water, I’m already thinking about wading in deeper.

Looking for a better GPU

The Nvidia GeForce RTX 3060 12GB VRAM is a couple generations behind the current GeForce RTX 50 Series GPUs, but it still has a respectable amount of VRAM needed for running local models without spilling out of the GPU and leveraging the CPU and RAM, which drastically increases the models execution time. The options available for a GeForce RTX with 16GB or more VRAM are relatively few. I started my homelab using my own secondhand machines with selective purchases of minimal hardware upgrades. Cost is still one of my primary constraints, but the repurposing of used hardware has become a personal challenge and cornerstone of this project. Trying to keep my hardware spend to a minimum, I was immediately concerned about the potential cost of a more capable GPU given the AI-driven demand for GPUs and memory.

Sticking with the 30 Series, the RTX 3090 (and 3090 Ti) is available with 24GB VRAM, but even now, a used or refurbished one may go for $800 to $1,250 and a new one from a third party (if you can find one) will likely be $2000 and up. Additionally, the Thermal Design Power (TDP) for the 3090 is over twice that of the 3060 so having a sufficient PSU is another factor to consider. I already updated the PSU in the g11cd when I added the RTX 3060 12GB VRAM GPU, so I’m not keen on upgrading it already.

By comparison, the current GeForce RTX 5070 Ti and 5080 with 16GB VRAM have a starting MSRP of $750 and $1,000 for base models with higher prices for those with premium features. Of course you have to be able to find them at a retailer who has them in stock and is still offering them at MSRP. TDP for the 5070 Ti and 5080 is also still a concern at roughly double the 3060. Though the 50 Series architecture is more efficient than the 30 Series, they have far more cores and run at a significantly higher clock speed.

Between those is the 40 Series. The RTX 4060 Ti comes in a 16GB VRAM variant. With an original MSRP of $500, most current listings of the diminishing new stock range from $600 to $750, approaching the price of a new 5070 Ti. One advantage of the 4060 Ti is that it only has a TDP of 165W – far less than either the 3090 or the 5070 Ti, and even less than that of the 3060. This gave me a good option to target with more VRAM, a more modern architecture and no additional power requirements. At that point, I only needed to watch for the right deal.

Comparing Nvidia GeForce RTX GPUs

Another Tower

While biding my time looking for a good price on a GPU, I watched for local and online deals. I came across a local business selling a variety of used PCs. One in their inventory caught my eye: a Dell Precision 3620. It was similar in age to my g11cd, and with a Intel Xeon processor having 4 cores and 8 threads it will be a good partner with comparable compute capacity, if not a little more. It only had 16GB DDR4 (2x8GB) with 2 additional open DIMM slots and a 290W PSU, but I already had 16GB DDR4 (2x8GB) to fill out the DIMM slots and a 550W PSU left over from upgrading g11cd. So for $150 and some spare parts, plus a $9 24-pin-to-8-pin ATX PSU adapter cable for Dell motherboards, I had a second machine capable of handling an RTX 4060 Ti GPU and able to host local LLMs. I just needed to settle on a GPU to purchase.

Scoring the GPU

Shopping for an RTX 4060 Ti 16GB VRAM, I rarely saw anything under $700 and some over $1,000… for used GPUs. Finally, I spotted a certified refurbished one from a reputable retailer for $470. I jumped on it before it sold out. I spent a total of $629 on this additional node for my homelab. Though this seventh node more than doubled the total spend on my homelab, it increases my homelab’s total number of computational threads nearly 30% and the memory capacity by 33%. However, the real win is having a second LLM capable machine that brings the total processing power of my homelab to 36.5 TFLOPS FP32 (1.7 CPU + 34.8 GPU) – a 160% increase. With the additional VRAM of the 4060 Ti, I should be able to step up from deepseek-r1:8b to deepseek-r1:14b, compare qwen3.5:9b and qwen3:14b, easily run llama3.2-vision:11b, phi4:14b, qwen2.5-coder:14b, and even push the bounds with gpt-oss:20b.

Resilience Philosophy

I’m still trying to keep my spend low, but I’ll concede this purchase may not have been absolutely necessary. It’s certainly a risk to invest in additional used hardware, and I’m reaching the point of diminishing returns. Fortunately the bulk of this cost was in a GPU that can be moved into a newer PC if needed. If additional compute capacity is required, future hardware investments should probably be new equipment. Until then, I’ve addressed a single point of failure risk by a lone GPU and can build more resilience and parallel AI inference into my photo archive pipeline.

Hardware Inventory

graph TD
subgraph Cluster["Proxmox VE Cluster"]
  inspiron["inspiron<br/>i7-3632QM · 16GB<br/>Cluster Member"]
  allinone["allinone<br/>i5-2400S · 16GB<br/>Cluster Member"]
  flex2["flex2<br/>i3-4030U · 16GB<br/>Cluster Member"]
end

g11cd["Proxmox Virtual Environment<br/>g11cd<br/>i5-6400 · 32GB<br/>RTX 3060 12GB<br/>GPU workloads"]
precision["Proxmox Virtual Environment<br/>precision<br/>Xeon E3-1245 · 32GB<br/>RTX 4060 Ti 16GB<br/>GPU workloads"]
vaio["Proxmox Backup Server<br/>vaio<br/>i5-M460 · 8GB"]
ideapadflex["Proxmox Datacenter Manager<br/>ideapadflex<br/>i7-4500U · 8GB"]

vaio -. backs up .-> Cluster
vaio -. backs up .-> g11cd
vaio -. backs up .-> precision
vaio -. backs up .-> ideapadflex
ideapadflex -. manages/observes .-> vaio
ideapadflex -. manages/observes .-> Cluster
ideapadflex -. manages/observes .-> g11cd
ideapadflex -. manages/observes .-> precision

style Cluster fill:#f7f8fa,stroke:#e2e5e9
classDef pve fill:#eef2fb,stroke:#1f4b8f,color:#1a1d21
classDef pbs fill:#eafaf1,stroke:#1a7a4c,color:#1a1d21
classDef pdm fill:#f3eefc,stroke:#6b46c1,color:#1a1d21
classDef gpu fill:#fdf0e1,stroke:#c2410c,color:#1a1d21

class inspiron,allinone,flex2 pve
class g11cd,precision gpu
class vaio pbs
class ideapadflex pdm