01 · Before power-up
650 W may be enough; cabling still has to be validated
Hardware correction: the new card is the RTX 5060 Ti 16 GB, not a 5070 Ti. The 5060 Ti reference board power is 180 W; the RTX 3060 reference board power is 170 W.
A sound 650 W unit can be adequate for this system. Approval depends on the exact PSU model, condition, +12 V capability and cable topology. If a replacement is needed because of missing connectors or margin, a quality 750 W unit is a reasonable target; 850 W is optional headroom, not a requirement created by these two GPUs alone.
Using 180 W for the 5060 Ti, 170 W for the 3060, about 120 W for the CPU and 50–90 W for motherboard, RAM, storage and fans gives roughly 520–560 W DC. This is a planning estimate, not a certified maximum.
At power limits of 150 + 120 W, plus 120 W CPU and 70 W for the rest of the system, the arithmetic total is 460 W DC. That improves headroom but does not make unsafe cabling safe, and a driver reset or lost profile can remove the software limit.
PSU load estimator
Cable decisions
| Connection | Decision |
|---|---|
| Second plug on the same PCIe cable already feeding the 3060 | Do not assume it is adequate. The connectors share part of the cable. Prefer independent PCIe cables when the PSU provides them, or obtain explicit manufacturer confirmation for the combined load. |
| Separate PCIe cable / supplied GPU adapter | Valid when it matches the exact board model and manual. Some 5060 Ti models use one 8-pin connector; others use a 16-pin connector. |
| Native 12V-2x6 cable | Appropriate for compatible 16-pin models. Fully seat the connector and avoid sharp bends or lateral load near it. |
| SATA/Molex to PCIe, generic splitter, EPS/CPU 8-pin | Do not use for this installation. EPS and PCIe pinouts are not interchangeable. |
| Modular cable from another PSU | Do not reuse unless compatibility is explicitly documented for the exact PSU. PSU-side pinouts vary. |
02 · Physical and electrical compatibility
Installation sequence
- Identify the exact parts: PSU model, exact RTX 5060 Ti model, X99 motherboard and E5-2695 revision (v3 or v4).
- Check space and PCIe lane mapping: card length, thickness, intake clearance and cable clearance matter. A physical x16 slot may operate at x4 or share lanes with NVMe.
- Power fully off: shut down, switch off the PSU and remove the wall cable before installing hardware. Never open the PSU.
- Start with the 5060 Ti alone on validated power. Confirm display, driver and temperatures, then add the 3060.
- If the second GPU does not appear, inspect motherboard documentation, BIOS lane mapping and Above 4G Decoding. Do not change CSM/boot mode blindly.
- Increase load gradually. Stop testing on resets, GPU errors, damaged connectors, burning smell or abnormal electrical noise.
On Windows, GPU-Z and HWiNFO can help inspect negotiated link width and sensors. On Linux, lspci -vv shows PCIe link capability/status. Avoid mining-style x1 risers for workloads that move significant data between host and GPUs.
03 · Where time is spent
28 GB of capacity does not become one 28 GB GPU
| Component | Capacity / bandwidth | Implication |
|---|---|---|
| RTX 5060 Ti | 16 GB; ~448 GB/s | Prefer models that fit entirely on it; higher bandwidth and compute than the 3060. |
| RTX 3060 | 12 GB; ~360 GB/s | Adds capacity or can run a second model; splitting one response across GPUs can add serial stages. |
| E5-2695 v4 + DDR4 | 4 channels, up to 76.8 GB/s theoretical at DDR4-2400; 18C/36T | 128 GB gives capacity; sustained memory bandwidth must be measured. |
| E5-2695 v3 | Up to ~68 GB/s with 4-channel DDR4-2133 | CPU revision changes the memory reference point. |
| X99 PCIe 3.0 | x16 ~15.75; x8 ~7.88; x4 ~3.94 GB/s each direction before overhead | Much slower than VRAM. Resident weights avoid moving the whole model every token. |
Dense models
Most weights participate in every generated token. Layer splitting keeps weights resident on their assigned GPUs and passes activations between stages. Throughput is therefore not the sum of 448 + 360 GB/s.
Idealized serial example: if 60% of bytes are served by the 5060 Ti and 40% by the 3060, 1/(0.6/448 + 0.4/360) ≈ 408 GB/s before compute and communication. That can still beat spilling a large fraction into system RAM.
Mixture-of-Experts models
MoE routes each token through only a subset of experts. All weights must remain addressable, but only selected experts are computed. Attention, routers, KV cache and buffers still consume memory.
Experts placed in system RAM can be computed by the CPU during decode while the GPU handles shared tensors or batched work. Exact behavior depends on backend version and kernels.
Prefill processes the prompt; decode generates new tokens. Prefill benefits more from batching and compute throughput; decode is often more sensitive to memory bandwidth, latency and dequantization. Low GPU utilization can mean the GPU is waiting on the CPU, and low CPU utilization can mean cores are waiting on RAM.
Do not reduce the number of experts activated by the model just to make it faster. That changes the computation. In llama.cpp, --n-cpu-moe 5 refers to MoE tensors from the first five layers being on CPU, not five individual experts.
04 · Choose by actual file size
Models that make sense on this system
Initially reserve roughly 2–4 GB across the GPUs for context, buffers and display use; real requirements may be higher. Use the exact GGUF file size and load log instead of relying on nominal parameter count.
| Model / family | Starting placement | Why |
|---|---|---|
| Llama 3.1 8B Q4; Qwen2.5 14B Q4 | 5060 Ti only | Good baseline for measuring a GPU-only workload. |
| Qwen3.6-35B-A3B IQ1_M (~10 GB as previously reported) | 5060 Ti only, no CPU experts if it fits | Repeat the prior ~53 t/s test with identical file, context and prompt. |
| Qwen3.6-35B-A3B Q4_K_M (~22.07 GB) | Both GPUs, all layers/experts in VRAM if buffers permit | High-priority quality upgrade over Q1 while avoiding system RAM. |
| Qwen3.8-27B Q4 (~17.74 GB); other dense 27–32B Q4 | Both GPUs | Does not fit by weights alone on 16 GB, but should fit within combined VRAM if buffers allow. |
| Bonsai 27B Q2 (~8.52 GB) | 5060 Ti, or independently on the 3060 | Useful for comparing single-response latency versus two-agent throughput. |
| Dense 70B Q4 (~40–45 GB order of magnitude) | RAM + GPUs | Fits system capacity but remains heavily dependent on CPU/RAM; not the first choice for maximum speed. |
| Qwen3.8 Flash-Next | Hybrid: shared tensors/buffers on GPUs, most experts in RAM | Even Q1 is far larger than 28 GB VRAM. |
Published Unsloth Flash-Next GGUF sizes
| Quantization | Listed size | Interpretation for 128 GB RAM + 28 GB VRAM |
|---|---|---|
| UD-IQ1_S / UD-IQ1_M | 72.5 / 74.5 GB | Start with the file already benchmarked to isolate the hardware change. |
| UD-Q2_K_XL / UD-IQ3_XXS | 78.9 / 82 GB | Reasonable candidates for a quality comparison. |
| UD-IQ4_XS | 93.7 GB | Fits in principle with careful memory use; this is the quantization used in the published dual-3060 test. |
| UD-Q4_K_XL | 111 GB | Tighter; inspect resident RAM, buffers and any CPU/GPU duplication. |
| UD-Q5_K_XL | 158 GB | Not comfortable: the file alone exceeds nominal 128 + 28 GB before overhead. |
Best first comparison: run Qwen3.6 Q4 fully across both GPUs for daily use, and use Flash-Next hybrid only when its quality justifies the extra latency. For two independent jobs, one model per GPU avoids cross-GPU communication.
05 · Published measurements are not forecasts
Useful benchmarks and their limits
| Test | Prompt processing | Generation | Limitation |
|---|---|---|---|
5060 Ti 16 GB · Llama 2 7B Q4_0 · Flash Attention · llama.cpp 89d1029 | 4,195.53 t/s (pp512) | 93.46 t/s (tg128) | Short-context microbenchmark. |
3060 12 GB · same model/FA · baa9255 | 2,407.67 t/s | 76.92 t/s | Different host and commit; not a controlled A/B. |
2×3060 + 7800X3D · Flash-Next IQ4_XS · llama.cpp · -ub 2048 | 303 t/s | 12 t/s | Synthetic 8k prompt; 131k context allocated. |
Same host/model · ik_llama.cpp · -ub 2048 | 407 t/s | 13 t/s | Only 64 generated tokens; one user's machine. |
The CUDA entries imply roughly 1.22× generation and 1.74× prefill between those particular 5060 Ti and 3060 results, but hosts and commits differ. The theoretical VRAM bandwidth ratio is only about 1.244× (448/360). The most important benefit of the 16 GB card may therefore be avoiding CPU offload rather than raw GPU speed alone.
06 · Starting points
Configurations to compare — not claims of optimality
Use a recent CUDA build that supports the GGUF architecture and both GPUs. First run llama-server --list-devices and llama-server --help. The examples assume CUDA0 = 5060 Ti and CUDA1 = 3060; change that if device enumeration differs.
A · 5060 Ti only: minimum communication
llama-server -m model.gguf --device CUDA0 -ngl 999 -sm none -c 8192 -b 2048 -ub 512 -fa on -ctk q8_0 -ctv q8_0 -t 18 -tb 18 --parallel 1 --host 127.0.0.1 --port 8080
Do not use --cpu-moe here. If Q8 KV cache is unsupported by the architecture/build, return to F16 and re-evaluate memory.
B · Two GPUs: model entirely in VRAM
llama-server -m model.gguf --device CUDA0,CUDA1 -ngl 999 -sm layer -ts 60,40 -c 8192 -b 2048 -ub 512 -fa on -ctk q8_0 -ctv q8_0 -t 18 -tb 18 --parallel 1 --host 127.0.0.1 --port 8080
60,40 is only a starting split. Test 55,45 and 65,35 while watching free VRAM and the load log.
C · Flash-Next: shared tensors on GPU, experts in RAM
llama-server -m model.gguf --device CUDA0,CUDA1 -ngl 999 -sm layer -ts 60,40 --cpu-moe -c 8192 -b 4096 -ub 512 -fa on -ctk q8_0 -ctv q8_0 -t 18 -tb 18 --parallel 1 --host 127.0.0.1 --port 8080
First confirm correct loading and generation, then test microbatch 1024 and 2048. Compare all experts in RAM with --n-cpu-moe N after removing --cpu-moe; vary N by layers, not by the number of routed experts.
07 · Find the best point on your machine
Optimize by measurement
Experiment order
- Fix GGUF, backend commit, prompt, context, KV cache and sampling. Compare 3060, 5060 Ti and dual-GPU.
- For Flash-Next include CPU-only and 5060-Ti-only hybrid baselines.
- On the v4 CPU test
-tat 8, 12 and 18; use 36 only as a control. More threads can increase memory contention. - Sweep microbatch 512, 1024 and 2048 while keeping batch ≥ microbatch.
- Only after power delivery is validated, test GPU power limits one variable at a time.
- Repeat at least three times after warm-up; record both short and long-prompt behavior.
Record
Prefill t/s, decode t/s, time to first token, total latency, peak VRAM, available RAM, paging, average GPU power, wall power and answer quality.
Energy efficiency: tokens/s ÷ wall watts = tokens per joule. Example: 10 t/s at 400 W = 0.025 token/J; 9.5 t/s at 300 W = 0.0317 token/J — about 27% better efficiency for 5% lower speed.
Power monitoring and limiting
nvidia-smi --query-gpu=index,uuid,name,power.draw,power.limit,memory.used,temperature.gpu --format=csv nvidia-smi -q -d POWER nvidia-smi topo -m # Admin/root; replace with real UUIDs after validating power delivery: nvidia-smi -i GPU-UUID-OF-5060TI -pl 150 nvidia-smi -i GPU-UUID-OF-3060 -pl 120
Use UUIDs so the intended board is targeted even if GPU numbering changes. Revalidate limits after reboot or driver reset. GPU telemetry does not measure the PSU as a whole; use an appropriate wall power meter for AC consumption. Software voltage readings do not validate ripple or transients.
Speed expectations without false precision
For models that fit on the 5060 Ti, published measurements support a moderate generation advantage over a 3060, but results vary strongly by model and backend. For Flash-Next there is no basis to promise that roughly 5–8 t/s will become 20–30 t/s. The 16 GB card can improve prefill, permit better quantization and reduce CPU work; decode only accelerates in proportion to how much of its critical path actually moves to faster hardware.
If 70% of runtime remains on CPU/RAM and the other 30% becomes 2.5× faster, Amdahl's-law speedup is 1/(0.7 + 0.3/2.5) = 1.22×. That is an illustrative calculation, not a measured forecast.