New hardware
My lab before 2024 refreshment consisted of three nodes:
- AMD Ryzen 9 3900XT 12-Core @ 3.8GHz, 128 GB RAM
- i7-7700 4-Core @ 3.60GHz, 64 GB RAM
- 1st gen Raspberry-Pi (as a Proxmox arbitrer node)
2nd node based on an old Lenovo Intel PC turned out to be not choice:
- Heterogenous CPU architecture does not work for Proxmox Live Migration. Very often the VMs froze or crash after LM.
- 4 Cores and only 64 GB was not sufficient to run more and more sophisticated k8s workloads.
Decided to retire the Lenovo node and purchase a new AMD-based node, something like:
| Node 1 (pve) | Node 2 (pve2) | Arbitrer Node 3 (pve3-pi) | |
|---|---|---|---|
| “Old” setup | Processor: AMD Ryzen 9 3900XT 12-Core @ 3.8GHz Motherboard: Ausu TUF GAMING X570-PLUS Memory: 128 GB , 3200 MHz | Processor: i7-7700 CPU (4 cores) @ 3.60GHz Motherboard: Lenovo Memory: 64 GB, 2400 Mhz See https://jhartman.pl/2018/06/14/my-current-home-virtualisation-lab-wieloryb/ | Raspberry-Pi, 1st gen |
| 2024 setup | Processor: AMD Ryzen 9 5950X 16-Core @ 3.4GHz Motherboard: Ausu TUF GAMING X570-PLUS Memory: 128 GB , 3600 MHz | Processor: AMD Ryzen 9 3900XT 12-Core @ 3.8GHz Motherboard: Asus TUF GAMING B550M-PLUS Memory: 128 GB , 3200 MHz | Raspberry-Pi, 1st gen |
The intention was to upgrade the “old” Node 1 with a new, 16-cores AMD 5950X and 3600 MHz RAM. Then repurpose the old 3900XT and 3200 MHz RAM in the “new” Node 2. That’s why decided for previous generation AM4 AMD CPU and DDR 4’s.
The new Node 2 will use 2x 1TB SSD and 2x old 2TB HDD from the “old” Node 2.
Bill of Materials
The decide on the BoM was quite simple (except the case…). I’ve had, a few base assumptions:
- Mini-ATX case + motherboard – most of the storage will be still in the Node 1, Node 2 should host only 2x SSD + 2x so no need for a big box
- Previous generation AMD CPU, to repurpose the old CPU from the Node 1
- A motherboard supporting 128 GB in 4×32 dices. Due to significant price difference I’ve accepted limitation of B550 chipset.
The PC case turned out to be most difficult to decide on. Somehow majority of the cases are full of gimmicky LED lights and glass windows… After all purchased Asus Prime AP201. But this decision took 2 days…
Photos
Benchmarks
System benchmarks executed with sysbench.
- Single-thread test (CPU speed: events per second)
root@pve:~# sysbench cpu run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time
...- Multi-thread test – with number of threads set to number of CPU cores * 2 (CPU speed: events per second)
root@pve:~# sysbench cpu run --threads=24
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 24
Initializing random number generator from current time- Memory (MiB/sec)
root@pve:~# sysbench memory run
sysbench 1.0.20 (using system LuaJIT 2.1.0-beta3)
Running the test with following options:
Number of threads: 1
Initializing random number generator from current time| Node 1 (pve) | Node 2 (pve2) | |
|---|---|---|
| “Old” setup 3900XT + i7-7700 | Single-thread: 2291.57 Multi-thread (24): 25992.38 Memory: 7474.94 MiB/sec | Single-thread: 1376.30 Multi-thread (8): 8906.59 Memory: 7351.21 MiB/sec |
| 2024 setup 5950X + 3900XT | Single-thread: 5251.13 (+129%) Multi-thread (32): 84626.83 (+226%) Memory: 8906.55 MiB/sec (+19%) | Single-thread: 2297.64 (+67%) Multi-thread (24): 26078.16 (+193%) Memory: 7687.19 MiB/sec (+5%) |