Next bit of the Home Lab update – a 10Gbs link between the 2 servers. For less than £50?
Mellanox MT26448 ConnectX 10GbE. (PCIe 2.0)
From the US. £48 delivered. In 8 days. 2 cards, 2 cables with SFP+ on each end. Courtesy of eBay.
installation went like this:
- Shut down server and unplug
- remove top cover and remove daughter-board
- Install card on low-profile side
- re-install daughter-board
- close server and reconnect power
- Fire It Up!
ESXi 6 detected it and installed the drivers without any intervention, happily displays 10GbE.
Same for the other server. Hook up the cable and lo and behold – 10Gbs goodness in your system.
First test – vMotion. 2 new vmk ports – 192.168.120.1 and 192.168.120.2 with a mask of 255.255.255.252. Swap the vMotion traffic over. I had used vMotion over the USB ports and it was pretty quick taking 30 – 40 seconds for a hefty vm. Over the new 10Gbs it went….. 2%… done. In about 5 seconds!
But its not all speed I’m interested in, its being able to run a number of VLANs and networks across it. This is a lab after all. I need a couple of decent SSDs and more RAM and VSAN is next on my target list.
To go along with VCSA 6.5 HA. I set that up this afternoon – running over 10Gb. Works really well.
The long-term plan is to find a managed switch with 4 x SFP+ and 24 or 48 copper ports then I can expand what I can do with 10gbs. With another 2 cards and cables……
Carrying on from my previous post – Home Lab – reloaded! I thought I would detail the additions I’ve made.
The first was a follow-on from a post on VirtuallyGhetto – William Lam’s excellent blog.
He has a post detailing how to set up a USB 3.0 single or dual NIC adapter.
I have followed the instructions and can confirm that it works a treat and takes 2 minutes to install. Download the VIBs from Williams page, follow the instructions and Robert is your mother’s brother!
I used the Dual-port StarTech – £50 from Amazon. USB 3.0 has a throughput of 5Gbs. So 2 x 1Gbs NICs are not exactly going to push it are they? Network speed is excellent, although I haven’t monitored it yet so don’t have the numbers. But I do now have 4 x 1Gbs ports.
The StarTech adapters also support LACP, Jumbo frames (4000kb max), and lots of other goodies. So if you are NIC-challenged – its an easy setup and an easy win.
Right. I have a new home lab. Still have the old one but the new one is SO much better.
I decided to bite the bullet and buy new. Restrictions included:
- Price – it is a home lab after all
- Power consumption
- Normal operating volume
I really liked the Supermicros but at the price point and with DDR4 being the price it is, they were just too expensive. So before Christmas I did some extensive research, and stumbled upon the HPE DL20 Gen 9. They are on the vSphere 6 HCL, They use the E3 v5 Xeons, and its the cheapest server I could find with the E3-1230-V5, the lowest powered of the E3 family with hyperthreading. So I bought 2. And was surprised to find them arrive with rapid-rail mounting kits! Its all good! And I am a very very happy camper!
And as far as the WAF goes – well she said that I could dip into savings to build my lab, but it MUST be future-proofed. And I MUST be happy with it afterwards. Would be rude not to comply. After all….
The story so far:
- 1U shallow 15″ depth.
- 2 x PCIe 8 slots on daughter board
- 2 x M2 headers on the Mobo
- 2 x 1gbs NICs – one shared with iLO
- Mine came with 16GB – 1 x DDR4
- They are whisper-quiet, the fans do spin up occasionally and it sounds like they are breathing. VERY Quietly!
- Booting off a SanDisk 32GB USB 3.0 drive from the onboard socket
- ESXi 6.0U2 – HP Custom Image
- No CD but who uses them these days? I have a USB BluRay player anyway if I need it.
Only negative is the inclusion of 2 x USB 2 ports on the front – but no VGA. Odd. 2 x USB 3.0 on the rear with a VGA there though. In a tight rack its a tad frustrating having to pull the server out to struggle to plug in the monitor though a small gap at the back in but I have resurrected an old 2-port KVM to use with them so that’s not necessary any more.
They come in 2 storage configurations – 2 x 3.5″ SAS and 4 x 2.5″ Hot-swap SAS. I opted for the 2.5″ hot-swap for greater flexibility. However no drive caddies. This IS HP don’t forget. However – Amazon to the rescue – £12. Not HP genuine but have the spinney round LEDs. I bought a pair of WD Red 750GB 2.5″ NAS drives. There are reasons for this:
- They are SATA – but consumer-grade SATAs are not designed for 24×7 operation. The WD NAS Drives are.
- They are designed for hot-swap and RAID – so can handle failing properly and being removed from a RAID without taking the whole set down.
- I already have 2 of the 3.5″ WD Reds in 3TB versions and really like them.
- Its a HOME LAB not an enterprise production datacenter. SAS drives are just too expensive. These were a good price. Don’t care about massive performance differences.
- NAS can present NFS datastores and run VMs happily. So these should do fine.
- I also run iSCSI from both servers to a 3TB WD Red in a Synology NAS.
This is the basic setup. I will add other posts to separately detail the further enhancements – USB 3.0 NICs, 10Gbs NICs, VCSA 6.5 and so on.
These little servers just keep giving and I’m really happy with them.