Home Lab – 10Gbs Install

Next bit of the Home Lab update – a 10Gbs link between the 2 servers.   For less than £50?

Mellanox MT26448 ConnectX 10GbE. (PCIe 2.0)

From the US.  £48 delivered.  In 8 days.  2 cards, 2 cables with SFP+ on each end.  Courtesy of eBay.

installation went like this:

  • Shut down server and unplug
  • remove top cover and remove daughter-board
  • Install card on low-profile side
  • re-install daughter-board
  • close server and reconnect power
  • Fire It Up!

ESXi 6 detected it and installed the drivers without any intervention, happily displays 10GbE.

Same for the other server.  Hook up the cable and lo and behold – 10Gbs goodness in your system.

First test – vMotion.  2 new vmk ports – 192.168.120.1 and 192.168.120.2 with a mask of 255.255.255.252.   Swap the vMotion traffic over.   I had used vMotion over the USB ports and it was pretty quick taking 30 – 40 seconds for a hefty vm.  Over the new 10Gbs it went….. 2%… done.  In about 5 seconds!

But its not all speed I’m interested in, its being able to run a number of VLANs and networks across it.  This is a lab after all.  I need a couple of decent SSDs and more RAM and VSAN is next on my target list.

To go along with VCSA 6.5 HA.  I set that up this afternoon – running over 10Gb.  Works really well.

The long-term plan is to find a managed switch with 4 x SFP+ and 24 or 48 copper ports then I can expand what I can do with 10gbs.  With another 2 cards and cables……

 

Leave a Reply