Building a Home Lab with ESXi 5.5, VSAN, and Mac Mini Server (6,2) (Part 1 of 3)

 

This will be a three part series around my experience building a lab environment utilizing Apple’s Mac Mini with ESXi 5.5 and VMWare VSAN therein.  This first post will focus on my choice of hardware components and supportability topics.  In part two I will provide a detailed account of the steps taken to run ESXi 5.5 on the Mini, the configuration of VSAN, and creation of a VM Storage Profile.  Lastly, in part three, I will focus on performance of my VSAN datastore using esxtop and testing failure/maintenance scenarios of the environment.  Please don’t hesitate to reach out to me on Twitter (@initDave) with any questions, comments, or critiques you might have.

Having a place to experiment on software and hardware without the fear of impacting the work or services of others is a beautiful thing.  You can make and break configurations all day long without any apprehension or fear of it affecting others.  Nesting an environment, that is running VMs within VMs, is also a great way to get your hands dirty in this way.  This is the method I used for the last several years on a Dell PowerEdge 2950 Server and it has served me very well.

With that, I am excited to announce this server’s retirement and to usher in my Home Lab v2.0, stepping away from nested while simultaneously reducing my footprint.  (Sort of, the Synology NAS and Cisco 200 Series switch may challenge that argument).  For the last three weeks I have been running vSphere 5.5 with VSAN Beta and it has been working great.  I’d say that I’m sad to see my rackmount server go, but I’d be lying.

 

old New

Home Lab v1.0

Dell PowerEdge 2950

2x Quad Core Xeon E5500

32GB RAM

6x 450GB 15k SAS (LSI Raid Controller)

2x 1GB NIC

Home Lab v2.0

3x Mac Mini Server

2.3GHz Quad-Core Intel Core i7

16GB RAM

128GB Samsung 840 SSD

1TB 7200RPM HDD

Synology DS1813+ NAS (8x 2TB WD Red)

Cisco SG200-26 24 Port GigE Switch (LACP and Static VLAN)

 

What drew me in to the idea of running this lab on the Mac Mini was the novelty, the challenge, and size of it.  I referenced William Lam’s work over at virtuallyghetto prior to and after purchasing my lab. I highly recommend you survey the waters prior to committing to any one build. Chris Wahl on his website, details his own lab while providing a plethora of links to HCL and non-HCL compliant builds of others.

An important choice I had to make when designing this home infrastructure setup was supportability.  The Mac Mini hardware is not on the official VMWare HCL list NOR is its AHCI controller as it pertains to VSAN.  With this project being a personal investment, it is important you understand and are comfortable with that.  However, while still not ‘official’, the VMWare Community is filled with a rock solid group of enthusiasts and professionals who impress me more and more everyday.  I utilized some stellar blogs in this little project of mine and I will be providing a consolidated reference list of all the articles I leveraged at the end of this series.  If you aren’t already reading or following the likes of William Lam, Duncan Epping, Scott Lowe, and Cormac Hogan, I highly suggest you do!

Lets get into the details of the hardware.

Compute

An additional benefit of the Mac as an ESXi host is the ability to run OS X virtualized without any special tomfoolery.  If you follow me on twitter you may have seen that I run a 2007 Mac Pro as a Plex Media server and I felt it was time to take this thing virtual.  The Mini was intriguing to me in conjunction with VSAN due to how small the footprint of the environment is.  My cluster capacity stands at 28Ghz across 12 Physical Hyper-Threading enabled cores, 48GB of Memory, 2.73TB of data storage, and 300GB of SSD read caching.  All of that and you can barely even tell they are powered on, which when compared to the full rackmount server, well.. you can’t compare.

Storage

Another key design element was a highly redundant form of storage for family photos, videos, and general backups, hence the Synology DS1813+.  The folks over at Synology caught my eye on the VMWorld floor with their iSCSI storage replication/failover features, scalability, Time Machine drive emulation, and VAAI support.  A great alternative would be the DS1513+, which is a 5-Bay NAS and is cheaper.  I was also interested in a storage solution that would function independently of my vSphere environment, which I fully intend to destroy and rebuild at least 10x in the next month or two.

Networking

For the network side I chose the Cisco SG200 series because I couldn’t afford a GigE Catalyst switch and this was a cheap alternative that fit my needs.  The 200 series supports LACP (up to four groups) and VLANs with static routing, which covers everything I was looking for in my attempt to simulate a segmented physical network.  I would have liked the additional features a full IOS or NX-OS implementation would bring, but my basement is not in that market segment unfortunately.

Look for part two in the next couple days where I dive into the technical bits of configuring ESXi 5.5 and VSAN on Mac Mini.  I’ll leave you off with a shopping list of the components inside this apparatus and links to some great immediate resources.

Apple Mac Mini Server (2012)

Corsair 16GB (2x8GB) DDR3 1600Mhz

Samsung 840 Pro Series 128GB SSD

SanDisk Cruzer Fit 8GB (For ESXi installation)

Apple Thunderbolt to GigE Adapter

Synology DS1813+ iSCSI NAS

2TB Western Digital Red NAS Hard Drive

Cisco SG200-26 24 Port GigE Smart Switch

 

Duncan Epping – http://www.yellow-bricks.com

William Lam – http://www.virtuallyghetto.com

Cormac Hogan – http://cormachogan.com

Post to Twitter Post to Delicious Post to Digg Post to StumbleUpon

  • darkdavid

    How did u cluster the 3 mac?

  • Keith W.

    Hi, I recently picked up some 2950s from work and I also have built a home lab with 2 white boxes and shared storage. Do you think have the 2 2950s around is beneficial or should I continue trying for the smaller footprint though these were free?

  • Cyril

    Where are part 2 and 3 ? I am building the same type of lab.

  • David Cunliffe

    Is this going to be updated anytime soon I’m dying to see the next part??????

  • Pingback: Building a Home Lab with ESXi 5.5, VSAN, and Mac Mini Server (6,2 ... - My Website()

  • samsonite801

    Your setup looks awesome with the Minis! I happened to notice you are using LACP (dynamic etherchannel) for your networking. I recently set up my own 3-node VSAN cluster on 3 Intel 1U blades at home, and for VSAN (over vDS) I use LBT with excellent results. Vsphere hosts and vswitches are not capable of looping so LAG/LACP is pointless. Do a Google search for ‘LACP vs LBT’ and you can find articles about this. Just food for thought is all… If you have specific reasons to run LACP in your environment then that’s fine. I do know that on mine with LBT, I am running my managed switch but there are no ehterchannels set up at all. I have all the VSAN ports plugged in with absolutely no switch configuration (default, I could use a dumb-switch for this). Vsphere doesn’t cause broadcast loops. It just works, and is simpler to set up.. It will also better utilize the uplinks for traffic balancing.