If you are looking to deploy multiple ESX/ESXi servers then there are plenty of methods and tools out there, some more complex than others. There are vendor specific deployment products available such as HP Rapid Depuployment Pack (RDP) which uses Altiris, or alternatively there are free deployment tools such as ESX Deployment Appliance (EsleeDA) and Ultimate Deployment Appliance (UDA). UDA is my favorite tool for the job as it offers great flexibility such as the use of subtemplates (discussed later), and therefore this will be the basis of this article. It was created by Carl Thijssen and thanks to Mike Laverick of RTFM, it also supports ESX/ESXi deployments, and the latest build supports ESX/ESXi 4.1.

I aim to share the basics of getting the UDA configured for your environment and not delve into anything too complicated here. The best way of learning to perform some of the more complex deployments are to grasp the basics first then experiment with other functionality in your own environment for yourself. Mike Laverick has an excellent guide on deploying and configuring UDA 2.0 beta.

For the purposes of this guide a number of assumptions have been made. You will have already deployed vCenter and at least one ESX/ESXi host running with some storage. If you are going to be testing this in a home lab then bear in mind that we’ll need to configure DHCP options 66 and 67, see the end of this article for details. You can do this with an existing DHCP server or use UDA as a DHCP server. Most deployments will be using an existing DHCP server.

Let’s get started…

Step 1: Download & Deploy the Ultimate Deployment Appliance

1) Download UDA from: http://www.rtfm-ed.co.uk/vmware-content/ultimate-da/ then extract it to your hard disk.
2) Import the appliance using your vSphere client (File > Deploy OVF Template...)
3) Prior to powering on the appliance, add a second disk that we’ll use to store your ISO images. I’d recommend at least a 20GB disk, and use thin to conserve disk space.
4) Power on and proceed to the next step.

Step 2: Installation and Setup

1) Click OK at the welcome screen
2) Enter the hostname (I recommend that you also add the hostname into your DNS server)
3) Enter the IP address configuration
4) Select the services you require (as a minimum I would just leave TFTP, HTTP and SSH selected)
5) Enter the root password.
6) Select Yes to apply the changes.

Once it has finished you will then be able to access your UDA from a web browser. Log in with admin and the password you set during the setup.

Step 3: UDA Configuration

When you first log into the appliance you will be presented with a welcome page. The first thing to configure is the additional disk space we need to store our ISO images. If you haven’t added a second hard disk, then shutdown the UDA (System > Shutdown), then add the additional disk and power on.

UDA has two partitions; systemlv and localv. ISO images are stored on the localv partition, so we’ll extend that.

1) Go to System and click Diskspace.
2) Select localv and then click Extend.
3) Select the device and then click Apply.

It may take a little while depending on how large the new disk is, but once that is complete we’ll load the ISO image for ESXi 4.1 Update 1 on to the UDA.

1) Go to System and click Upload.
2) Click Choose File, select your ISO then click Upload.

It will store the ISO on the localv partition in /local.

Step 4: Add an OS & Deployment Template for ESXi 4.1

Now for the exciting bit. We’ll add our OS (ESXi) and template for deployment which will also create the basics of our Kickstart script which we’ll add to later. The OS section allows us to tie an ISO image to a Flava name, in this case we’ll call it ESXi41. I’d recommend using a better naming convention for your environment though.

1) Go to OS and click New.
2) Enter ESXi41 for the Flava Name.
3) From the dropdown, choose VMware ESXi 4.1 Installable and click Next.
4) Select the ESXi 4.1 ISO you uploaded and then click Finish.

Next, the template which contains our Kickstart script. The template also contains a subtemplate which allows us to define our own variables, such as IP address and hostnames for each ESXi host we are deploying. This is why I like using UDA so much, because it avoids having to create a separate Kickstart script for each ESXi host.

1) Go to Templates and click New.
2) Enter a Template Name. For example: esxi-build and give it a meaninful description.
3) From the Operating System dropdown choose VMware ESXi 4.1 Installable and click Next.
4) From the Flava dropdown choose your OS (Flava), ESXi41 and click Next then Finish.

The next stage is to perform the configuration of our Kickstart script and subtemplate. Go to Templates, select your template (esxi-build) and click Configure. You’ll see three tabs here; General, Subtemplates and Advanced. Click on Advanced. This is where our Kickstart script lives, and you’ll see it contains a basic configuration.

Here is a breakdown with explanations for each line:


# Accept the VMware EULA - The script will fail without this.
accepteula

# Set the root password using MD5 crypt
rootpw secret

# Auto partition the disk
autopart --firstdisk --overwritevmfs

# Location of the install media
install url http://[UDA_IPADDR]/[OS]/[FLAVOR]

# Configure IP address and hostname. --addvmportgroup=0 will disable creation of the VM Network portgroup
network --bootproto=static --ip=192.168.178.200 --gateway=192.168.178.1 --nameserver=192.168.178.1 --netmask=255.255.255.0 --hostname=esx41i --addvmportgroup=0

# Reboot the host when the installation is complete
reboot

The first thing we need to change is the root password which by default is set to secret and is in plain text. We can use a crypted MD5 password here using rootpw --iscrypted followed by the crypt.

1) Log into your UDA with PuTTY (SSH) and log in as root (same password you set during the initial setup).
2) Type grub-md5-crypt
3) Enter your password and it’ll generate an MD5 string for you.
4) Go back to the Advanced view in the web interface and change the rootpw line to (replacing the crypt with the one you generated):

rootpw --iscrypted $1$3vkd233f/wksSo$fhniM3fdcV6hr0

Now we need to change the network configuration to use the appropriate IP address, gateway, subnet mask and hostname. We could just replace the default IP address configuration here, but a better method is to use variables which we can set in subtemplates. Variables are set inside square brackets, such as [IPADDR], [FQDN] and so on. The variable names are up to you. Change the network configuration, replacing the IP address and hostname with variables as follows:

network --bootproto=static --ip=[IPADDR] --gateway=192.168.4.1 --nameserver=192.168.4.1 --netmask=255.255.255.0 --hostname=[FQDN] --addvmportgroup=0

Select Subtemplates and click on Edit. The first line we will create contains the word SUBTEMPLATE then the variable names that we want to use (separated by semi-colons). Subsequent lines contain the values for each host. See my example below:

SUBTEMPLATE;IPADDR;FQDN;VMKIPADDR
ESX01;192.168.4.11;esx01.home.lab;192.168.4.211
ESX02;192.168.4.12;esx02.home.lab;192.168.4.212
ESX03;192.168.4.13;esx03.home.lab;192.168.4.213
ESX04;192.168.4.14;esx04.home.lab;192.168.4.214

You’ll notice in my example I’ve added a VMkernel IP address that in the case of our ESXi deployment we will use for VMotion.

Click on the Advanced tab. After the network configuration line we need to add the following command which allows us to run our esxcfg commands on first boot:

%firstboot --unsupported --interpreter=busybox

Now add the following commands to the Kickstart script (Advanced), which will add a VMotion portgroup and IP address. You can leave the # comments in if you wish:

# Setup VMotion portgroup on vSwitch0
esxcfg-vswitch -A VMotion vSwitch0

# Setup VMotion IP address
esxcfg-vmknic -a VMotion -i [VMKIPADDR] -n 255.255.255.0

# Wait for previous command to finish before enabling VMotion
sleep 10

# Enable VMotion (ESX uses vmware-vim-cmd and ESXi is vim-cmd)
vim-cmd hostsvc/vmotion/vnic_set vmk1
vim-cmd hostsvc/net/refresh


Notice in the vim-cmd command above that it sets VMotion on vmk1. vmk0 will be the management IP address used in the network command at the top of our script. This is fairly straightforward to work out, but if anyone knows of a better method then feel free to comment!

Finally click on Save.

The Deployment

Now we have the UDA configured and ready to roll with our ESXi installation all we have to do now is PXE boot our hosts and providing you have configured DHCP options 66 & 67 (see the end of this article) you should be presented with the UDA menu (see first screenshot).

The rest is easy, just select the host from the menu (we set this in SUBTEMPLATE) and it will install and configure ESXi with no user intervention. Using a scripted installation can be very powerful and a lot more can be configured than I’ve included here. Even if you don’t have mass ESXi deployments, this is a good way of ensuring that your ESXi hosts maintain your standard build. If you have an Enterprise Plus license then you can also use Host Profiles.

Kickstart script:

accepteula
rootpw --iscrypted $1$3vkd233f/wksSo$fhniM3fdcV6hr0
autopart --firstdisk --overwritevmfs
install url http://[UDA_IPADDR]/[OS]/[FLAVOR]
network --bootproto=static --ip=[IPADDR] --gateway=192.168.4.1 --nameserver=192.168.4.30 --netmask=255.255.255.0 --hostname=[FQDN] --addvmportgroup=0
reboot

## THE FOLLOWING IS OUR FIRSTBOOT CONFIGURATION ##

# Configure additional commands at first boot.
%firstboot --unsupported --interpreter=busybox

# Setup VMotion portgroup on vSwitch0
esxcfg-vswitch -A VMotion vSwitch0

# Setup VMotion IP address
esxcfg-vmknic -a VMotion -i [VMKIPADDR] -n 255.255.255.0

# Wait for previous command to finish before enabling VMotion
sleep 10

# Enable VMotion (ESX uses vmware-vim-cmd and ESXi is vim-cmd)
vim-cmd hostsvc/vmotion/vnic_set vmk1
vim-cmd hostsvc/net/refresh

SUBTEMPLATE:

SUBTEMPLATE;IPADDR;FQDN;VMKIPADDR
ESX01;192.168.4.11;esx01.home.lab;192.168.4.211
ESX02;192.168.4.12;esx02.home.lab;192.168.4.212
ESX03;192.168.4.13;esx03.home.lab;192.168.4.213
ESX04;192.168.4.14;esx04.home.lab;192.168.4.214

Configuring DHCP Options

1) Set option 66 to the IP address of your UDA
2) Set option 67 to pxelinux.0

In this article I detail the steps required to configure your vMA as a Syslog server, and configure your ESX/ESXi hosts to send logging information to the vMA. Logging is often overlooked, but when managing multiple hosts it is far easier to send your logs to a Syslog server. I’m studying for the VCAP-DCA exam, and using vicfg-syslog is a requirement of the exam (Section 6.1) and the vMA is also essential to understand (Section 8.1). I hope my notes help you as they have helped me.

Configuring your VMware vMA as a Syslog server is very easy to do, just follow these steps

Log on to your vMA as vi-admin and enter the following commands:

sudo service rsyslog stop
sudo vi /etc/sysconfig/rsyslog
sudo service rsyslog start
sudo iptables -I INPUT -i eth0 -p udp --dport 514 -j ACCEPT
sudo service iptables save (this will save the new firewall rule and ensure it doesn’t disappear after a reboot)

Job done! The vMA server is now ready to accept syslog connections on UDP port 514.

Configuring your ESXi host to send logging information to the Syslog server

You can do this via the vSphere client (Configuration > Advanced > Syslog, syslog.remote.hostname), but for the VCAP-DCA exam it might be useful to know how to do this using vMA.

Using vMA:

First ensure your have your ESXi host set as the fast-pass target, then use vicfg-syslog:

# vifptarget -s <ESXI_HOSTNAME>
# vicfg-syslog -s <VMA_IP_ADDRESS>

Configuring your ESX host to send logging information to Syslog server

You can’t use the vSphere client or vMA to configure Syslog on an ESX host, so we need to edit /etc/syslog.conf and add the following line to the very bottom:

*.* @<SYSLOG_IP_ADDRESS>

In this example, replace <SYSLOG_IP_ADDRESS> with the IP address of the vMA.

Next we need to allow UDP port 514 out from your ESX host. To do this run the following command (make sure you are logged in, or su - as root):

# esxcfg-firewall -o 514,udp,out,syslog

Note: If you log in to your ESX hot then su as root make sure you type (su -) with the hyphen and that will load the environment variables of the root account. Otherwise you may find that you get the error ‘bash: esxcfg-firewall: command not found‘.

Finally, restart the syslog service:

# service syslog restart

Checking the ESX logs on vMA

Now for the fun bit!  Log on to your vMA server as vi-admin, and type:

# sudo tail -f /var/log/messages

This will tail the messages file, and -f will output data as it is added to the file.

To send a test message, log on to the ESX host that you configured for syslog (above) and type:

# logger syslog test message

You should see your test message output on the vMA. Cool huh!

Checking ESXi logs on vMA

The logger command isn’t available with ESXi, so another way of checking that your Syslog is working for an ESXi host is to filter the results to show the IP address of your ESXi host:

# sudo tail -f /var/log/messages | grep <ESXI_IP_ADDRESS>

It shouldn’t take long to see messages from your ESXi host, and filtering on the IP address using grep is a really neat way of finding results for a particular host.

VMware have released vSphere 4.1 Update 1 which adds support for additional operating systems (RHEL 6, RHEL 5.6, SLES 11 SP1 for VMware, Ubuntu 10.10, and Solaris 10 Update 9). ESX/ESXi 4.1 Update 1 now supports 160 logical processors. Looking at the number of patches for ESX and ESXi below, it makes  me wonder whether this will be the last release of ESX, in favour of ESXi?

Release Summary:

ESX 4.1 Update 1 Patches:

ESXi 4.1 Update 1 Patches:

See the following URL’s for full details:

• ESX 4.1 Update 1
• ESXi 4.1 Update 1
• vCenter 4.1 Update 1

Here is a very quick and dirty tech tip on changing the IP address of your VMware vMA. There are two ways of doing this, the Linux way or the VMware Setup Wizard.

Method 1: The Linux Way

Using your favorite text editor (Vi in my case) edit the following file using sudo:

# sudo vi /etc/sysconfig/network-scripts/ifcfg-eth0

You should see something similar to the following:

DEVICE=eth0
BOOTPROTO=static
PEERDNS=no
ONBOOT=yes
IPADDR=192.168.4.42
NETMASK=255.255.255.0
GATEWAY=192.168.4.1

Make the necessary changes and write-quite (:wq)

Restart the network service:

# sudo service network restart

Method 2: Using the vMA Setup Wizard

Change the working directory to /opt/vmware/vma/bin/

# cd /opt/vmware/vma/bin/

Run the vmware-vma-netconf.pl script:

# sudo ./vmware-vma-netconf.pl

Follow the on-screen instructions. Job done.

VMware HA (High Availability) admission control is something I wanted to understand better so I started making notes gathered from various sources on the subject, and in particular the way slot sizes are calculated. Duncan Epping’s Yellow Bricks site already covers HA very well and I bow down to his knowledge on the subject, well worth checking out. Also I would strongly recommend VMware vSphere 4.1 HA and DRS Technical Deepdive by Duncan Epping and Frank Denneman which I purchased at Comcol.nl which they shipped to me in the UK in just two days.

That said, I thought I would share my own views and notes I have taken on the subject. The vSphere Availability guide states “A slot is a logical representation of memory and CPU resources. By default, it is sized to satisfy the requirements for any powered-on virtual machine in the cluster.” – In simple terms a slot can be consumed by a single virtual machine, but a virtual machine may consume more than one slot.

When you create an HA enabled cluster, the default admission control policy is set to “Host failures cluster tolerates”. It is important to understand how this works to avoid any ‘schoolboy errors’ or mis-calculations in your design.

Rule # 1: Remember this, “The hostess with the mostess” – the ESX host with the most available slots will be taken out of the equation when calculating HA admission control. If your ESX cluster has 6 hosts with 16GB memory, and one host with 32GB memory then this will be excluded from the calculation.

Calculating the Slot Size (memory and CPU)

Firstly, do any virtual machines have a reservation on the CPU or memory? If so, then the virtual machine with the largest reservation on a given host is taken into account. For virtual machines that have no reservations, the slot size is calculated with the CPU at 256Mhz and memory at the VM overhead (E.g. 123.73MB – See page 28 of vsp_41_resource_mgmt.pdf). Repeat this for the remaining virtual machines on the host (see flowchart).

Flowchart Example

Lets say that we have 10 virtual machines (1 vCPU each) and two of them have reservations as follows:

VM 1 – Has a CPU reservation of 700Mhz, and no memory reservation.
VM 2 – Has a CPU reservation 500Mhz and memory reservation of 1024Mb.
VM 3 to 10 – No reservations on memory or CPU. (keeping this example simple)

The slot size calculation will take the largest reservation for memory and CPU. In this example the result would be:

CPU: 700Mhz
Memory: 1147.73MB (1024Mb plus overhead)

Rule # 2: Remember that the biggest reservation will be used for memory and CPU. If no reservations are set, then it will use 256Mhz for the CPU and 0MB + virtual machine overhead for the memory.

Calculating the Number of Slots Available

Now we have the slot size for our ESX host, the next step is to work out how many slots we have. In order to calculate this, simply divide the CPU and memory available on the host by CPU and memory slot size. For example, using our figures above (700Mhz and 1144.73Mb), say our host has 5934 Mhz CPU available and 7126MB memory, our slot sizes will be 8 for the CPU and 6 for the memory.

5934 /700 = 8
7126 / 1147.73 = 6

Result: Take the smaller of the two, and our ESX host has 6 available slots. Do this for each host in the cluster and you have the total number of slots available in your cluster (minus the host with the most slots).

Rule # 3: It will always use the smallest number of slots from CPU or memory.

What Does This All Mean?

Now we understand how slots are calculated on each of our ESX hosts, it gives us a better insight when determining how your HA will be configured. Here is a simple check-list of questions you should ask yourself:

• Do any of my virtual machines  have CPU or memory reservations set?
• Are all of my ESX hosts of the same specification, or do I have one or two with significantly more CPU or RAM?
• How many host failures do I need to tolerate?

Some possible solutions:

• Don’t use virtual machine reservations unless absolutely necessary, instead use resource pools with reservations as these are not used in the slot size calculation.
• Keep the hosts in your cluster of similar specification (same CPU and memory) otherwise it could be eliminated when calculating the number of slots to use.
• Do you need an HA policy based on a number of host failures?   Use ‘Percentage of cluster resources reserved as failover spare capacity’.

If you are involved in DR for your organisations IT infrastructure and are replicating virtual machine VMFS datastores then you may be familiar with DisallowSnapshotLUN in ESX 3.x. Let’s start with a background on what these advanced settings are and why they are there.

Since virtualization changed the landscape for disaster recovery  some time ago now, most businesses have embraced SAN storage replication for DR (see my other post). This is old news now, but unless your SAN vendor integrates with something like VMware Site Recovery Manager (SRM) then you will have a number of manual tasks involved in your DR recovery process.

When you replicate your datastores (or LUN’s) from site A to site B, you will be using an asynchronous mirror between these two sites. In other words, data blocks will be scheduled (or queued) for replication to site B. If it was synchronous then as data is written to disk, the write operation wouldn’t be acknowledged until it is written to both disks or arrays. Obviously synchronous replication would be far too slow across a WAN to a DR site.

In order to use the replicated LUN in your DR site, you will need to take a snapshot of the replicated LUN then present the snapshot LUN to your VMware ESX hosts in the DR site. Different SAN vendors will have slightly different methods and management capabilities, but the underlying requirement doesn’t change. Without modifying the advanced settings of your ESX 3.x host you will not be able to present your snapshot LUN (VMFS datastore) to the ESX host.

ESX 3.x: What you’ll need to do is set the following parameters:

EnableResignature=0
DisallowSnapshotLUN=0

By switching off DisallowSnapshotLUN, the ESX host will allow your replicated LUN to me added and it won’t re-signature the volume. By default DisallowSnapshotLUN is set to 1. You can find this setting in the Configuration tab of the ESX host > Advanced Settings.

ESX 4.x:

VMware already have a KB article on this here: kb.vmware.com/kb/1011387

It says that you can just use the Add Storage wizard and it will display the VMFS label. If it’s not mounted then you can assume it is a snapshot (or volume copy) and can go ahead and mount it. Using the GUI will force-mount a VMFS volume and also  make it persistent. In fact the advanced settings (EnableResignature and DisallowSnapshotLUN) are no longer present with ESX 4.x.

Useful KB Articles:

kb.vmware.com/kb/1003641 – Cannot access the LUN as it is marked as a deactivated snapshot
kb.vmware.com/kb/6482648 – VMFS Volume Can Be Erroneously Recognized as a Snapshot
kb.vmware.com/kb/1015986 – Force mounting a VMFS datastore residing on a snapshot LUN results in the error: Cannot change the host configuration

The third part of this series continues with the vSphere build on my whitebox server, the Asus Rampage II Extreme with Intel Core i7 2.8Ghz and a 120GB SSD. Following on from the video in part 2 where we installed ESXi on to the USB drive, we are now ready to access the physical ESXi host and start creating some virtual machines.  Since this is a home vSphere lab environment, accessing the lab from anywhere (not just at home) is a major advantage for me, so I’ll be taking you through the steps to create a Microsoft Windows Server 2008 R2 virtual machine with an RD Gateway (Remote Desktop Gateway). We will also need shared storage in order to use vMotion, so I will also guide you through the setup of an OpenFiler iSCSI virtual SAN.

Part 1 | Part 2 | Part 3

To Do List:

If  you have followed part 2 then your physical ESXi host should be up and running and sitting in your garage, shed, spare room (wherever its new home is!). You can now switch on your laptop or desktop PC, get a warm drink and we will get the vSphere lab configured!

Note: I’d also like to mention that whilst I decided to install VMware ESXi 4.1 to a USB drive, there is no reason you can’t install the full ESX 4.1 (with Service Console) to the local SSD drive.

• Download and install the vSphere client
• Creating the core virtual machines:
  • Domain Controller
  • vCenter server
  • RD Gateway
  • OpenFiler SAN
• Finally create virtual ESXi hosts.

vSphere Lab Environment Topology:

A note on licensing

You will also notice that the ESXi server will automatically assign a 60-day evaluation license. Although you can register for a free ESXi license, the free edition will be extremely limited in features and totally unsuitable unless you simply wish to run virtual machines.

I will be assuming that all licenses for your home lab will either be evaluation or purchased. The downside of evaluation licensing of course is that you’ll have to re-install your environment every 60 days.

Download and Install the vSphere Client

1. Navigate to https://<ESXi-IP-ADDRESS> (E.g. https://192.168.4.10)
2. Download vSphere Client and install!

When you launch the vSphere Client, you’ll need to enter the IP address of the ESXi host followed by the username (root) and password. You should have this information following the installation and initial configuration in part 2. You will receive a security warning stating that the SSL certificate is untrusted.

3. Select “Install this certificate and do not display any security warnings” then click Ignore.

Creating the core virtual machines:

I’ve called these the ‘core’ virtual machines as these are the VM’s that make up the home lab and provide the core functionality. I won’t be guiding you through the steps to create virtual machines, as this article is aimed at someone with at least a basic understanding of creating a VM, however, I will be summarising the configuration required.

The following specifications are well below the recommended, however, this is what I have used in my lab environment and it works well for me. All virtual machine hardware is version 7.

I created a domain controller on Windows Server 2008 R2 and added (A) records for CORE-ESX, and ESX01 – 05. Nothing more to this stage.

Build a virtual machine running Windows Server 2008 R2 and join it to your domain. Next, install vCenter with Microsoft SQL 2005 Express. I won’t detail the steps required to install VMware vCenter, just follow the installation defaults.

The RD Gateway server enables me to securely connect over port 443 to any of my machines. It only takes a few minutes to setup, and it gives me the flexibility to connect from anywhere. Read my article on installing and configuring the RD Gateway here.

Installing & Configuring the Openfiler SAN:

The Openfiler SAN is a virtual appliance and can be downloaded from www.openfiler.com. As a virtual appliance you just need to upload it to your datastore and add it to the ESXi server (browse datastore and add to inventory), no configuration of the virtual machine hardware is necessary. For your information here is the configuration of the Openfiler:

I’ll be adding a guide to the Openfiler setup soon, but I’m slacking due to the Christmas and New Year break (that’s my excuse anyway!)

Part 1 | Part 2 | Part 3

Following on from part 1 of my VMware vSphere ‘whitebox’ sever series, this video provides an overview of the build, and installation and configuration of ESXi.

Part 1 | Part 2 | Part 3

Take note that I’m using the ESXi installable ISO downloaded from VMware, as opposed to ESXi embedded.

Continue to Part 3

Introduction

Way back when VMware VI3 was released in 2006 (doesn’t time fly!), I built a home-brew lab server for ESX 3.0 and used it partly to study for my VCP exam. That particular machine is now my home theatre PC (HTPC) as it wouldn’t stand a chance of running VMware vSphere, so here is my mission to build a whitebox VMware vSphere lab server. I must also give credit to Simon Seagrave and Simon Gallagher their vSphere lab server articles which have inspired me to do something about it and build a vSphere lab at home. Simon has lots of great articles on building a vSphere lab, and I urge you to visit his site.

Part 1 | Part 2 | Part 3

vSphere Lab Environment Topology:

Reasons for a home lab?

Throughout my entire IT career I have used equipment at home to help study for certifications and enhance my technical knowledge. As an IT professional you’ll not only need a lab to study for your certifications, but you will need a test-bed to experiment and try things for the first time. There have been occasions as a technical consultant where a fairly niche customer requirement presents itself and is something that requires very specialist skills. Having the flexibility to remotely connect to my home lab and test a procedure or new technology gives you the confidence to carry out that piece of work in front of the customer, even if it’s the first time you’ve done it. We all have to start somewhere, and there is a first time for everything… better that it’s in a test environment than a live production system though!

Ever since I first started in IT back in 1997, I have always had a home server, lab, test-bed or whatever you want to call it. I remember installing NT4 on a home built server in 1997, promoting it to a PDC and then configuring WINS and DHCP. I even had a couple of Cisco 2500 routers so I could emulate a serial connection between two subnets, and at the time I felt that I was at the cutting edge. For the past 5 years, VMware has made it a whole lot easier to create test and development platforms. VMware Workstation is the saviour of many developers and server administrators as it allows them to very easily create virtual servers for testing, experimenting or learning, whether it’s on their laptop or home PC.

The most important factor of having a home-lab for me personally has been the freedom to access a broad range of server technologies that I can do whatever I like, from build to destruction.

Server Choice

I desperately wanted to avoid buying a branded server from HP or Dell, because to get anything near the specification of my whitebox would cost a fortune. Not only that but the ML115 G5 is becoming very hard to get hold of, unless you find a bargain on Ebay. The later ML115 G6 is reported to work, but the B110i storage controller won’t work. I find that these type of servers can also be very noisy and my wife doesn’t understand the novelty of these things when they’re powered on in the spare bedroom!

Officially you would need to buy a server that is listed on the VMware HCL but thanks to VM-Help.com they have put together a whitebox HCL which has the Asus Rampage II motherboard on the whitebox HCL list. Also my intention is to build a system that I can put to other uses in the future such as a gaming PC, should later versions of vSphere not be compatible for any reason.

So for me the decision is clear… my own whitebox vSphere server!

My VMware vSphere ‘Whitebox’ Server Kit List

MOTHERBOARD: Asus Rampage II Extreme Intel X58 1366
CPU: Intel i7 930 2.8GHz Socket 1366 8MB L3 Cache
RAM: 12GB OCZ Gold Low Voltage (6x2GB) DDR3 PC3-16000C10 2000MHz Triple Channel
NETWORK: Intel Pro/1000 MT Dual port 1GbE (the two onboard GbE NIC’s don’t work with vSphere)
SSD: OCZ 120GB Vertex 2E SSD 2.5″ SATA-II Read = 285MB/s, Write = 275MB/s 50,000 IOPS
GRAPHICS: Asus HD 4350 512MB DDR2 DVI VGA HDMI Out PCI-E Low Profile Graphics Card
PSU: Antec TruePower New 650W Modular PSU
OTHER: LiteOn IHAS124-19 24x DVD±RW DL & RAM SATA Optical Drive
CASE: Antec Plusview 1000AMG

Network Switch: Linksys SLM2008
Thanks to Techhead for pointing this out. This is ideal for my home lab environment as it has 8 x 10/100/1000 GbE ports and supports 802.1q VLAN segmentation, and Jumbo Frame support (9KB). It even supports PoE (Power over Ethernet) from the first port, although I doubt I’ll use that. It is an awesome switch and pretty cheap at around £70.

Total Cost: £970 (inc VAT and Postage)*

*Best price I could find for all components listed as of December 2010

To help understand what features my setup will support, I’ve put together this table…

Features & Hardware Requirements

* Fault tolerance would need more than one ESX host in an HA cluster, and a processor that supports virtualisation (VT), however the virtual ESX hosts will not provide VT therefore Fault Tolerance cannot be used.

So here is the case I’m using, it’s an Antec Plusview 1000 AMG which is larger than I would normally use, but as it was just sitting in my garage I decided to put it to good use. It will also sit quite happily in my spare room under the desk, and the only fans I am intending to use in this case are the CPU fan and obviously the built-in PSU fan. Other than those two the graphics card and motherboard are silent which means that my wife won’t get too upset!

In part 2 I’ll show you the hardware build, configuration and then the installation of ESXi 4.1 and vCenter.

Update: I have had issues with the Asus Rampage II Extreme detecting 8GB instead of 12GB memory, see this link for further information.

Part 1 | Part 2 | Part 3

Continue to part 2 (video)

Snapshots are a fantastic way of providing a quick and reliable method of rolling back the state of a virtual machine, should something go astray following an patch or update. VMware VCB also uses virtual machine snapshots to quiesce the VM prior to taking the backup data.

However, in larger environments where there may be tens or hundreds of VMware ESX servers, snapshots can also be a pain in the backside if there is no control over who is using them. Why? Because snapshots work by creating a delta VMDK that records the changes in blocks, a process called copy-on-write (COW). Over time the delta VMDK file will grow, and depending on the level of I/O within the VM it could grow faster on some virtual machines and not others.

The danger only presents itself if the datastore where the VMDK resides reaches it’s capacity. When this happens, virtual machines that are not thin-provisioned should continue to run with no problems, but think about these situations:

1) You have other virtual machines in the same datastore using snapshots.
2) You have one or more virtual machines on thin-provisioned disks.
3) You have powered off virtual machines, that need to be powered on.

In all of the above scenarios if the datastore is full then the affected virtual machines will be suspended (paused). Virtual machines with thick-provisioned disks will continue to operate as the VMDK already has the full allocated of storage space available. Virtual machines that are powered off, and need to be powered back on will fail as they won’t have enough disk space to create the virtual swap file.

The simple rules to follow to avoid these situations is:

1) Allow enough free disk space in each datastore for the snapshots to reside. If snapshots are only ever created during a VCB backup, and the virtual machines have little I/O then you won’t need as much space as busy environments.
2) Monitor the use of snapshots in your environment. I’ve included a command you can run to find snapshots below.
3) Don’t forget about snapshots and leave them in place. They will get bigger!
4) Change the permissions to restrict who can take virtual machine snapshots. The fewer the better, and try to establish some internal procedures to make IT staff aware of the potential issues.

A simple, yet useful command you can run on each ESX server to find snapshots is this:

Login to your ESX server and change the working directory to /vmfs/volumes. Then use the grep command to find vmsn (VMware Snapshot) files.

# cd /vmfs/volumes
# ls -Rh | grep "vmsn"

This can be really useful, especially when you don’t know who is creating snapshots and forgetting they are there!