Category Archives: Lab

SDDC Lab, Home Lab

Home Lab vSAN

New Home Lab – vSAN 6.6 Ready

More details coming up. For now, screenshot first.

Home Lab vSAN



vSAN AF Performance on a 1Gbps network.


Some other performance testing before setting up the vSAN cluster.

Single Node vSAN on Supermicro E300
Cache Tier: Samsung NVMe SSD 960 Evo M.2
Capacity Tier: Samsung Evo 1TB SSD
SM E300 SIngle Node vSAN MBpsSM E300 Single Node vSAN IOPS

Samsung Evo 1TB M.2 convert to SATA on Supermicro E300
SM E300 Evo 1TB M.2 to SATA MBpsSM E300 Evo 1TB M.2 to SATA IOPS

Samsung NVMe SSD 960 Evo M.2 240GB on Supermicro E300
SM E300 NovSAN Samsung 960 NvME Mbps
SM E300 NovSAN Samsung 960 NvME IOPS


Compare with NUC Performance also running NVMe 500GB

Also did some comparison when running NFS over 1500 vs 9000 MTU
*Cannot remember what were the disks. 

Install ESXi6.5 on SM300 Fail2

ESXi 6.5 fail to install on mSATA SSD on Supermicro E300 / E200


Was pretty disappointed when I found out I am not able to install ESXi 6.5 on my SM E300 on mSATA SSD, which means no vSAN 6.6 goodness for me. I purposely purchased more SM to run a 4 node vSAN cluster so replace my unstable NFS on a single VM, single SSD. I also took the chance not to use USB for the boot drive because I have been burnt many times with USB. Either the USB drives gone missing or just turn bad. I have decided to use SSD like SATADOM. But SATADOM is too expensive and when I explore further, the SM E300 has a mSATA port and since its unused, perfect. So I went to purchased 16GB mSATA SSD to be boot drives. ESXi 6.0 install went fine but had issues with ESXi 6.5. Screenshot below. After days of troubleshooting, finally found the solution to it and the issue was related to the vmw-ahci driver that ESXi 6.5 is using.

Screenshot showing ESXi 6.5 install failed. The error is something related to SSD not found. Install ESXi6.5 on SM300 Fail2


Installer was able to show all the SSDs but after selecting the 16GB, the install failed.
Install ESXi6.5 on SM300 Fail1


Below are some screen shots, showing the troubleshooting.


This was after a reboot after installing ESXi 6.5 on a previous installed ESXi 6.0. After the reboot, you can see the SSD drives were missing.
SM300 ESXi6.5 Fail


I followed Anthony blog on storage performance issue to try disable the vmw-ahci.
Screen Shot 2018-01-20 at 11.05.22 AM

Even that, it still doesn’t work, after reboot, the box boot back to 6.0U2. I have to go to the altboobank and edit the boot.cfg and remove the vmw_ahci.v00. I also edit the bootstate from value 3 to 1.
Screen Shot 2018-01-20 at 11.12.19 AM


After disabling and reboot, you can see the drives are showing in ESXi 6.5.
Screen Shot 2018-01-20 at 11.08.01 AM




Screen Shot 2018-01-20 at 10.25.03 AM

Create a custom ESXi 6.5 ISO for Supermicro E300 / E200 install

For those Supermicro E200 or E300 users, if you have tried installing ESXi 6.5 on a SSD which is connected to the AHCI controller, you might not able to install successfully. Therefore, you might need to remove the conflicting AHCI driver. The below steps are used to create a custom ESXi 6.5U1 ISO with the removed vmw-ahci driver.

Add-EsxSoftwareDepot “”

New-EsxImageProfile -CloneProfile “ESXi-6.5.0-20170702001-standard” -name “ESXi-6.5.0-u1-SM300″ -Vendor “”

Remove-EsxSoftwarePackage -ImageProfile “ESXi-6.5.0-u1-SM300″ -SoftwarePackage “vmw_ahci”

Add-EsxSoftwareDepot “C:\VMware\vSphere\ESXi\”

Add-EsxSoftwarePackage -ImageProfile “ESXi-6.5.0-u1-SM300″ -SoftwarePackage “net-ixgbe”

Export-ESXImageProfile -NoSignatureCheck -ImageProfile “ESXi-6.5.0-u1-SM300″ -ExportToISO -filepath C:\VMware\vSphere\ESXi\ESXi-6.5.0-u1-SM300.iso

Screen Shot 2018-01-20 at 10.25.03 AM


Screen Shot 2018-01-20 at 10.24.33 AM


Screen Shot 2017-12-19 at 9.40.18 PM

NSX Test App

A few of you have watched my youtube videos such as VIC, SRM with NSX and requested for the NSX Test App that I used for my demos. Alright, I thought why not share to the community the simple PHP Web application that I wrote. I also written a Lucky Draw App for my demo use which I use at large audience events, to have a little engagement with the audience. I will also share with the community, maybe the next time round.

The NSX test App is basically a PHP file. You can download the PHP file from Once you downloaded, you can use it with for example Apache+PHP or a LAMP. Usually its place in the /var/www folder.

If you open up the PHP file, you will see the servername which is the IP address of the DB VM. I will export the DB into an OVA.       Screen Shot 2017-12-19 at 9.28.02 PM

You can either use the database server in the LAMP or use another VM to be the database. In my demos, I usually use a separate VM as the DB server so mimic a real world scenario, like a 2 or 3 Tier Application. So save you the trouble, I export the whole DB as ova so you can easily import it.

This is the DB OVA. You can download from here

It will look like it if you get it work.

Screen Shot 2017-12-19 at 9.30.30 PM


You watch the VIC with NSX video here which utilising the web application.

Screen Shot 2017-11-01 at 2.11.32 PM

virtuallyGhetto Automated NSX-T 2.0 Lab Deployment Gotchas

I was very excited when William Lam develop an automated NSX-T 2.0 Lab deployment and I always wanted to try it out. I had tried to install NSX-T 2.0 manually couple of weeks back and I know its quite painful to install it so since there is an automated way of doing it, its going to save me lotsa time. However, its not so straight forward to use William script whole sale and below I am going to list a few things to be taken note of. These were what I have experienced over the last 3 days of troubleshooting.


A few things to note here

  • I was using PowerCLI 6.3.0. PowerCLI need to be updated due to the new cmdlets for NSX-T. Since 6.5.1 onwards, the way to upgrade the powercli is very different. I followed the instructions here. However, I was having some issues on the Install-Module. Went some googling and finally found out that you will need to install PackageManagement_x64.exe
  • Once the package management powershell modules are installed, you can continue with the instructions such as Install-Module -Name VMware.PowerCLI
  • With the latest version of PowerCLI, we can run the script. The script requires you to have a vcenter, so therefore since my host is a standalone esxi, I need to install a vCenter. I chosen 6.5U2 VCSA and I use the cli script to install.
  • First time using the script, I used VSS for the network for the new VCSA and NSX-T Manager, since like the cmdlets are not able to find the virtual portgroup. Once I changed to VDS, the script works.
    $VMCluster = "Primp-Cluster"
    $VirtualSwitchType = "VDS" # VSS or VDS
    $VMNetwork = "dv-access333-dev"
    $VMDatastore = "himalaya-local-SATA-dc3500-1"
    $VMNetmask = ""
    $VMGateway = ""
    $VMDNS = ""
    $VMNTP = ""
    $VMPassword = "VMware1!"
    $VMDomain = ""
    $VMSyslog = ""
    # Applicable to Nested ESXi only
    $VMSSH = "true"
    $VMVMFS = "false"
  • Some network issues such as reverse DNS not being set up. Undecided to use existing management subnet or create another new subnet and since the ESXi host is hosting some other VMs, I had the script trying to connect to existing vcenters and hosts. Lesson Learnt: try not to use the same IPs as the existing Lab setup to prevent any confusion.
  • Although my vCenter is 6.5U2 but my ESXi host is still in 6.0U2. When running the portion of the script where is using PutUsbScanCodes()I got an error of input argument. I suspect it got to do with the ESXi host, and therefore I went to try the PutUsbScanCodes() on a recent 6.5U2 vCenter and 6.5U2 host using the script here, and true enough the PubUsbScanCodes() works perfectly. So afterwhich, I went back to update the 6.0U2 host to 6.5U2 and confirm that the PutUsbScanCodes() works correctly.
Screen Shot 2017-07-02 at 5.14.20 PM

Powershell script to customise drivers into ESXi

The Supermicro E300 require the igxbe drivers for their 10GE NICs as the standard ESXi ISO does not natively support. Therefore I’m require to custom build the ESXi ISO.

The igxbe 4.5.1 drivers were from Paul Blog –

Download here.

The powershell script were from here. 

These are the commands used.

PowerCLI C:\> C:\Users\Administrator\Downloads\ESXi-Customizer-PS-v2.5.ps1 -izip C:\Users\Administrator\Downloads\ -pkgDir E:\pkg

PowerCLI C:\> C:\Users\Administrator\Downloads\ESXi-Customizer-PS-v2.5.ps1 -izip C:\Users\Administrator\Downlods\ -pkgDir E:\pkg

Screen Shot 2017-07-02 at 5.14.20 PM

You must be wondering why can’t I just update the drivers after installation. I wanted to PXE boot for ESXi installer and somehow or rather the NIC on the E300 that support PXE boot were the 10GE NICs. That was the reason why I have to custom build the igxbe driver into the ISO.

vSAN 6.5 Lab1

vSAN 6.5 ROBO All Flash – 2-node Direct Connect Testing

vSAN 6.5 ROBO All Flash – 2-node Direct Connect Testing

Screen Shot 2017-01-18 at 12.05.30 PM

vSAN 6.5 now supports using a low cost Cat6 or SFP+ TwinAX cables for cross-connect the 2 hosts. I had setup the lab to perform this ROBO setup. vSAN 6.5 also supports iSCSI which I plan to test it as well.

vSAN 6.5 Lab1


I plan to install vCenter 6.5 on these two nodes as well and therefore need to bootstrap the installation. I follow the instruction from William Lam blog – How to bootstrap vCenter Server onto a single VSAN node Part 1.

I did not want to go through the click through installation and therefore follow again his blog to install the vCenter using ovftool CLI method.

Had some issues with using the ovftool at the moment. Will have to update this blog again.

25 June 2017 Update

Finally got some time to perform some testing. I have been using these nodes to run nested ESXi for NSX Multi-site Lab. They are configured as individual standalone vSAN. Here are the specs to the boxes.

Supermicro E300 – SYS-E300-8D
Intel Xeon CPU D-1518 @ 2.2Ghz (4 cores per socket)
64GB of RAM
Cache: Sandisk SDSSDA240G 240GB
Capacity:  Samsung SSD 850 Evo M.2 1TB
2 x 1GE : Intel I210
4 x 1GE: Intel I350
2 x 10GE : Intel X552

ESXi 6.5 Build 4564106

Screen Shot 2017-06-25 at 5.55.51 PM Screen Shot 2017-06-25 at 5.58.28 PM

Screen Shot 2017-06-25 at 6.04.37 PM

Screen Shot 2017-06-25 at 6.04.52 PM

Some performance testing…

Screen Shot 2017-06-25 at 5.50.24 PMScreen Shot 2017-06-25 at 5.51.20 PM




SDDC suitcase1

NSX / SDDC in a suitcase

SDDC suitcase1

Sometime last year around September 2016, I had an idea of setting up a mobile NSX/SDDC Demo Lab that is compact enough to fit into a suitcase so that I can bring them around. However, I didn’t have the time to set them up. Guess what? Another colleague of mine in VMware US that works for the federal agencies set up something that I had envisage.


You can read more about it here. We basically had the same model of servers ie. the Supermicro E300.