Home labs used to test virtualization solutions are very often limited by physical resources which either require to size down test environments or to accept a general slowness. This is particularly painful with pseudo-converged infrastructures or nested ESXi’s. Regarding the latter, my opinion is that it has a great drawback with the use of storage at 1/10 of available IOPS in comparison to a VM installed directly on a physical ESXi host.
This frequently results in a situation where a simple operation on a VM such as deployment from a template or via vRealize Automation, a DR test of a Protection Group with Site Recovery Manager or simple SDRS operations were taking dozens of minutes, thus blocking other operations on a nested cluster.
Since I was under pressure to provide some documentation for a project, I promised myself that in a free time I will look for a solution to tackle the issue I described above. More recently, I decided to look for small Linux distros that would fulfil the following conditions:
- be very small in size (in my particular case the IOPS was the limit)
- use a small amount of physical memory
- have VMware Tools or equivalent enabled
Initially, I found that there is no perfect solution to satisfy all of the aforementioned requirements. As a workaround I decided to make a lightweight VM myself only to land with an Ubuntu 14.04 with open-vm-tools, 192MB RAM and 5GB HDD (slimmed down to 2GB thanks to thin-provisioning). That was kind of a progress but was still fairly big in terms of storage.
Fortunately, many homelab and virtualization blogs mentioned Tinycore (here, here, here, here). VMware communities also provided a richness of information about this distro (here, here, here). Personally, until now I used to have some rather unsuccessful experience with TC but I decided to give it a chance (again). TC is a very interesting case of a lightweight Linux distro designed to run pretty fast even on a very old hardware. Alternatively, it is used for appliances due to its architecture which implemented busybox to make OS non-persistent across reboots.
I dig deeper in order to determine if it would be a right choice, especially with regards to Tools installation. The problem is that although creating a VM and making configuration persistent wasn’t much of a fuss, getting VMware tools working on top of it was a completely different story. My plan was to build a VM that would satisfy following scenarios:
a) Have a smallest footprint possible with regards to RAM and HDD
b) Have VMware Tools or Open VM Tools (which are officially recognized by VMware as an equivalent of VMware Tools) installed
c) Provide very basic services such as SSH server and HTTP server (in order to play around connectivity scenarios such as NSX or SRM)
A rather logical decision was to use the most up to date release of TinyCore 6.4 (based on Linux kernel 3.16) but after around 10 different iterations I landed with a VM that required around 116MB of RAM to be turned on successfully. Moreover, the stability was not perfect because sometimes during the boot process there was no memory left which forced TC to start killing off processes in order to avoid a kernel panic (which it eventually did anyway).
I was not discouraged by this and instead of using the newest distro I went back to earlier releases 3.x, 4.x and 5.x. I decided to install it from scratch through the cli via a core ISO (9MB in size) to make sure that only essential services are installed. If you want to know the step-by-step procedure to do it on your own, click here. I compiled open-vm-tools version 9.10 on a separate VM with an identical distro to leave all the rubbish behind and just obtain a clean .tcz extension with dependencies.
Eventually, I found that an initial concept developed itself into a small project which resulted with an extensively tweaked distro. I called it yVM, whereby the prefix is borrowed from -yocto measure base to highlight its small size. It is based on TinyCore 5.4 (kernel 3.8.13) and has the following characteristics:
- 48MB RAM,
- 64MB HDD, divided into two partitions:
- one 32MB /dev/sda1 ext3-formatted partition
- one 32MB /dev/sda5 swap partition
- open-vm-tools installed as per instructions from Lapawa + some manual tweaking;
- openssh, nano and basic nginx installation (http only);
- available usernames: tc and root;
- password for both accounts: VMware1!.
Some remarks about the above release:
- The interface is CLI-only. I consider it to be destined purely for testing purposes in VMware-based environment without much interaction with the the OS itself (apart from connectivity tests i.e. ssh and http);
- I removed all unnecessary peripherals such as floppy drive etc.;
- In order to maintain backward compatibility with previous VMware vSphere environments, the vmx-hardware level is set to 8 (which is vSphere 5.0 compatible).
All in all, the VM has a size of 19MB when thin-provisioned, it still retains around 6MB of free RAM and 12MB of free HDD. The IP is obtained dynamically through DHCP, however that can be changed if necessary. It was tested succesfully in following scenarios:
- Graceful shutdown, SDRS, vMotion and HA on vSphere 6.0;
- Protection Group test, failover and failback with SRM 5.8;
- VXLAN overlay and layer-2/3 connectivity via Edge Gateway with NSX 6.2;
- Replication with VRM 5.8.
Soon I am planning to test it against VMware FT, as well as vROPs and VeeamONE. In my spare time I will also pre-package it into an OpenStack- and Docker-compatible image.
If you are interested in trying it out, feel free to download an OVA file by clicking here. (Redirection to Google Drive, MD5 checksum: cff70e1fdcc4ef3307ff364e2f93c775).
If you want to build your own small VM, plase read this post.