What is an ESXi cluster, and how do you cluster ESXi servers

ESXi clusters involve a combination of ESXi hosts, VMware services, and vCenter to optimize load balancing, availability, and resource management for virtual machines (VMs). These clusters feature a vCenter server that centralizes the management process to facilitate shared resources that drive higher availability, scalability, and load-balancing capabilities. 

Unpacking the components of ESXi clusters

An ESXi cluster contains a synergy of components responsible for pooling the shared resources required for powering VMs while minimizing downtime. An ESXi cluster typically contains the following core components:

ESXi hosts

These physical servers each host their own ESXi hypervisor, a type-1 hypervisor that contains the various VMs. A host’s resources are utilized only for the VMs and services that are currently running on them. Through additional processes such as vSphere HA and vMotion, virtual machines can be automatically moved between hosts based on optimal resource allocation, ensuring high availability.

ESXi clusters also allow you to cluster your networking and include things like distributed virtual switches, which allows you to set up a single (or multiple) switches that are shared by all of the ESXi hosts. 

Virtual machines

VMs play a critical role in ESXi clusters, serving multiple functions that include consolidating servers, maintaining high availability, and providing disaster recovery under emergency situations. 

Specifically, VMware ESXi runs on top of hardware without requiring manual operating system installations. The process enables VMs to share physical hardware resources like CPU, memory, network, and storage while maintaining isolation. ESXi cluster’s advanced configuration can outperform traditionally hosted hypervisors. These can help enhance overall cost-effectiveness and performance.

A Virtual Center, or vCenter, is the centralized management platform for ESXi hosts, VMs, and services. vCenter is the brain behind everything: it provides all of the tools for enabling and configuring high availability and load balancing. It also allows you to configure shared storage such as setting up iSCSI drives and datastores, and it allows you to configure virtual networks that all ESXi hosts communicate with.

Shared storage

The shared storage component of ESXi clusters enables multiple hosts and serves to access a VM when required. A well-established connection between shared storage and ESXI hosts provides VMs with high availability across various scenarios, which is crucial for complex enterprise infrastructures with multiple users.

Popular examples of shared storage used in ESXi clusters include Network File Systems, Storage Area Networks (SANs), Virtual SANs, and datastores. 

Enterprise managers should choose the most suitable shared storage type according to cost-effectiveness and specific system performance requirements. For example, managers may choose SANs with dedicated networks to support workload-intensive databases or high-traffic web applications.  

Unpacking the advantages of ESXi clusters

Essentially, ESXi clusters optimize complex enterprise systems by providing the technical capabilities of running multiple applications through VMs, overcoming the limitations of a single hosted server. These can establish a wide array of system improvements, mainly through high availability and optimal load balancing. An additional benefit of ESXi clustering is that vCenter provides a centralized location for managing all ESX hosts and VMs.

High availability

ESXi clusters offer failover capabilities by tapping on High Availability (HA) clusters. There are two different HA systems that run in vCenter, but they are not related. The HA that we are interested in is the vSphere HA. vSphere HA provides HA for the VMs and relies on primary and secondary hosts, as well as admission control. 

The most important thing about admission control is that it ensures that there are enough resources reserved for virtual machine recovery in the event of a failure. There are several different policies that can be used in order to determine how the ESXi hosts’ resources are split up.

Optimal load balancing

The distributed load balancer (DRS) utility of ESXi clusters automatically balances host workloads by assessing the resource demands of their VMs. As such, you can expect optimized resource management for maintaining critical enterprise workloads.

Setting up ESXi clusters

The first step involved in setting up ESXi clusters is choosing between scale-out and scale-up strategies. A scale-out configuration involves fewer ESXi hosts per cluster but often ends up with more configured clusters. You can set up a scale-out cluster by adding more nodes to a cluster, which increases performance through a distributed system. 

Alternatively, a scale-up configuration features more hosts but a significantly lower number of overall clusters. You can prepare a scale-up cluster by adding resources like CPU and networks to a single node, improving its performance and capacity.

Once you have decided on the cluster type, you can start creating your layout with the following steps:

1. Use the VMware ESXi installer to upload ESXi on each physical server within your system. 
2. Establish the vCenter Server that serves as the centralized management touchpoint for your clusters.
3. Log into vSphere Web Client to configure the host network arrangements.
4. Create your cluster in the vCenter Server and designate it to a host. You’ll also need to make the necessary configurations for each cluster. These include sorting through functions like admission controls for determining the resources required to run a VM. 
5. Set up the appropriate shared storage type for your clusters based on system needs.
6. Plan your VMs and link them to hosts based on system requirements, such as resource availability.
   

Additionally, it is important to keep ESXi clusters consistently monitored and managed via the vCenter Server. The centralized management platform offers a comprehensive view of your cluster status, through insightful metrics like CPU performance and network latency. You’ll also want to make sure that the vCenter version you plan to install supports the version of the ESX on the hosts. The detailed vCenter Server reports help guide the necessary tasks for keeping systems at top performance. 

Managing ESXi clusters

You can modify an existing ESXi cluster according to your enterprise needs by adding servers or VMs for scaling system performance. These configurations and changes take place via the centralized vCenter Server. 

For adding hosts

You will need to ensure that each host runs a version of ESXi that is supported by the chosen vCenter distribution. Also, they should not feature a manual vSAN configuration or networking configuration. 

You can add an ESXi server by accessing the vSphere Client (Desktop) or vSphere Web Client through the vCenter Server. Select a cluster within the datacenter, where you can add and make configurations to its existing lists of hosts. You’ll have the opportunity to view a host summary before completing the process, which offers important details of added hosts and technical warnings. 

For adding VMs

Similarly, you can add VMs utilizing the vSphere client. Navigate to your target cluster, select New Virtual Machines in the action drop bar, and follow the prompts from the Virtual Machine wizard for configuring the new VM. You will also need to allocate the resource/hardware settings for your VM and the shared storage system. Similar to adding hosts, you’ll receive a detailed summary of your new VM before you proceed to complete the process. A new VM can be created directly in the cluster or can be assigned directly to a certain host. If an ESXi host is part of a cluster, all of its VMs will be part of the cluster. 

The resource pool

Clusters also have an additional level of organization called a resource pool. This is what lets you fine tune how resources are allocated to different groups of VMs. For instance, if there are production critical VMs, we can ensure that they have more CPU/mem allocated than the rest of the VMs.

At LogicMonitor, we help companies transform what’s next to deliver extraordinary employee and customer experiences. Want to learn more? Let’s chat.