Suresh Babu Nekkalapudi (Suresh) is currently a Senior Member of Technical Staff at VMware R&D. Expertise include Virtualization(VMware), Business Continuity, High Availability, Data Protection(Backup/Recovery), Servers & Storage.
Some deep dive into vSphere Replication Architecture for folks interested in Business Continuity and Disaster Recovery
Below screenshot is from VMware vSphere Replication administration guide
vSphere Replication Appliance (VR Appliance): Appliance with 2 integrated components
- vSphere Replication Management Server (VRMS)
- Manages the whole of replication infrastructure
- Maintains mapping between source and target Virtual Machines
- Handles authentication
- Provides management interface to Site Recovery Manager
- Configured with internal DB (vPostgres) or External DB (MS SQL Server/Oracle). All configuration and monitoring information is stored in the DB
- Only one VRMS can exist in a vCenter
- vSphere Replication Server (VRS)
- Component at receiving end that receives replicated data.(Source->Target or vice-versa)
- Upto 9 additional vSphere Replication Servers can be deployed in a vCenter for load balancing needs
- Some of the configuration information from VRMS DB is pushed to VRS embedded MySQL database.
- vSphere Replication Server is available as an add-on component in the form of an appliance
Note – While replicating Virtual Machines from Source->Target, vSphere Replication Server component at Source site is not playing any role. It is the vSphere Replication Server component at Target site that is actually receiving data. If the replication direction is from Target->Source then the vSphere Replication Server at Source will play a role in receiving the replicated data from Target.
VR Agent (vSphere Replication Agent) – It has 2 sub components
- vSphere Replication Service
- Integrated with ESXi hostd agent
- Manages the replication process for virtual machines
- Takes care of replicating virtual machines meta data and storing virtual machine replication configuration
- Importantly it coordinates the schedule to achieve configured RPOs
- vSCSI Filter
- vSCSI Filter driver is part of VMKernel
- Tracks the regions of disks that have changed there by recording that information in memory
- Driver that actually replicates the data from Source site to vSphere Replication Server on Target site
vSphere Replication under the hood
Initial Replication (Initial Full-Sync)
- Administrator configures replication on a VM which is provisioned by VRMS that has plug-in integrated into vSphere Web Client
- An Initial Full-Sync of VM starts which actually creates an empty disk(vmdk) at Target site
- MD5 (Message Digest hash algorithm) compares the Source and Target disk
- As the Target disk is empty, the vSCSI Filter will start preparing data for replication to Target
- If network compression is enabled for VM while configuring replication, FastLZ (an Open Source faster compression algorithm) will take of compressing the data at source before replicating to target (See FastLZ benchmark details – http://fastlz.org/index.html)
- The compression significantly reduces network bandwidth utilization though it costs few CPU cycles at Source and Target for compression and de-compression
- If application quiescing is enabled while configuring replication, VSS Requestor and VSS Provider available as part of VMware tools will work with Microsoft VSS component to take application consistent snapshot for VSS aware applications (Note that VSS application quiescing is only for Windows)
- For application quiescing in Linux, custom scripts specific to the application have to be used.
- Once the replication traffic hits the vSphere Replication Server at receiving end, a CRC (Cyclic Redundancy Check algorithm) is used for error detection.
- MD5 & CRC stated above, together helps for data integrity and consistency of replicated data
- The replicated data from vSphere Replication Server is copied to the target datastore through NFC(Network File Copy) service available with ESXi
- The replicated data is first written to a snapshot file (redo log file) and then commited to base disk once all replicated data is available in redo log file
Subsequent Replication (Delta Sync)
- vSCSI Filter tracks all the changed blocks since last replication
- vSphere Replication Service gives a green signal to go as per RPO setting and vSCSI Filter sends the data to vSphere Replication Server at target site.
- Steps 4a to 8 remains the same
VDP by default has one internal proxy with a throughput of 8 (ie- The internal proxy can run upto max 8 concurrent backups and restores). The throughput of a proxy is configurable (1 to 8) in VDP configure page.
VDP also provides an option to deploy upto 8 external proxies. An external proxy is a light-weight appliance with the following benefits:
- Run more concurrent backups with external proxies
- Perform hot-add backups where VDP appliance do not have access to the protected VMs storage
- FLR on EXT4 based file systems
Note – Once an external proxy is configured, the internal proxy is automatically disabled
To take full advantage of external proxies, deploy them to the hosts that have access to protected VMs storage. The throughput of each external proxy is also 8.
Though from math, (Max external proxies (8) * Throughput of each external proxy (8) = 64) but VDP supports only 24 concurrent backups for various reasons.
Examples:
- For 2 external proxies, set the throughput to 6 >>> 12 concurrent backups
- For 4 external proxies, set the throughput to 5 >>> 20 concurrent backups
- For 8 external proxies, set the throughput to 3 >>> 24 concurrent backups
However the performance recommendations vary for Level 0 & Level 1 backups.
- Level 0 backups – Increase throughput/proxy than the number of proxies
- Level 1 backups – Increase proxies over throughput
Refer VDP Admin guide-Proxies section for complete details.
Deploying an External Proxy
Invoke VDP Configure page – https://<IP_address_VDP_Appliance>:8543/vdp-configure/ and login into the appliance
The default vSphere Replication Appliance deployed can be referred as ‘vSphere Replication Management Server‘
vSphere Replication Management Server includes components
- To maintain mappings between source and replica virtual machines
- To manage authentication
- To receive replicated data (vSphere Replication Server is the integrated component)
- To support SRM
- To support vCloud Air Disaster Recovery
For smaller deployments the default vSphere Replication Appliance should suffice. If the deployment is large, it is recommended to deploy additional vSphere Replication Server for load balancing needs.
Unlike vSphere Replication Appliance(vSphere Replication Management Server), plain vSphere Replication Server is meant only to receive replicated data and no management components are present. No VRMS service runs inside vSphere Replication Server.
In a vCenter environment there can be upto 10 vSphere Replication virtual appliances
- 1 vSphere Replication Appliance (aka vSphere Replication Management Server)
- 9 vSphere Replication Servers (aka vSphere Replication Server Add-on)
Let me walk through the deployment and configuration of additional vSphere Replication Server (add-on)
VMware Validated Designs is a comprehensive set of documentation for specific SDDC deployment architectures and scenarios. The designs are created and validated by VMware experts. The design provides standardized and streamlined architectures for various deployment scenario’s and use-cases. The designs are available as customer facing documentation, VMware Professional Services (PSO) delivery kits and certified partner architectures.
Useful links for your reference >>>
Official Announcement
http://www.vmware.com/software-defined-datacenter/validated-designs.html
VMware Validated Designs (Overview) – Video
https://www.youtube.com/watch?v=HRhbHCLf4qU&feature=youtu.be
Reference Architectures
Foundation Design
http://www.vmware.com/files/pdf/techpaper/vmware-v2d-foundationRA.pdf
Automated Provisioning Design
http://www.vmware.com/files/pdf/techpaper/vmware-v2d-singleregionRA.pdf
Deployment Walkthroughs
https://featurewalkthrough.vmware.com/#!/vmware-validated-designs/deployment-walkthroughs
Operations Walkthroughs
https://featurewalkthrough.vmware.com/#!/vmware-validated-designs/operations-walkthroughs
Join the community
https://my.vmware.com/web/vmware/registration?skipCommunities=true&redirectURL=http://www.vmware.com/betanext/publicbeta.portal%253fbetaProgramCode=V2D
Blog article by Mike Brown from Office of the CTO
https://blogs.vmware.com/cto/introducing-vmware-validated-designs/