Re-configure proxies for VDP – Limit the number of parallel backups

By now, we all know that VDP by default can take backup of 8 VMs in parallel. This is because, VDP internally configures 8 proxies while deployment. User do not really have any control here to limit the number of proxies.

So you have a backup job created to backup 20 VMs, VDP will pick 8 VMs at a time for backup. So at any given time VDP cannot backup more than 8 VMs in parallel.

The maximum no of active proxies is 8.  In any way, it cannot go beyond 8.

But can it be re-configured to 3 ? Well, I would need it to limit my resource consumption. I am running a bit low configuration servers.

Though there is no direct and officially recommended way, I have done a small experiment couple of weeks ago to re-configure the proxies to 3 by running 2 simple steps.

*Note – Before you start, make sure to disable all the backup jobs. You can enable them back once you are done with reconfiguration. Ideally I would prefer doing it in backup window as I have control on it.

Steps –

•  SSH to the VDP appliance
• cd /usr/local/avamarclient/etc/
• ./registerproxy.sh
  • === Client Registration and Activation ===This script will register and activate the client with the Administrator server.Enter the Avamar server domain [clients]: <Press ENTER here>Number of proxy clients (1 to 8) [8]: 3 <Enter the number of proxies to limit. Max is 8>

You are done now. The appliance will take of the rest.

Output from the script for your reference>>>>>>>>>>

SCRIPT : /usr/local/avamarclient/etc/avagent.d
PARAMETERS : register 127.0.0.1 clients

Avamar avagent configuration:

Avamar home : /usr/local/avamarclient
avagent : /usr/local/avamarclient/bin/avagent-vmware.bin
list of agents : /usr/local/avamarclient/etc/avagent-list
common options : /usr/local/avamarclient/etc/avagent-flags
library path : /usr/local/avamarclient/lib:/usr/lib/vmware-vix-disklib/lib64 :

*** vApp .pin missing ***
Setting number of instances to 0.

Number of vapp proxies : 0
Number of file proxies : 3
Number of image proxies : 3

Configured agents:

name vardir pin
—- —— —
proxy-1 var-proxy-1 vmwfile-linux.pin,vmwfile-windows.pin,vc bimage-linux.pin,vcbimage-windows.pin,vdrmigration.pin,nwcomm.pin,unix.pin
proxy-2 var-proxy-2 vmwfile-linux.pin,vmwfile-windows.pin,vc bimage-linux.pin,vcbimage-windows.pin,vdrmigration.pin,nwcomm.pin
proxy-3 var-proxy-3 vmwfile-linux.pin,vmwfile-windows.pin,vc bimage-linux.pin,vcbimage-windows.pin,vdrmigration.pin,nwcomm.pin
proxy-4 var-proxy-4 (none)
proxy-5 var-proxy-5 (none)
proxy-6 var-proxy-6 (none)
proxy-7 var-proxy-7 (none)
proxy-8 var-proxy-8 (none)

…. all avagents stopped

Common agent settings:

–informationals
–sysdir=/usr/local/avamarclient/etc
–bindir=/usr/local/avamarclient/bin
–flagfile=”/usr/local/avamarclient/var/avagentAll.cmd”
–mcsaddr=127.0.0.1
–dpndomain=clients

avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-1/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-1/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-2/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-2/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-3/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-3/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-4/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-4/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-5/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-5/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-6/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-6/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-7/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-7/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-8/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-8/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
…. registered all agents!

…. Check the MCS status – Tue Nov 19 23:38:56 PST 2013
Identity added: /home/dpn/.ssh/dpnid (/home/dpn/.ssh/dpnid)
dpnctl: INFO: MCS status: up.
…. MCS appears to be ready.

avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-1/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-1/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5417>: daemonized as process id 13180
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-2/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-2/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5417>: daemonized as process id 13186
avagent Info <5008>: Logging to /usr/local/avamarclient/var-proxy-3/avagent.log
avagent Info <5174>: – Reading /usr/local/avamarclient/var-proxy-3/avagent.cmd
avagent Info <5174>: – Reading /usr/local/avamarclient/etc/avagent-flags
avagent Info <5417>: daemonized as process id 13192
…. agents started.

Registration Complete.

I enabled my backup job to run ‘Backup Now’. I could see my VDP appliance now picking up only 3 VMs at a time for backup instead of 8.

Running my VDP appliance without any issues since couple of weeks.

Emergency Restore – VDP 5.5 (Case Study)

Characters : TOM & JERRY ( TOM helps JERRY here…and *NO FUN*, all SERIOUS) 

JERRY:

• My vCenter is crashed and unable to start/connect
• I also have a crash on one of my production VM.
• The VM is business critical and need to be restored immediately. Of course, I have protected it through VDP 5.5
• But VDP is tightly integrated with vCenter. Will I be able to restore my production VM, even when vCenter is down ?

TOM: Yes, It is possible to restore your production VM without vCenter, with the cool new feature called ‘Emergency Restore‘ in VDP 5.5

JERRY: Thats great!. Where do i start now ?

TOM:

• First login into vdp-configure page of the VDP appliance (https://<ipaddress-of-VDP-appliance>:8543/vdp-configure/)
• Click on ‘Emergency Restore’ tab and check if your production VM is listed

JERRY:

• Yes, I could see my production VM listed from one of the 6 VDP appliances in my environment
• Where can i restore the VM to ?

TOM:

• You can restore the VM only to the ESXi host on which VDP is residing
• Make sure of these pre-requisites now
  • DNS of VDP should be able to resolve its hosting ESXi host name
  • The ESXi host on which VDP is hosted should be disassociated from the vCenter, by connecting to the host directly through vSphere client -> Summary -> Host Management -> ‘”Disassociate Host from vCenter“

JERRY: I have no track of the ESXi host where my VDP resides. Also i have DRS enabled on the cluster, so VDP might be on any of my 32 hosts. Can’t VDP identify it automatically ?

TOM:

• Currently VDP cannot identify its hosting ESXi automatically. I have requested a feature for this.
• You have no option left other than connecting to each host directly through vSphere client and then disassociate the ESXi host on which VDP resides.

JERRY:

• With a bit of pain, I could identify the specific ESXi host and disassociate it from the vCenter. Worth it, if I can recover my VM successfully from VDP

TOM:

• Sure, I can help you with that. Follow the steps as mentioned below.
• Goto VDP configuration Utility(https://<ipaddress-of-VDP-appliance>:8543/vdp-configure/) -> Emergency Restore tab.
• Select the VM and ‘Restore-point’ you would like to restore. Click ‘Restore’, which allows you to enter the ESXi host details on which VDP resides as in screen below
• 
• In the above screen, it would have been nice if VDP could detect the ESXi host automatically. (Feature Requested)
• You may notice an error like this if the ESXi host is not disassociated from the vCenter (below screen)
• 
• In the next step, you need to provide a name for the VM being restored and the destination datastore. If you are in this screen, means that VDP successfully resolved its residing ESXi host to proceed further
• 
• Now restore will be started and you can monitor the status in the loading bar (screens below)
• 
• 

JERRY: Perfect ! Glad that VDP provides such a nice feature called ‘Emergency Restore’. (Restore a backed up VM even in case if vCenter is down).  KUDOS !!!!!!!!!

VDP(vSphere Data Protection) 5.5 – Deploy & Configure – Quick Start

• Deploy the OVA appliance from Web-Client (Unlike VDP 5.1, you will not see appliance per capacity. In VDP 5.5, you will have only 1 appliance which can be configured for 0.5TB,1TB & 2TB in vdp-configure wizard)
• Note – You would still need a IPv4 static IP for VDP 5.5 with proper DNS Name assigned (Check forward and reverse lookup before using the IP for deployment)
• Once the OVA is deployed, you can notice the appliance settings as below.

• Once the appliance is powered ON, you should be able to access the vdp-configure page unless you have any network issues.
• https://<IP of the appliance>:8543/vdp-configure/ in the browser
  • Default User – root
  • Default Password – changeme
• Self Explanatory (Network Settings)

• Self Explanatory (Time zone)

• Self Explanatory (VDP Credentials)

• vCenter registration : For vCenter username, do *NOT* use the format User@Domain, instead use Domain\User  in case you are specifying a domain. (Using @ format will create issues while generating VDP related tasks in web client). Also note that, the user should be provided administrator role)

• Create Storage: This is newly introduced in VDP 5.5 (You have an option to choose the capacity here OR attach backup disks from another VDP appliance)

• Device Allocation: This is new as well in VDP 5.5 (Unlike VDP 5.1, you have an option now to choose datastores for each of the data disk OR you can choose to store with the appliance as well)
  • 0.5 TB (3 disks of 256 GB each)
  • 1 TB (6 disks of 256 GB each) & 2TB (12 disks of 256 GB each)
  • You have the option to choose the number of storage disks for each datastore

• Ready to Complete (Performance Analysis – Optional): This is again new in VDP 5.5. (You have an option to run performance analysis. If chosen to run,  VDP will execute (Write, Read & Seek) tests onto the above selected datastores and reports information about the write/read/seek throughput and whether the datastores met the minimal performance recommendations of VDP)
  • First starts with creating/configuring disks on the datastore

• Secondly, runs the performance analysis

• Once done, restart the appliance. You should be able to monitor the progress of the VDP appliance configuration from the web client

• Once the configuration is completed, re-login into the web client and you should see the VDP plug-in (vSphere Data Protection 5.5) in the web client. (To avoid common connection issues, make sure the VDP appliance time is in sync with the VM/Host time on which SSO is deployed)
• Due some technical reasons,VDP 5.5 create’s separate plugin (vSphere Data Protection 5.5) in the web-client without merging to older 5.1 plugin (vSphere Data Protection). Older VDP 5.1 appliance’s can be accessed using the old plugin(vSphere Data Protection), and VDP 5.5 appliances using new plugin(vSphere Data Protection 5.5).