vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
vPro-1085-R Course for RHV/oVirt/OLVM Implementation & Administration Lab Exercises
Credentials and access details
Attribute Value
Download URL http://10.40.0.253/lab-materials/vprotect/vpro-1085
vProtect host 10.41.0.4
vProtect username root
vProtect password St0rL@bs
vProtect Web UI https://10.41.0.4
vProtect Web UI username admin
vProtect Web UI password vPr0tect
RHV manager UI https://rhv-m.storware.lab/ovirt-engine
RHV user admin@internal in vProtect in UI, and admin in RHV manager UI
RHV password St0rL@bs
Lab 1 - Demo of all-in-one installation
In this section we'll show you how to install vProtect components quickly using all-in-one setup scripts. Before installation steps please update and then reboot system
dnf -y update
Remote repository (option 1)
1. Export VPROTECT_REPO variable to point to the repository URL
export VPROTECT_REPO=http://10.40.0.253/vprotect/current/el8
2. Execute script:
bash < <(curl -s http://repo.storware.eu/vprotect/vprotect-local-install.sh)
1 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
Lab 2 - Installation with RPMs
In this section you're going to install vProtect using RPMs - so that all necessary steps are done
Prerequisites
1. Access vlab.vpro.proxy.v3
2. Open putty on your vlab.vpro.proxy.v3
3. Connect to vProtect machine with a root access
4. Use your CentOS 8 minimal
5. Make sure your OS is up to date:
dnf -y update
If kernel is updated, then You need to reboot your operating system.
6. Install vProtect repository - create file /etc/yum.repos.d/vProtect.repo:
# vProtect Enterprise backup solution for virtual environments repository [vprotect] name = vProtect baseurl = http://10.40.0.253/vprotect/current/el8 gpgcheck=0
7. Install MariaDB repository (vProtect Server host only)
usually you generate .repo file at MariaDB download site
copy and paste generated repo file into /etc/yum.repos.d/MariaDB.repo, so it looks similar to this (if the repo file doesn't exist):
# MariaDB 10.4 CentOS repository list - created 2020-05-11 09:20 UTC # http://downloads.mariadb.org/mariadb/repositories/ [mariadb] name = MariaDB baseurl = http://DNF.mariadb.org/10.4/centos8-amd64 module_hotfixes=1 gpgkey=https://yum.mariadb.org/RPM-GPG-KEY-MariaDB gpgcheck=1
vProtect Server installation
2 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
vProtect consists of server (central management point with WebUI) and one or multiple nodes (which can be installed on the same host as server or on other machines). First step is always to install server.
1. Install vprotect-server using DNF:
dnf -y install vprotect-server
2. Setup DB for vProtect:
vprotect-server-configure
Please provide a MariaDB password
3. Start vProtect Server (it can take around a minute for server to be started):
systemctl start vprotect-server
4. Open 8181 port on your firewall. Here is example:
firewall-cmd --add-port=8181/tcp --permanent firewall-cmd --complete-reload
5. For this lab we'll open plain HTTP port 8080 as well:
firewall-cmd --add-port=8080/tcp --permanent firewall-cmd --complete-reload
6. By default, vProtect console running on HTTPS port (8181), there is a script in /opt/vprotect/scripts directory to setup forwarding from port 8181 to 443 for you:
cd /opt/vprotect/scripts ./ssl_port_forwarding_firewall-cmd.sh
1. Now you should be able to log into the web console using URL: https://VPROTECT_HOST, where VPROTECT_HOST is hostname or IP of your vProtect Server
by default vProtect has one admin account - admin with password vPr0tect (with zero)
Attribute Value
3 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
Attribute Value
vProtect Web UI https://10.41.0.4
vProtect Web UI username admin
vProtect Web UI password vPr0tect
vProtect Node installation
vProtect Node is component that executes all tasks. It can be installed together with Server (it is common to have 1 server and just 1 node). More nodes can be always added later.
Local node
1. Create a directory mkdir /vprotect_data - will be later used as a mount point for staging space
mkdir /vprotect_data
2. Install vprotect-node using DNF:
dnf -y install vprotect-node
3. Register node with NODE_NAME of your choice ADMIN_USER user name which you would like to use and URL to vProtect API and provide password when prompted:
Syntax:
vprotect node -r NODE_NAME ADMIN_USER http(s)://VPROTECT_SERVER:PORT/api
In our case we'll use HTTP to make this lab simpler
local node (installed to together with server is going to be used RHV) - over HTTP (port 8080):
vprotect node -r node-local admin http://localhost:8080/api
use an admin password - vPr0tect
4. Start vProtect Node:
systemctl start vprotect-node
4 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
Now you should be able to see new entry in Node section of vProtect Web UI with your node in RUNNING state.
5. Run script to configure OS for Node:
vprotect-node-configure
6. Reboot vProtect VM to apply OS-specific settings:
reboot
vProtect is installed
Lab 3 - Initial configuration
1. Upload your license key:
download it first to your proxy machine desktop from the location provided during labs log in to the vProtect Web UI and go to the Settings -> License and upload your license.key file
2. Run from vProtect Web UI Dashboard -> Configuration wizard:
3. Define RHV manager - oVirt (v4) / RHV (v4) / Oracle Linux VM (credentials provided during classes)
Attribute Value
RHV manager UI https://rhv-m.storware.lab/ovirt-engine/api
RHV user admin@internal
RHV password St0rL@bs select DISK_IMAGE_TRANSFER strategy confirm to initiate inventory synchronization when asked
4. Create File System backup destination:
name: FS with deduplication retention: defaults Enable deduplication: deduplication device: /dev/sdb enable Mount deduplicated file system... Deduplicated file system mount point: /vprotect_data (we want this file system to act also as a staging space) on the next screen set Storage path to /vprotect_data/backups - backup destination still must have different path from than staging (even when using same file system for both)
5 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
leave Encryption and Pre/Post access commands - DISABLED set it as a default on the last subscreen and Create
5. In SSH console - verify your backup destination creation has completed successfully with the:
df command to check if it is mounted in /vprotect_data directory vdostats to see if VDO device has been initialized and is ready
6. Go back to the wizard
7. Skip VM backup schedule section
8. Skip VM backup policy section
9. Configure daily vProtect DB backups (schedule and policy) using node and backup destination that have been created
10. Confirm to initiate vProtect DB backup when asked
11. Now go to Users section - to set admin accounts:
make sure to set the correct time zone for your user - default admin account has UTC by default.
12. Create another admin account:
click Create in Users section provide username, first name, last name, time zone and language click Save button
Lab 4 - Managing node configuration
1. Log in to vProtect Web UI
2. Go to Nodes -> Node Configurations tab
3. Create a new configuration
provide a name of your choice change maximum number of export threads to 2 in Task tab assign your FS with deduplication backup destination to the node config
4. Update node and change node configuration to the one you created (go back to nodes tab)
Lab 5 - Dell EMC DataDomain as a backup destination
1. Prepare your PowerProtect DD as a backup destination:
Attribute Value
DD URL https://10.41.0.51/ddem
DD username sysadmin
DD password St0rL@bs
6 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
login to PowerProtect DD and verify you have Storage Unit called vprotectbackup
go to data management -> MTree on the left menu
download BoostFS RPM from URL provided during labs
curl -O http://10.40.0.253:8181/vprotect/current/lab- materials/DDBoostFS-7.0.0.0-633922.rhel.x86_64.rpm
install BoostFS:
rpm -ivh DDBoostFS-7.0.0.0-633922.rhel.x86_64.rpm
create mount point directory
mkdir /dd-backups
save password for BoostFS (DD_BOOST_IP should be replaced with IP of your DD):
/opt/emc/boostfs/bin/boostfs lockbox set -d DD_BOOST_IP -u vprotect -s vprotectbackup
provide password: St0rL@bs
add /etc/fstab entry:
DD_BOOST_IP:/vprotectbackup /dd-backups boostfs defaults,_netdev,bfsopt(allow-others=true) 0 0
Notice that for manual, one-time mount you can run this command:
/opt/emc/boostfs/bin/boostfs mount -o allow-others=true -d DD_BOOST_IP -s vprotectbackup /dd-backups
mount
mount -a
7 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
set ownership to vprotect user on directory /dd-backups:
chown vprotect:vprotect -R /dd-backups
confirm with df that your /dd-backups create backup subdirectories that we will need in the next steps
mkdir /dd-backups/short-term mkdir /dd-backups/long-term
1. Log in to vProtect Web UI
2. Go to Backup Destinations tab
3. Create a new File System backup destination - in the details:
provide name: short-term destination specify retention days for full and incremental backups = 14 days specify retention versions for full backups = 2 specify retention versions for incremental backups = 7 set path as /dd-backups/short-term assign this node config that you have created in your previous lab explore other options in the settings
4. Create a new File System backup destination - in the details:
provide name: long-term destination specify retention days for full and incremental backups = 365 days specify retention versions for full backups = 999 specify retention versions for incremental backups = 999 set path as /dd-backups/long-term assign this node config that you have created in your previous lab explore other options in the settings
5. Go to Settings -> Internal DB backup -> Overview (click on details icon)
6. Run backup manually (top right corner)
specify your new backup destination as a target and proceed with backup
7. Verify that backup has been successfully stored
refresh backup list in the view when completed
8. Check backup details in Backups tab and show its details (notice backup files listed)
9. Explore contents of the /dd-backups/short-term folder (notice how versions are stored)
8 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
Lab 6 - Policies & schedules
Policy
1. Log in to vProtect Web UI
2. Go to Virtual Environment -> Backup SLAs tab
3. Create new backup policy
name: Production VMs enable auto-assignment - mode Assign only add rule based regular expressions - .*[Aa]lpine.* - matches all VMs with substring Alpine or alpine Go to Rule and Select Backup Destination
4. Save Policy
5. Go to Hypervisor Managers tab in the Infrastructure section
6. Run inventory synchronization on your RHV environment
7. Verify that all your RHV VMs have been assigned to the policy
Schedules
1. Go to Virtual Environment -> Backup SLAs tab
2. Change tab to Schedules
3. Create a schedule for full backup:
name: Production VMs - Full backup type: Full time: 22:00 days of week: just Sunday choose policy on the right: Production VMs
4. Create a schedule for incremental backup:
name: Production VMs - Incremental backup type: Incremental time: 22:00 days of week: just Monday-Saturday choose policy on the right: Production VMs
5. Go to the Virtual Environments
6. Verify that you have your VMs marked as outdated
Lab 7 - VM Backup & Recovery
Backup 9 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
1. Log in to vProtect Web UI
2. Go to Virtual Environments -> Instances
3. Click backup button (on the right) near the VM that resides on RHV
If you are unsure, click on the name of the virtual machine to enter the details of vm. In the upper right corner you will see the name of the hypervisor
4. Leave Full backup as the backup type
5. Run backup
6. Verify if it has succeeded
7. Go to Virtual Environment -> Instances details of your VM
Backup history tab - look for entry related to the backup that has just been created Snapshots tab - look for the snapshot that has just been created and check if it has been marked as Current for incremental
8. Go to the dashboard and check values of the used space size and charts
9. Go to the the RHV console
Attribute Value
RHV manager UI https://rhv-m.storware.lab/ovirt-engine
RHV user admin
RHV password St0rL@bs
10. Log in as admin and go to Events tab
check recent activity related to snapshots go to the Compute -> Virtual Machines -> VM that you have backed up -> Snapshots verify that snapshot has been left for future incremental backups
Restore
1. Go back to the vProtect and try to restore VM: Virtual Environments instances list - button on the right select your hypervisor manager and default storage leave rest of the fields with defaults 2. Once the job completes, check if it has succeeded and verify again in your RHV console that VM is present
Lab 8 - VMs details & settings
Virtual environments details
1. Log in to vProtect Web UI
10 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
2. Go to VM that resides on the RHV that has been previously backed up 3. Browse through charts at the top - activities / backup size / backup time / restore time 4. Go to the Settings verify which Settings are available for RHV VMs
Lab 9 - Pre/Post snapshot command execution
In this lab we'll simulate remote command execution before and after snapshot takes place. Usually you would execute commands on the remote host, but to skip unnecessary steps required to setup network, we'll execute commands on the 127.0.0.1 (which is basically node which executes commands)
1. Log in to vProtect Web UI 2. Go the the VM settings running on RHV 3. in Pre snapshot command execution:
enable it add command argument: /bin/bash add command argument: -c add command argument: ip a > /tmp/PRE-SNAPSHOT
1. in Post snapshot command execution:
enable it add command argument: /bin/bash add command argument: -c add command argument: hostname > /tmp/POST-SNAPSHOT
1. in Settings -> SSH access tab:
SSH host - 127.0.0.1 SSH user - root click Save
1. in Change SSH password tab:
SSH password - St0rL@bs - you need to click Change ssh password
1. Run backup with the button at the top-right corner 2. We need to Login with SSH to the vProtect Server and Node operating system and verify if files /tmp/PRE-SNAPSHOT and /tmp/POST-SNAPSHOT are present and check their contents
Lab 10 - File-level restore
Automatic mount
1. Log in to vProtect Web UI 2. Go to the Virtual Environments tab 3. For the VM running on RHV that you have backed up previously select mount (3rd icon) 4. Select Mount filesystems automatically and node-local 5. Run the job and once it completes go to the Mounted Backups tab in the Virtual Environments section 6. Click magnifying glass icon to go to the details
11 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
7. At the bottom you should be able to see single File System entry - corresponding to the root file system 8. Click browse 9. Navigate to /etc/passwd file, select file and click Download button 10. Verify that contents ZIP archive contain the file you restored 11. Log in to the node-local using SSH and navigate to the mount point of the mounted backup - by default in /mnt/vprotect 12. Check if you are able to list files in this directory 13. Go to Virtual Environments -> Mounted Backups -> click Delete icon on the right of the mounted backup and wait for task to complete 14. Refresh the page and make sure backup is no longer on the list
Manual mount
1. Log in to vProtect Web UI 2. Go the the Virtual Environments tab 3. For the VM running on RHV that you have backed up previously select mount (3-rd icon) 4. Select Specify filesystems to be mounted and node-local 5. Select just first file system on the list and accept default values for rest of the parameters 6. Run the job and once it completes go to the Mounted Backups tab 7. Click magnifying glass icon to go to the details 8. At the bottom you should be able to see single File System entry - corresponding to the selected file system 9. Click browse 10. Navigate to /etc/passwd file, select file and click Download button 11. Go to Virtual Environments -> Mounted Backups -> click Delete icon on the right of the mounted backup and wait for task to complete 12. Refresh the page and make sure backup is no longer on the list
iSCSI share (Linux)
1. Log in to the node-local over SSH and check what is IQN of this iSCSI client
cat /etc/iscsi/initiatorname.iscsi
2. Log in to vProtect Web UI
3. Go the the Virtual Environments -> Instances tab
4. For the VM running on RHV that you have backed up previously select mount (3-rd icon)
5. Select Share drives over iSCSI and node-local
6. Select just first disk on the list
7. Click Add Client and provide IQN that you have found on the node-local
8. Accept default values for rest of the params and run the job
12 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
9. Once it completes login with SSH to the node-local again
Discover iSCSI targets
iscsiadm --mode discovery --type sendtargets --portal NODE_LOCAL_IP
Login to iSCSI target (change target name to match yours of the target)
iscsiadm --mode node --targetname iqn.2013-03.storware.eu:216b66a1- 9797-41e6-a1f6-a74f767dfc60 --portal NODE_LOCAL_IP --login
Verify block device is present
lsblk
10. Go to Virtual Environments -> Mounted Backups -> click Delete icon on the right of the mounted backup and wait for task to complete
11. Refresh the page and make sure backup is no longer on the list
Lab 11 - Snapshot Management
1. Log in to vProtect Web UI 2. Go the the Virtual Environments -> Snapshots SLAs tab 3. Create new policy with the name RHV-1d 4. Leave auto-assignment disabled 5. In Virtual Environments section - assign VM that you previously backed up on RHV environment 6. In Rule section specify: Retention versions: 2 Retention days: 1 7. Save policy 8. Now create a schedule that will periodically create snapshots - go to the Virtual Environments -> Snapshots SLAs section and then change tab to Schedules 9. Click Create button and select Interval as Schedule execution type 10. Set time of interval to 08:00-16:00, frequency: every 180 minutes 11. Change days of week for Friday 12. Select policy that you have just created 13. Go to the Virtual Environments -> Instances VM you selected 14. In the details -> top right corner there should be additional Snapshot icon (camera) 15. Click it, so that snapshot task appears 16. Verify if snapshot has been created in RHV (Compute -> Virtual Machines -> Snapshots tab of this VM) 17. Submit 2 more snapshot tasks 18. Verify in RHV client that first snapshot has been removed
13 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
events tab on the left menu 19. Now in vProtect UI -> VM details -> Snapshots tab click icon on the right 20. Verify in RHV that VM has its snapshot reverted (Events tab)
Lab 12 - Daily Administration
1. Log in to vProtect Web UI
2. Go the the task console
3. If there are no tasks:
run inventory synchronization for the RHV host run full backup of the VM residing on the RHV host run snapshot on the VM residing on the RHV host
4. Filter tasks by
type string (try with the part of the VM name) status
5. Click button that cleans up tasks that has completed (success of failure) - cross with green and red dots icon
6. Go the Nodes section
7. Check status of the node
8. Click stop icon on the node-local
9. Wait for the node to shutdown - refresh periodically with the icon in the top right corner until it changes to Stopped state
10. Log in to node-local using SSH
11. Check if service is running:
systemctl status vprotect-node
12. Start vprotect-node:
systemctl start vprotect-node
13. Verify if the node is in he Started state in the vProtect Web UI
14. Verify available staging space on the node
15. In the Nodes -> Node Configuration that you created earlier go to the General tab
14 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
16. Check the value of the Min. free space for export (staging space threshold) [%]:
if the amount of free space drops below this value new export tasks will be waiting in the queue until there is available space on the staging again
Lab 13 - Command Line Interface
1. Log in to node-local
2. Run core vprotect command to see help of the main command - it will return different modules corresponding to different sections in vProtect Web UI or administrative tasks
vprotect
3. Run vprotect hv command to see help of the hv module
vprotect hv
you can use analogical commands with other modules used in this lab such as task, brvm, etc.
4. Log in to vProtect interface (when prompted provide your admin password if it has been changed):
vprotect login -u admin
5. List available hypervisor managers and hypervisors
vprotect hvm -l vprotect hv -l
6. Initiate inventory synchronization task on your RHV manager by specifying its GUID, i.e.:
vprotect hvm -s 2e8843a9-5548-4a48-b900-be1cb440c779
7. List tasks in assigned to the current node
vprotect task -l
8. List all tasks in the queue
15 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
vprotect task -L
9. List VMs in the inventory - note the GUID of your RHV VM
vprotect vm -l
10. List backup destinations - note the GUID of you backup destination
vprotect bd -l
11. Run backup incremental backup on the RHV VM (first ID is the VM GUID, second is backup destination GUID):
vprotect brvm -i f3692602-4637-41d8-94c6-e43d19af2bea 4a1a96c2-6505-4afc- 9da5-f294eb4ddfba
12. Note the GUID of the task returned - monitor its progress the following command (use your task GUID):
vprotect task -g 3acccb0f-899d-4fb9-ad76-1c12c83fe9a9
13. List backup history of the VM that you have just used to perform backup - note the last full backup GUID:
vprotect vm -L f3692602-4637-41d8-94c6-e43d19af2bea
14. Find hypervisor manager GUID (ID in the vProtect environment):
vprotect hv -l
15. Find UUID (ID in the RHV environment, not in vProtect) of the hypervisor storage called data
vprotect hs -l
16. Find UUID (ID in the RHV environment, not in vProtect) of the hypervisor cluster called Default
16 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
vprotect hc -l
17. Restore last full backup to the RHV manager and storage that you found:
first ID is the last full backup GUID second ID is the HV manager GUID third ID is the storage UUID last ID is the cluster UUID
example:
vprotect brvm -M 4db7e81e-ef32-4d44-a120-db24cdebdef2 2e8843a9-5548-4a48- b900-be1cb440c779 66bdfabd-f4cc-4d33-99d5-be330dca1ba2 70094294-8f11-11ea-b41e-005056a62f45
18. Monitor task as previously
19. Mount last full backup on the node-local:
first ID is the last full backup GUID nodes can be referred by names - here node-local mode is auto this mode assumes that it needs location where / (root) must be mounted - here: /mnt/vprotect/test
example:
vprotect mnt -m 4db7e81e-ef32-4d44-a120-db24cdebdef2 node-local auto /=/mnt/vprotect/test
20. Monitor task as previously
21. Verify that in /mnt/vprotect/test backup has been mounted:
df -h
and you can see backup files:
ls -l /mnt/vprotect/test
Lab 14 - Recovery Plans
17 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
1. Log in to the vProtect Web UI 2. Go to Virtual Environments -> Recovery Plans tab 3. Create new plan with the name Prod-DR-Plan 4. Click Save 5. In the next screen - click Add rule to add first rule for RHV production environments: Name: RHV-prod Virtual Environments - choose RHV Hypervisor v4 type and your RHV VM No schedules assigned Parameters - make sure to have your RHV hypervisor and Data as import to storage set 6. Double check if you have assigned VMs - if not - edit rule, select VMs and update them 7. Go to Virtual Environments -> Recovery Plans tab -> click Execute recovery plan button on the right of your Recovery Plan and confirm by clicking Run 8. You should have task created in the console that restores backup to the RHV Hypervisor 9. Go to Virtual Environments -> Recovery Plans -> your Recovery Plan details 10. Notice that in Restore History you have a new entry with all restore jobs that have been executed as a part of this recovery plan
Lab 15 - Application backup
Application backups are generic mechanism that allows administrators to integrate their existing scripts or native backup mechanisms of a particular applications (such as databases) into vProtect.
In this lab we'll simulate backup of a MariaDB database.
1. Log in to the vProtect Web UI
2. Go to the Applications -> Command Execution Configurations tab
3. Click Clone button (on the right of) MySQL/MariaDB configuration and provide it your name, i.e. My MariaDB Backup Command
4. Click insight it to see details and browse through different tabs - especially check how and what command is invoked, and which parameters will be required by this configuration
5. Click Save
6. Go to Instances tab and click Create Application Definition
name: My MariaDB node: node-local or node1 Command Execution configuration: My MariaDB Backup Command SSH access: host: 127.0.0.1 (we'll simulate remote host, but in the lab technically it is going to be vProtect's database user: root
7. Click Save
8. Scroll down to Configure SSH password and provide your labs root password to the vProtect machine, then click Change password
18 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
9. Scroll to Environment variables and set few parameters, which are required by this script (investigate all parameters by :
VP_MYSQL_USER: root VP_MYSQL_PASSWORD: vPr0tect VP_MYSQL_LOGFILE: /tmp/mymaria.log VP_MYSQL_STOREPATH: /tmp/mymaria VP_MYSQL_DBLIST: vprotect
10. Click Save
11. Now copy the script to the "target" machine - in this case we have script provided by the vProtect Server package - we'll simulate it by copying script to the different location on our vProtect machine:
cp /opt/vprotect/scripts/application/vp_backup_mysql.sh /root
12. Make sure to have executable permission:
chmod +x /root/vp_backup_mysql.sh
13. Create output directory for backup command:
mkdir /tmp/mymaria
14. Execute your backup in vProtect UI - click Backup button on the right of your application definition and point to one of your backup destinations.
15. Wait for backup to complete
16. Check charts in application details, check details of the backup in backup history tab
17. Now restore your application - click Restore button and specify:
Application as the target (this will simulate transfer over SSH to the remove host) path: /tmp/restored
18. Click Restore
19. Wait for the restore operation to complete and go to the /tmp/restored directory
cd /tmp/restored
20. Untar archive created by the script:
19 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
tar xvf 'My MariaDB-2020-06-01__13.41.tar'
21. Investigate extracted backup with zless:
zless tmp/mymaria/vprotect_2020-06-01_13:41:21.dump.gz
Lab 16 - Internal DB Backup & Recovery
1. Log in to the vProtect Web UI
2. Go to Settings -> Internal DB backup tab:
check current status of your DB backup check current status of your DB backup schedule
3. Click details button near Overview
4. Click Backup icon in the top right corner
5. Wait until backup is completed and verify in Backup History that you have your DB being protected
6. Sometimes you may want to have a separate copy of the DB as a regular file - log in to the host with SSH with vProtect Server and run:
/opt/vprotect/scripts/backup_db.sh /tmp/vprotect-backup.sql.gz
7. Simulated disaster - let's remove all schedules in Schedule tab (in vProtect Web UI)
8. Using SSH session - stop vProtect Server:
systemctl stop vprotect-server
9. Grab DB credentials to MariaDB (last 2 lines of payara.properties file):
tail -2 /opt/vprotect/payara.properties
10. Using these credentials connect interactively to MariaDB (notice that there is no space between -p and password), i.e.:
mysql -u vprotect -pSa56JosHx7V9Ud3rOpTGSrJoa2ptlA8W
20 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
11. In MariaDB CLI (let's assume you have reinstalled vProtect Server and it already created a new DB):
drop original DB:
drop database vprotect;
create empty DB:
create database vprotect;
quit from MariaDB CLI:
quit
12. Restore DB with previous file:
gunzip < /tmp/vprotect-backup.sql.gz | mysql -u vprotect - pSa56JosHx7V9Ud3rOpTGSrJoa2ptlA8W vprotect
13. Start vProtect Server:
systemctl start vprotect-server
14. Log in to the WebUI and check if your schedules are present
15. In this example we used file in known location: /tmp/vprotect-backup.tgz
if you have working environment you can first restore DB backup file from Settings -> Internal DB backup -> Overview details tab to the specific path on the node
if you have lost everything but your data in backup destination - for DD Boost FS it would be the last file in the following path:
/PATH_TO_YOUR_BACKUP_DESTINATION/CONTAINER_ID/ORIGINAL_PATH/BACKUP_FILE
* example (last element - `1588865802118 ` is your `vprotect_db.sql.gz` file):
``` 21 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
/vprotect_data/backups/c4c5d5a7-0ce0-4c32-a0c3- c0f85fc7eb53/vprotect_data/ app-c4c5d5a7-0ce0-4c32-a0c3-c0f8fc7eb53/vprotect_db.sql.gz/1588865802118 ```
Lab 17 - Storage providers
Backup
1. For this lab we'll assume that you have a share (NFS/SMB, or any other file system) mounted on your node - in our case /opt/vprotect is the directory that simulates such share - and vprotect user has access to it
2. Log in to the vProtect Web UI
3. Go to the Storage Providers -> Infrastructure
4. Click Add Storage Provider:
select File System as type specify FS provider as a name and select node-local as node responsible for backup
5. Click Save and click Yes in the dialog confirming that you want to define Storages
for file system storage provider you define paths manually, for other providers such as Ceph RBD or Nutanix Files - these are synchronized with providers inventory
6. In the Create Storage dialog provide:
My Share as a name /opt/vprotect as path
7. To enable support for incremental backups, we need to have at least 1 incremental schedule - go to Backup SLAs submenu
8. Click Create Policy and provide
Production shares as the name expand Auto-assignment - to see options available to automatically assign storages based on names or pools select your storage that you have defined make sure you have your backup destination selected click Save
9. Go to Schedules tab, click Create Schedule and provide:
Weekly full as a name Full as a backup type Sunday as the days of week
22 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
Production shares policy on the right click Save
10. Create another schedule Create Schedule and provide:
Daily incremental as a name Incremental as a backup type every day except Sunday as the days of week Production shares policy on the right click Save
11. Go to Instances and initiate backup with the button on the right of the newly created storage:
backup type: Full leave rest of the fields with the default values
12. One it completes - refresh the list of storages and check protection status
13. Click on the storage name to see the details
check backup/restore statistics charts check Backup history tab and click on the details of your backup check Settings tab - pre/post snapshot commands can be invoked on providers supporting snapshots - such as Ceph RBD
14. Log in with SSH to your node and generate some file with:
sudo -u vprotect dd if=/dev/urandom of=/opt/vprotect/LAB_TEST bs=1M count=100
15. Initiate backup with the button in the top right of the details
backup type: Incremental leave rest of the fields with the default values
16. Click on the storage name to see the details
check Backup history tab and click on the details of your backup
Restore to storage provider
1. Go the the Storage -> Instances 2. Initiate restore of your last backup using button on the right and provide: select Restore to storage provider note that Restore backup files to the node means that RAW backup files will be restored to specific location (backup image not individual files) your FS storage provider /vprotect_data/restored-share as the restore path 3. Navigate using SSH to /vprotect_data/restored-share and check if files have been restored - including your randomly generated file from incremental backup 23 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
File-level restore
1. Go the the Storage -> Instances 2. For the storage that you backed up previously select mount (3rd icon) 3. Select Mount filesystems automatically and node-local 4. Run the job and once it completes go to the Mounted Backups tab 5. Click magnifying glass icon to go to the details 6. At the bottom you should be able to see single File System entry - corresponding to the root file system 7. Click browse 8. Navigate to /opt/vprotect/LAB_TEST file, select file and click Download button 9. Verify that contents ZIP archive contain the file you restored 10. Go to Mounted Backups -> click Delete icon on the right of the mounted backup and wait for task to complete 11. Refresh the page and make sure backup is no longer on the list
Lab 18 - Troubleshooting
Logs
1. Let's assume you have an issue with backup and you would like to collect logs for this job
2. Log in to the vProtect Web UI
3. Go to the Virtual Environments
4. Go to details of a VM with at least 1 backup
5. Go to Backup History and on the right of the backup click second icon to download logs
6. vProtect filters logs based on the IDs of the tasks and presents log narrowed to this particular backup
7. Now click Logs (top right corner) and click Download All Logs
8. Select directory for node-local (some of them may be mounted over NFS if remote nodes are used)
9. Download vprotect_daemon.log
10. Compare contents of the 3 packages that you have downloaded - notice log level that was used
11. Now log in using SSH to the node-local (which is also server) and browse contents of /opt/vprotect/logs and /opt/vprotect/mnt/logs directories - this is the place actual logs are kept or mounted on server
12. Edit /opt/vprotect/log4j2-node.xml change INFO to DEBUG in
13. Start vprotect-node service:
systemctl restart vprotect-node
14. Edit /opt/vprotect/log4j2-server.xml change INFO to DEBUG in
24 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
15. Start vprotect-server service:
systemctl restart vprotect-server
16. Run inventory synchronization on the RHV manager to generate activity
17. Check logs again in /opt/vprotect/logs/node-local directory and check for entries on DEBUG level
18. Check server logs from /opt/vprotect/logs/api and /opt/vprotect/logs/appserver directories.
Common Server problems
1. While it is more common to see integration/configuration-related issues, in this scenario we'll simulate server-related connectivity issues
2. Stop server service
systemctl stop vprotect-server
3. Try to access server with the browser
4. Check node logs in /opt/vprotect/logs/node-local/vprotect_daemon.log and check for messages indicating that server is unreachable
5. Typically you start troubleshooting by checking connectivity on the Server port
6. On the node-local run:
curl http://localhost:8080/api
7. Check if the firewall allows incoming traffic on port 8080:
firewall-cmd --list-all
8. Now verify if the service is listening on port 8080 (on the node-local which is also server)
ss -tlnp
9. Check server service status
systemctl status vprotect-server
10. Start server service
25 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
systemctl start vprotect-server
11. Check node logs in /opt/vprotect/logs/node-local/vprotect_daemon.log and check for messages indicating that server is unreachable
12. Try to access server with the browser and log in
Common Node problems
1. In this scenario we'll investigate a typical issues with staging space
2. Change /vprotect_data path to some non-existing path (i.e. /wrong-path) in webUI -> Nodes -> Node Configuration (edit one assigned to your node-local - this simulates that path is not available, or not mounted properly, or that directory not exists.
3. Restart vprotect-node service to see how it behaves when staging is not available:
systemctl restart vprotect-node
4. Analyze logs and check what is the message
5. Check status of the node in the UI
6. Check status of the node service
systemctl status vprotect-node
7. Change back path to original one in Nodes -> Node Configurations - the one that you previously edited:
8. Restart vprotect-node service to see how it behaves when staging is not available:
systemctl restart vprotect-node
9. Analyze logs and check what is the message
10. Check status of the node in the UI
11. Check status of the node service
12. Now let's simulate problem with permissions on staging space
chmod a-w /vprotect_data
26 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
13. Run database backup from UI (Settings -> Internal DB backup -> Overview -> backup button in the right top corner)
14. Check if backup tasks has failed
15. Check the message in the node logs
16. Now let's bring back original permissions on staging space
chmod 0775 /vprotect_data
Lab 19 - Reporting
In this lab we'll browse through different reporting-related options in UI.
1. Log in to the vProtect Web UI 2. Go to Dashboard and check Success rate this success rate relates to your last 24h activity - based on statuses of backups that have been performed within last 24h 3. Check main chart at the bottom of the screen - this shows total backup size performed over last 14 days 4. Click Details button - this should open detailed reporting view here you can find tabs related to backup/restore operations performed within specified period of time you can filter report entries by status, policies, hypervisors etc. there is option to send e-mail to pre-configured recipients 5. Change date range and filters and see how the charts and the report details are changing 6. Go to Settings -> E-mail tab and check what options are available to configure e-mail notifications
you can enable notifications about recently failed backups (within last 15 minutes) notice that you can enable grouping of the VM entries in the report (by cluster, hypervisor, etc.)
Lab 20 - API Integration
In this section we'll connect to vProtect API. We'll use a simple script called call_api.sh which basically wraps curl to invoke APIs and return responses with formatted JSON.
1. Download script called call_api.sh, login.json and export.json to ndoe1 or node-local which is also server
curl -O http://10.40.0.253:8181/vprotect/current/lab-materials/call_api.sh
2. After download we need to change call_api.sh permissions
chmod 0775 call_api.sh
27 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
3. Install vim and jq packages
dnf -y install vim jq
4. Analyze contents of call.api.sh:
we provide default values for environmentmal variables such as VP_API, VP_USER and VP_PASS but these can be changed either by exporting them in the current shell or by providing values inline with the command such as VP_API=http://myhost:8080/api VP_USER=myadmin VP_PASS=pass ./call_api.sh ... in our default values we assume localhost connectivity and default credentials script expects parameters: ./call_api.sh
5. Check connectivity
./call_api.sh GET version
6. Check in the output what was complete URL, METHOD called
7. Analyze login.json file:
this file is actually a template - as the credential variables are substistituted when invoked we provide login and password in this case
export VP_USER=admin export VP_PASS=vPr0tect
8. Login to API (in our case we assume localhost connectivity and default credentials):
./call_api.sh POST session/login login.json
9. Check contents of cookie.txt file (notice JSESSIONID): 28 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
cat cookie.txt
10. List hypervisors
./call_api.sh GET hypervisors
11. Get current backup statistics for last 24h from dashboard:
./call_api.sh GET dashboard/backup-stats
12. Get last backups statistics for 24h (from dashboard):
./call_api.sh GET dashboard/backups-in-last-24h
13. Get task list:
./call_api.sh GET tasks
14. Check contents of export.json file and check how exporrt task can be created via API
notice that some parts are parametrized - such as: VP_EXPORT_BACKUP_TYPE VP_EXPORT_BD_GUID - GUID of your backup destination VP_EXPORT_VM_GUID - GUID of your VM notice that we can pass an array of VM GUIDs - in this lab it is just a single entry
15. List virtual machines - record GUID of VM running on RHV that you backed up previously
./call_api.sh GET virtual-machines
16. List backup destinations - record GUID of your backup destination
./call_api.sh GET backup-destinations
17. Export required variables and then call API
export VP_EXPORT_VM_GUID=f9851a7c-c920-4daa-86b6-f5cacac5eb8c
29 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
export VP_EXPORT_BD_GUID=4a1a96c2-6505-4afc-9da5-f294eb4ddfba
export VP_EXPORT_BACKUP_TYPE=FULL
./call_api.sh POST tasks/export export.json
18. Analyze JSON that has been sent and what was the response from API
19. Check task console in WebUI and look for the export/store tasks that have appeared
Lab 21 - RHV Web UI Module Integration
1. Download vprotect-plugin.zip from repository:
https://github.com/Storware/ovirt-engine-ui-vprotect- extensions/releases/download/2020.10.15/vprotect-plugin.zip
2. Copy this file to RHV manager:
scp FILE_PATH root@RHV_MANAGER:DESTINATION_PATH
example:
scp vprotect-ovirt-plugin_19_02_2020.tar.gz [email protected]:/tmp
3. Login over SSH to RHV manager and install plugin
extract it:
tar xvf vprotect-ovirt-plugin_19_02_2020.tar.gz
modify URL in vprotect.json file to match your vProtect server (change to the IP of your server) so it looks similar to this:
{ "name": "vprotect", "url": "plugin/vprotect/plugin.html", "resourcePath": "vprotect-resources", 30 / 31 vPro-1085-R - Storware vProtect - Implementation & Administration Lab Exercises - v7.md 2/24/2021
"lazyLoad": false,
"config": { "vProtectURL": "https://10.40.1.148:8181/api", "username": "admin", "password": "vPr0tect" } }
move files to the plugins directory:
mv vprotect.json vprotect-resources /usr/share/ovirt-engine/ui-plugins
1. We use port 8181 and its SSL certificate which may not be accepted yet in your browser:
in the other tab open URL of your server and navigate to https://10.40.1.148:8181/api/version optionally accept certificate
2. Open in your browser and navigate to RHV manager UI (administrative portal) and login as admin
3. You should have VM backup section in the left menu available
4. Navigate to VM backup -> Dashboard
5. Navigate to VM backup -> Virtual machines
6. Near the VM that you have backed up previously click action menu on the right and choose Backup
7. Leave default values and click Backup
8. Navigate to VM backup -> Tasks and see status of your backup
9. Navigate to VM backup -> Virtual machines
10. Click action menu on the right of your VM and choose Show backup history
11. Navigate to VM backup -> Virtual machines
12. Click action menu on the right of your VM and choose Restore
13. Leave default values and click Restore
notice: if storage data is not available - choose Other and copy UUID of the data storage from vProtect UI
14. Navigate to VM backup -> Tasks and see status of your restore job
31 / 31