Veritas Flex Appliances with NetBackup Design Guide
Bringing resilient, scalable and fully integrated data protection from the edge to the core to the cloud.
The Veritas Flex Appliance family delivers enterprise data protection services, both on-prem and in the cloud. This white paper highlights the solution benefits and features, use cases, architecture, best practices, sizing guidance and deployment of Flex Appliances with Veritas NetBackup™.
Veritas™ White Paper | October 2020 Contents
Introduction 3
Executive Summary 3
Scope 3
Target Audience 3
Flex Appliance Key Values and Features 3
Enable Consolidation and Increase Agility and Simplicity 4
Provide Multi-Tenancy with Network and Storage Segregation 5
Ensure Resilience and High Availability 5
Ransomware resiliency 6
Integrate and Automate with API 8
Support Fibre Channel 9
Flex Appliance Use Cases 9
Flex Appliance Models 10
Flex Appliance High-Level Architecture 11
Application Container Image and Instance 11
Flex Architecture Deep Dive: Platform as a Service Components 13
Management Plane 13
Control Plane 14
Resource Plane 14
Best Practices 14
Default Configuration 15
Performance Tuning 15
LUN Sharing 16
Sizing 16
Flex Appliance Deployment 17
Choose an Application Container Image 17
Create an Instance on a Flex Appliance 18
Monitoring 19
Conclusion 19
References 19
Versions 19 2 INTRODUCTION
Executive Summary
Today, the IT organization is more than the core data center. It spans from the edge to the core and to the cloud, including virtual environments and hybrid clouds along with traditional data protection deployments. One appliance model cannot meet the needs of all these different use cases. Organizations must quickly adapt their data protection infrastructure to rapidly changing business environments. IT organizations are also under increasing pressure to consolidate data protection solutions and to reduce costs.
The Veritas Flex Appliance family delivers enterprise data protection services, both on-prem and in the cloud. Flex Appliances bring agility, resilience, scalability and simplicity to NetBackup data protection. Using the simplified web interface, you can run multiple NetBackup and NetBackup CloudCatalyst deployments on a single Flex Appliance and create new deployments and upgrade in minutes without increasing your hardware footprint. With Flex Appliances, you get enterprise-wide on-premises and cloud data protection delivered on demand that you can rapidly adapt to meet the changing requirements of the business. (See Figure 1.)
Consolidation High Availability Ease of Use
• Reduce data center foot print • Data protection services • Operational simplicity • Simplify management on demand • Install & upgrade NetBackup • Eliminate data center sprawl • Easy to bring in new line of services in minutes business & protect their data • Edge to core to cloud • Fast deployment protection on the same appliance • Reduce planned & • Automate with APIs unplanned downtime
Figure 1: An overview of the benefits of Veritas Flex Appliances.
Scope
The purpose of this document is to provide technical details to assist in understanding the Flex Appliances with NetBackup solution. This white paper describes the solution benefits and features, use cases, architecture, best practices, sizing guidance and deployment of Flex Appliances with NetBackup. For installation, configuration and administration of each of the products discussed in this white paper, please refer to the appropriate Veritas product documentation.
Target Audience
This document is for customers, partners and Veritas field personnel interested in learning more about the Flex Appliances with NetBackup solution. It provides a technical overview, architecture, guidance in sizing and highlights some best practices.
FLEX APPLIANCE KEY VALUES AND FEATURES
The Veritas Flex Appliance family is a new concept in delivering enterprise data protection services both on-prem and in the cloud. Rather than relying on complex and costly data protection environments consisting of many converged or single-function backup, data deduplication, cloud tiering and storage silos spread across the enterprise, Flex Appliances offer a single, highly available and scalable solution that uses containerization to deliver enterprise-wide data protection services on demand. Table 1 provides a summary of Flex Appliance benefits and features.
3 Benefits Feature Description
Adapts with DevOps-like agility to • Containers provide a lightweight, secure environment to run standardized versions of business needs NetBackup software. • With no hypervisor, operating systems or NetBackup to install, admins can deploy new instances of NetBackup in minutes. • Quick NetBackup upgrades. Simplifies infrastructure and • Consolidate multiple NetBackup domains in a single Flex Appliance. reduces costs • Containerization of NetBackup enables administrators to scale NetBackup’s powerful data protection capabilities quickly and easily to many customers on commodity hardware. • MSDP container provides deduplication. On-demand deployment of NetBackup • Create NetBackup media, primary and CloudCatalyst servers on demand. • Run multiple versions of NetBackup software with a single Flex Appliance. • Use a streamlined administration console to deploy NetBackup application container. Adapts with DevOps-like agility to • Containers provide a lightweight, secure environment to run standardized versions of business needs NetBackup software. • With no hypervisor, operating systems or NetBackup to install, admins can deploy new instances of NetBackup in minutes. • Quick NetBackup upgrades. High availability • Container isolation prevents a container application failure from impacting other applications. • Deep monitoring of the primary server’s critical operation services provides remedial actions during the failure. • The Flex Appliance Shell provides hardware component information. • Auto-support and call home. Security and compliance • Provide WORM capability, retention locks and platform hardening against ransomware and malware threats. • Use SELinux to provide intrusion detection and prevention system. Scalability • Add capacity as needed, non-disruptively and automatically. Multitenancy • Data and network connectivity are segregated to each deployed container. • Flex uses Veritas Optimized Operating System (VxOS) security profiles to control container access. Long-term retention on-prem and in • Efficiently tiers to the cloud with CloudCatalyst containers. the public cloud • Connect with the Veritas Access Appliance, a software-defined storage appliance for long-term data retention with multicloud capability. Automate operations • Public APIs for integration and automation. • Available for any operations on the UI.
Table 1: Flex Appliance Benefits and Features
Enable Consolidation and Increase Agility and Simplicity
Flex Appliances provide container support through use of containerization. Table 2 lists the advantages of container technology compared to virtualization.
Containerization Virtualization
Size 10s MBs Several GBs Boot time Almost instantly Several minutes Modularity Can split applications into modules for easy management and Not available enhanced security
Table 2: Comparison of Containerization and Virtualization Comparison
4 With containerization, the NetBackup application container provides the following benefits: • Operational reliability when moved between nodes in the cluster • Increased modularity • Simplicity • Application/process isolation • Improved security • Faster startup and shutdown
Provide Multi-Tenancy with Network and Storage Segregation
All application containers running on Flex Appliances need to share the hardware resources of the node such as CPU, memory, disk I/O and network. Flex Appliances use network and data segregation and the Veritas Optimized Operating System (VxOS) security features to provide multi-tenancy to customers.
Network Segregation
Flex Appliances use the Macvlan network driver to assign a MAC address to each container’s virtual network interface; each MAC address is bound directly to a physical network interface. This approach provides external connectivity to and from the containers as well as network isolation between them. Macvlan provides the best network isolation for containers and allows NetBackup Appliance containers to use an actual IP address. NetBackup instances on a Flex Appliance support multiple interfaces, included physical interfaces and bonded interfaces. Support for multiple networks enables NetBackup media server instances to span multiple networks.
VxOS Security Features for Containers
The VxOS kernel provides name spaces, control groups and secure computing mode to control processes and resources at the OS level. Flex Appliances use these features to control access and manage resources.
Namespaces
The concept of namespaces is a feature of the VxOS kernel that provides fundamental support for containers in VxOS. Namespaces ensures a group of processes only sees its set of assigned resources and another group of processes only has access to its own, discrete services. Neither group of processes can see the resources assigned to the other.
Control Groups
Control groups (cgroups) provide resources management for the CPU, memory, disk I/O and networking. Using cgroups protects an appliance from being taken down by one container consuming all available resources on the physical system. Cgroups can be used to defend against denial-of-service (DoS) attacks on Flex Appliances.
Secure Computing Mode
The VxOS kernel seccomp (secure computing mode) feature limits the number of system calls a process can make through secure, one- way transactions. Flex Appliances use seccomp to control the security of the NetBackup containers with a seccomp profile. Each profile represents a list of privileged system calls that are blocked within the container.
Ensure Resilience and High Availability
Flex Appliances provide resilience and high availability (HA) to your NetBackup environment by container isolation, monitoring, AutoSupport and Call Home functionalities.
5 Container Isolation
Container isolation prevents a container application failure from impacting other applications. Containerized application architecture segregates network connectivity and eliminates inter-service interference. Backup administrators often test new software versions prior to deploying them in service. Flex Appliances streamline this process and enable backup admins to rapidly bring new versions of NetBackup online for testing. After testing, you can simply replace a containerized application with the new version or run multiple versions simultaneously.
Flex 5340 Appliance with High Availability
The Flex 5340 Appliance includes Veritas InfoScale™ Availability components, enabling HA support. When configured with HA, the Flex 5340 includes two server nodes in a cluster configuration. The server nodes communicate through redundant, direct or crossover 1 GbE connections. In an HA configuration, all services and applications on the Flex Appliance—NetBackup services like master server, media server and CloudCatalyst as well as the system services—are resilient.
Primary Server Deep Monitoring on Flex Appliances
From Flex 2.0, Flex Appliances monitor primary servers’ critical operation services. Remedial actions like restarting a service or failing over the primary server to the other node in the cluster will take place when a failure is detected.
Ransomware Resiliency
Backups are an organization’s key to recovery. To ensure your critical and most important asset—data—and your IT infrastructure is protected from an attack, Veritas focuses on data integrity to help backup files remain safe and untouched from malicious invaders.
Data Encryption
NetBackup software supports data encryption in transit and at rest.
• In-transit encryption—Ensure your data is being sent to authenticated environments and is protected while in transit. This solution leverages Veritas or customer-provided TLS 1 .2 certificates, with 2048-bit key support to ensure data encryption during transit.
• At-rest encryption—If hackers are successful in getting to the data, having it encrypted protects it from being exploited. Veritas offers AES 256-bit, FIPS 140-2 cryptography with our own key management while allowing customers to leverage their preferred key management using the Key Management Interoperability Protocol (KMIP).
Flex Appliances Immutable Storage
NetBackup and Flex Appliances provide immutable and indelible storage that reduces the risk of malware or ransomware encrypting or deleting backup data, thereby making it unusable. Within the Flex Appliance, the NetBackup WORM storage server offers a secure, container-based MSDP solution. Flex Appliances offer Enterprise and Compliance lock-down modes, so you can choose the right immutability strength (see Figure 2). The NetBackup and Flex Appliance solution has completed a third-party Immutability Assessment from Cohasset Associates, an industry-recognized assessor of immutability controls, specifically SEC Rule 17a-4(f), FINRA Rule 4511(c) and the principles of Commodity Futures Trading Commission (CFTC) in regulation 17 CFR § 1 .31(c)-(d).
Flex Appliances come with a wide variety of security features (see the Flex Security document for details) that include:
• OS security hardening, including Security-Enhanced Linux (SELinux).
• Intrusion Detection System (IDS) / Intrusion Protection System (IPS).
• Robust, role-based authentication.
• Locked-down storage array.
6 The NetBackup 8.3 primary server communicates with the storage unit to gather immutability and indelibility capability and WORM retention period (min/max) settings. Then the primary server sets up immutability controls on the storage unit and applies the WORM retention period policy. NetBackup software provides backup image management with visual representation of immutable lock, image deletion after WORM retention period (via CLI) and honor legal hold on the catalog. The Flex Appliance runs the immutable storage server to provide WORM capability, retention locks and platform hardening against ransomware and malware threats. Compliance Clock is used for the retention period and is independent from OS time. The Flex Appliance has two lock-down immutability modes: Enterprise and Compliance. You can enable the appliance lock-down state at any time. You can choose either a Compliance mode or Enterprise mode MSDP storage container, but you cannot mix them.
Single Flex Appliance Rack Flex Appliance (Optional) Rack Compliance Clock Independent of OS time
Immutability mode Compliance or Enterprise Primary Storage • Primary catalog information • Backup policies
MSDP plug-in using OST NetBackup Client Source Data NetBackup Primary NetBackup Media WORM (Objects) Server Server(s) Storage Server(s) Flex Container Storage Can contain immutable and non-immutable images
Figure 2: An overview of the Flex immutable storage architecture.
Solution Hardening
Flex Appliances eliminate root account access to the appliance OS and MSDP container; only the host admin account can log in to compute nodes. Account policies are used to allow elevated users certain administrative commands and access to Flex Shell and web UI operations.
The following lists describe the firmware security hardening.
• Boot
ν Eliminate “single user” mode/“rescue mode” boot options
ν GRUB menu editing disabled
• Storage
ν No storage reset (factory reset/reimage allowed)
ν Locked down storage array
7 User Management
From Flex 2.0, Flex Appliances force password changes during initial configuration to ensure the default password does not remain active on the system. You can set your own password policy, including the option to use the Security Technical Implementation Guides (STIGs) for validation. You can use the Flex Appliance Console to edit the password policy for user passwords. The password policy is enforced for local Flex Appliance Console users and the hostadmin user in the Flex Appliance Shell.
The default password policy is as follows:
Password complexity:
• Minimum characters: 8
• Minimum numbers: 1
• Minimum lowercase characters: 1
• Minimum uppercase characters: 1
• Minimum special characters: 0
• Minimum different characters: 0
• Maximum consecutive repeating characters: 99,999
• Maximum consecutive characters of the same type: 99,999
Password age:
• Days before password must be changed: 99,999
• Days before password can be changed: 0
• Days before password expires to display warning message: 10
• Minimum different passwords before allowing reuse: 7
For details, refer to the NetBackup Security and Implementation Guide and Flex Security documents to support secure deployment.
Integrate and Automate with API
Veritas Flex API is available at the Veritas™ Services and Operations Readiness Tools (SORT). Customers and developers can integrate customized codes into Veritas products and applications. API promotes innovation, provides additional channel for customer insights, and enhances user experience by enabling collaboration.
You can automate management tasks and enhance performance monitoring with Flex API:
• Instance deployment
• Accessing appliance and instance performance data
ν CPU, memory, disk read & write and network
ν “Joint” or “correlated” API to correlate what work the instance is doing in the NBU context to what is happening on the physical layer of the system
• Apply add-ons and updates.
8 Support Fibre Channel VM VM VM From Flex 2.0, Flex Appliances support VMware and tape out backups over Fibre Channel, enabling you to:
• Protect VMware infrastructure using VMWare APIs for Data VMware ESXi Protection (VADP).
• Reduce risk and stay compliant using long-term retention with tape.
You must assign Fibre Channel ports to Flex instances. Note the following information about port sharing and multipathing:
• Port sharing—You can assign the same port to multiple instances if the instances belong to the same tenant. You can also use the same port for both VMware and tape out backups.
• Multipathing—If multiple ports are connected to the same devices, you can assign those ports to a single application instance in case one of the paths fails. You cannot assign the ports to different application instances. Figure 3: Using a Flex Appliance to back up VMware VMs.
For VMware backup, hypervisor-level snapshots are taken for virtual machines (VMs) and the VM data is streamed directly from storage to the backup server over Fibre Channel. (See Figure 3.)
FLEX APPLIANCE USE CASES
Veritas offers two appliance models—the Flex 5150 Appliance and the Flex 5340 Appliance—to protect your data anywhere it lives from the edge to the core to the cloud.
The Flex 5150 is a purpose-built, cloud-connected NetBackup data protection solution in a self-contained, compact, easy-to-use Appliance. The Flex 5150 expands the NetBackup family of appliances to the edge of the enterprise network and to departmental organizations within the enterprise. Below are some Flex 5150 use cases:
• Remotely managed edge solutions by the enterprise backup team.
• Standardized backup platform across all locations.
• Multiple remote locations with a small number of clients and a limited amount of data to protect; data required to be protected locally and backups replicated to a data center or the cloud for disaster recovery (DR).
The Flex 5340 Appliance is designed for data centers and large locations. It consolidates multiple data management applications in a single, scalable solution. Below are some Flex 5340 use cases:
• Running multiple NetBackup instances on a single Flex 5340.
• Possible NetBackup instances include primary server, media server and media server cloud.
• Each server instance runs in an isolated container, independent from any others.
• Can run multiple domains on a single appliance with full segregation.
9 FLEX APPLIANCE MODELS
The Flex 5150 is a 1U server system with no external storage shelf component designed for remote offices with smaller workload requirements. (See Figure 4.) This design offers a simplified appliance that is reliable and cost-effective. You can run the protection job at the remote Figure 4. The Flex 5150 Appliance. office on a nightly basis and replicate the backup data to a central office or data center. You can manage the Flex 5150 remotely without on-site technical support.
The Flex 5340 Appliance consolidates multiple NetBackup deployments into a single resilient and scalable solution with a high-availability (HA) option. (See Figure 5.) With this architecture, a single Flex 5340 enables multiple NetBackup and CloudCatalyst configurations, reducing costs and complexity. The Flex 5340 hardware configuration includes HA server nodes in an active/active cluster configuration, powered by InfoScale Availability, providing HA for all container images running on the Flex 5340.
Both the Flex 5150 and the Flex 5340 provide a complete, cloud-connected NetBackup data protection solution in a self-contained, compact and easy-to-use appliance. Table 3 Figure 5. The Flex 5340 Appliance. lists the Flex 5150 and Flex 5340 position and hardware configurations.
Flex 5150 Flex 5340
Position Streamlined protection for remote or Run multiple Veritas software products on a single, branch offices. flexible converged solution with HA option. Deployment guideline 1 media server, 1 primary server, Up to 6 primary servers and 6 media servers 1 CloudCatalyst server Usable storage capacity 13.23 TiB 240 TiB–1,920 TiB 4 TB drives or 8 TB drives Maximum 4 storage shelves (1,920 TiB) CPU 1X Intel Xeon 3106 CPU 2x Intel Xeon 6138 CPUs @ 2.0GHz, 20 cores each DDR4 2666 RAM 64 GB 768 GB or 1.5 TB 10 GBASE-T Ethernet ports Up to 2 10 Dimensions H x W x D cm 4.32 x 48.26 x 79.38 (3.5 x 19.0 x 30.5) / (8.9 x 48.3 x 76.9) (inches) (1.7 x 19 x 31.25) Maximum weight with disk 36 / 16.4 Appliance 43 / 19.5 drives (Pounds/Kilograms) 6/2.7 optional mounting rails Maximum power consumption 160 watts 600 watts Operating temperature (oF/oC) 50–95 / 10–35 50–95 / 10–35 AC voltage range 100–127 volts / 200–240 volts 90–140 volts / 180–264 volts
Table 3: Flex 5150 and Flex 5340 Position and Technical Details
For more information on Flex Appliance models, refer to the following data sheets:
• Flex 5150 data sheet • Flex 5340 data sheet
10 FLEX APPLIANCE HIGH-LEVEL ARCHITECTURE
Flex Appliances tightly integrate with NetBackup and simplify your environment by providing a common platform for Veritas applications. The Flex Appliance lets you deploy data protection services and change them on demand simply by selecting and configuring the NetBackup application container as needed. Once you’ve selected the container image and configured an instance, NetBackup applications are deployed into production across the business—in minutes. You can consolidate multiple NetBackup and CloudCatalyst deployments (domains) on a single Flex Appliance, substantially reducing data center costs and complexity.
The Docker container software runs directly on the VxOS, which is a Linux-based OS. The VxOS provides the Flex Appliance kernel, runtime library and container engine. Flex Appliances use the container isolation and security technology to ensure users are kept separate from one another when using different instances of NetBackup on a single appliance. Between the kernel features built into the VxOS and the network and data segregation, consumers of NetBackup services are effectively firewalled from one another. This multi-tenant architecture simplifies your NetBackup environment by allowing multiple NetBackup domains to run on this common platform. (See Figure 6.)
Flex Appliances efficiently tier to the cloud with CloudCatalyst containers. CloudCatalyst integrates into existing NetBackup environments to help automate a customer’s path to multicloud environments. They optimize data reduction with CloudCatalyst and speed end-to-end deduplication up to an average of four times faster than alternative solutions before sending data to the private or public cloud. This approach dramatically reduces time, cost and network bandwidth of cloud storage. You can also perform fast recovery of data directly from CloudCatalyst to the NetBackup client.
NetBackup Primary NetBackup Primary NetBackup Service Administration Service Administration Cloud Catalyst
NetBackup Media Service Data Segregation NetBackup Media Service NetBackup Deduplication (MSDP) Segregation Network Deduplication (MSDP) Cloud Catalyst Public Cloud
Access
Veritas Optimized Operating System (VxOS)
Access 3340 LTR Flex Appliance Appliance
Figure 6. An overview of the Flex Appliance’s multi-tenant architecture.
Now, with Flex Appliances, enterprise-wide on-premises and cloud data protection can be delivered on demand and rapidly adapted to meet the changing requirements of the business.
Application Container Image and Instance
Before you can create an application container instance, install an application add-on or upgrade or update the appliance software, you must first add the applicable image files to the repository.
11 Container Images
Application container images are static and immutable. Adding support for multiple versions of NetBackup is easy with a Flex Appliance. You can download the NetBackup Container Images from the Veritas support website independently of the Flex 5340 Appliance software.
Figure 7 shows Flex Appliance support for different Figure 7. An overview of the Flex Appliance Console showing support for NetBackup by version. NetBackup software versions.
There are different types of container images, as shown in Table 4.
Use Case Content
Application Instantiate and run a container An application and changes required to make the application run and working Add-On Provide add-on components to Add-on libraries and binaries application containers Appliance Upgrades and Updates Upgrade and update an application container New binaries, libraries and configuration files
Table 4. Container Image Types by Use Case
Container Instances
Application container instances are built from static NetBackup NetBackup NetBackup Immutable MSDP container images. One container has a single instance per Primary Server Media Server Storage Server role. Supported roles include NetBackup primary server, media server (including MSDP and Advanced Disk) and deduplication (MSDP) and NetBackup Immutable MSDP Advanced MSDP Storage Server. (See Figure 8.) You can create an instance Disk with the Flex Appliance’s web GUI. Figure 7. The application container instance roles supported by Flex Appliances.
An application container instance has persistent data and non-persistent data to simplify the application instance upgrade process. (See Figure 8.) Static Docker Container Instances
• The persistent data includes the primary service Persistent Data Persistent Data catalog, backup images and configuration settings. Configuration settings, data Configuration settings, data
• The non-persistent data includes base NetBackup Non-Persistent Data Non-Persistent Data Application base, services Application base, services software components, services and binaries. NetBackup Primary Container Service NetBackup Master Container Service
Below are the processes during upgrades and maintenance:
• Original container instance is shut down (less than a minute).
• New instance based on the new container image is started.
NetBackup Primary NetBackup NetBackup Container Image Media Cloud Media Cloud • Persisted data is mounted to the upgraded Container Image Container Image container instance. Static Docker Container Images • NetBackup is available to do backups at the new version
(about a minute). Figure 8. An overview of the container instance upgrade process.
12 FLEX ARCHITECTURE DEEP DIVE: PLATFORM AS A SERVICE COMPONENTS
The Flex Appliance architecture’s objective is to build a next-generation, multi-tenant appliance platform with scalability, high availability and agility and integration with Veritas applications. The platform owns the resources to provision applications, storage, network and compute on demand. The service components have a management plane, a control plane and a resource plane. (See Figure 9.)
Management Plane
Login HTTPS Dashboard Identitiy Service
• Service-catalog • Local Users • Management Token • LDAP/AD Config IP address • RBAC Settings • Private Data
User Operations Operation Response
Control Plane Resource Plane
Persistent Management Telemetry Service Image Service Playbook Compute Node Service Service Variables Notifier Registry Host Agent etcd Queue Execute Constraint Playbook Check Analyzer Response Playbook Modules Metadata Playbooks • Application • Networking Flex OS Storage Flex Container Storage • Storage File Storage/Time Series Persistent Time-series Image Block Storage • Settings Datastore Store Store Store Raid-6 Disks • Upgrade Raid-1 Disks
Figure 9. An overview of the Flex Appliance architecture’s service components.
Management Plane
The management plane provides a dashboard and identity service. The dashboard passes user login information to the identity service. After verification, the identity service returns an access token with the objects and roles the user has back to the dashboard.
Dashboard
The dashboard is a UI that lets users interact with the appliance to perform duties within their designated roles. The dashboard provides functionalities such as appliance- or node-specific changes; managing appliance, applications and instances; monitoring the various components and resource usage. Users are prompted for their credentials to access the dashboard.
Identity Service
The dashboard forwards user credentials to the identity service to validate users and also understand their roles. The identity service allows the super administrator to configure one of the following types of identity providers (but not limited to):
• Local users
• Active Directory
The responsibilities of the identity service are to:
• Maintain the local user database and network identity provider details.
• Maintain a list of roles that can be assigned to users and user-groups.
• Maintain a mapping of users/user-groups, objects and their roles on the objects.
• Validate the user by the respective identity provider using the specified credentials.
• Return an access token with the objects and roles the user has.
• Verify the access token when presented by a resource owner in the attempt to access the resource.
13 Control Plane
Upon a successful user login, the dashboard contacts the management service when a user performs actions. Before taking any action on user operations, the service contacts the identity service to validate the user and related capabilities using the access token sent by the dashboard.
Management Service
The management service executes the operations on applications, networking, storage, settings and upgrades. The playbook modules include the specific instructions for each node with a set of variables/constraints to complete the action. The management service has various modules for different system capabilities such as storage and tenants that may require special processing. For example, adding a tenant and setting its properties requires only saving the information in the persistent store. The information can be used by all tenant operations to enforce constraints.
A management service instance can be hosted on one of the appliance nodes or externally as long as it can communicate with the appliance. This setup also makes the service capable of managing multiple appliances to provide a central management console.
For all write operations, the service records the change in a persistent store for a desired system state. This information can be used later to synchronize a node or re-create the system.
Persistent Service
The persistent service provides a stand-alone and technology-agnostic persistent store. The responsibilities of the service are to save and return requested information in the persistent store it owns. The service can be replaced with an embedded persistent store if there aren’t too many users.
Registry Service
The registry service maintains and centrally serves Docker container images delivered by application images. The service is contacted by the orchestration layer when an application instance is provisioned and started.
Resource Plane
The management service uses the playbook corresponding to the API along with user input and any other constraints to a random or set of target nodes, as appropriate, by calling the async task REST API exposed by the host agent in the resource plane. The service checks the state of the task periodically to keep the user apprised of progress.
Host API Service
The host agent exposes a set of sync and async task REST APIs that can accept a predefined playbook and variables as a task. The agent runs the playbook and posts progress updates to the caller. The agent makes use of playbook modules installed on the nodes to achieve the desired state. The playbook modules understand the technologies and how to get to a desired state.
BEST PRACTICES
Flex Appliances use container technology to enable multiple workloads and applications to run on a single Flex Appliance. Each application runs within an independent container, but all applications share the same underlying hardware resources (CPU, memory, disk I/O and network); therefore, it is a best practice for organizations to plan the applications that run on the Flex Appliance and properly size, configure and tune the Flex Appliance.
14 Default Configuration
We recommend that a single Flex 5340 Appliance host no more than 6 media MSDP and 6 primary instances (12 in total) and the Flex 5150 should host 1 media, 1 primary and 1 CloudCatalyst container (3 in total). Generally, we recommend that no more than 60 percent of system memory is configured for the MSDP fingerprint cache. This limitation will help reserve 40 percent of system memory for operating system and application processes.
Performance Tuning
Load, stress and performance tests tune the Flex Appliance kernel parameters to maximize system performance. Memory allocation and storage layout are the two areas that need to be customized based on the performance requirements for each container.
Memory
The MaxCacheSize setting determines how many fingerprint indexes can be cached in memory per container, which can potentially influence the deduplication ratio. If MSDP can’t find an index in the cache that matches the new fingerprint index, that index will be treated as new and the associated data segment will have to be written to the storage pool.
To avoid memory starvation resulting in excessive swapping, we recommend the aggregate amount of fingerprint cache size be set at 60 percent, and 70 percent as an absolute maximum if the system is not constantly overloaded with a high number of concurrent jobs .This recommendation is based on the number of MSDP containers configured on the Flex Appliance and the size of the MSDP pool allocated to each container. From NetBackup 8.2, the default MaxCacheSize is set at 60 percent. If more than one MSDP instance is provisioned, the MaxCacheSize can be tuned based on each container’s MSDP storage size but cannot exceed 60 percent.
The formula for calculating the MaxCacheSize for an MSDP instance is ((Predicted Allocated MSDP pool size of the instance in TB * 0.5)/(1.5 * 1024) * 100). This formula is for use when a system has 1.5 TB of RAM installed. If the system has the default 768 GB of memory, then replace the (1.5 * 1024) in the above formula with 768.
The total amount of RAM required for media MSDP instance on a Flex Appliance is calculated based on the general rule of thumb: 1 GB of RAM for each TB of storage and no more than 50 percent of memory for fingerprint caching. For example, if the storage allocated for a media container is 80 TB, the total RAM required to run the instance would be 80 GB with no more than 50 percent of the 80 GB RAM for fingerprint cache. A Flex Appliance by default is configured with 768 GB of RAM, so the MaxCacheSize, which is expressed as a percentage of the total RAM on the Appliance, should be set to 5 percent. The 5 percent is derived and rounded from ((40GB/768GB) * 100) %. The percentage is a best practice recommendation; however, there is nothing wrong if you prefer to set it to 6 percent or higher as long as all MaxCacheSize added together does not exceed 60 or 70 percent of the total physical RAM.
To change the MaxCacheSize, you need to restart the pdde-storage process after the value is changed. The procedure below shows you how to change the MaxCacheSize: ssh appadmin@
Check the changed MaxCacheSize as below: sudo /usr/openv/pdde/pdag/bin/pdcfg --read /mnt/msdp/vol0/etc/puredisk/contentrouter.cfg --section CACHE --option MaxCacheSize
Restart the pdde-storage process with following commands: sudo /etc/init.d/pdde-storage force-stop sudo /etc/init.d/pdde-storage start
15 Storage
The storage allocation plays an important role in media MSDP performance. Veritas recommends the following best practices for storage configuration:
• Allocate enough storage to meet the size requirement of each instance.
• Avoid or reduce multiple instances sharing the same LUN to reduce I/O contention and achieve the optimal I/O performance.
A fully populated Flex 5340 storage shelf (for example, RBOD/EBOD) is configured with 6 x (11D+ 2P) RAID6 data LUNs, and a half- populated shelf is configured with 3 RAID6 data LUNs. See Figure 10 for Flex 5340 storage options:
Figure 10. Flex 5340 Appliance storage options. LUN Sharing
The Flex Appliances’ storage shelves are populated with either 4 TB or 8 TB. The size of a RAID6 LUN (11D + 2P) can be either 40 TB or 80 TB per LUN. LUN sharing will happen when you create an MSDP instance smaller than the LUN size or when the MSDP size is not a multiple of the LUN size. Follow the best practices below to minimize the possible I/O contention and achieve good I/O performance with LUN sharing:
• Limit to no more than two instances sharing the same LUN, if possible.
• Choose 4 TB or 8 TB shelves based on the MSDP storage pool size profile. Choose 4 TB shelves if the storage pool of multiple instances is 20 TB or less. This choice can reduce the need for more than two instances sharing the same LUN.
• Create an instance with the highest I/O performance requirement first. The storage pool of the first created instance will occupy the outer layer of the LUN, which can outperform the storage pool located on the inner layer of the LUNs.
• Use the backup schedule and Storage Lifecycle Policy (SLP) to reduce I/O contention and achieve the best I/O performance.
• Multiple instances sharing the same LUN will not affect performance unless they are activated at the same time. If the workload of small instances can be finished in few hours, then stagger the activation of each instance to avoid job overlap and thus I/O contention.
• Instances with a high deduplication ratio do not generate a lot of write IOs. If multiple instances (two or more) must share the same LUN and the backup workload can’t be staggered, then choose instances with high deduplication.
Sizing
Veritas Sales Engineers (SEs) can use an internal tool to combine historical performance information with customer-provided input to offer deployment options for a wide variety of operating conditions. Veritas recommends customers provide data protection requirements to SEs. It is best practice that all appliance sales include the use of this application to help accurately size customer environments.
16 FLEX APPLIANCE DEPLOYMENT
With Flex Appliances, IT can now deliver agile, enterprise-wide data protection services without the need or time required to acquire, deploy, configure, re-configure or manage a complex software and server environment. This section covers how to create an instance on a Flex Appliance. For information on how to modify settings, manage users or monitor and reconfigure, refer to the Veritas Flex Appliance Getting Started and Administration Guide.
Choose an Application Container Image
Before you can create an application instance, you must first add the applicable files to the repository and create a tenant. Veritas recommends you create a NIC bond that provides fast performance and fault tolerance in the event of a network adapter failure; however, creation of a NIC bond is optional.
1. Log in to the Flex Appliance web interface, click on Repository on the left menu and click Add Image to upload the NetBackup software from the Veritas Support Site to the Flex repository
2. Click Network interfaces and then click on Create bond. Choose two 10 Gbps NICs to create a NIC bond and enter the Netmask, Gateway and MTU information in the window. Then click Save.
17 3. Click on Tenant and Add tenant. Enter the Tenant name, Label, Network configuration, Search domains and Name servers or Hosts file entries.
Create an Instance on a Flex Appliance
With containerization, it takes only three steps to create a NetBackup instance on Flex Appliances. (See Figure 12.)
Figure 12. An overview of the process of creating a NetBackup instance on a Flex Appliance.
1. Click on System Topology and then click on Create instance. At the select application Window, choose an application such as Primary Service 8.2.
2. Create a NetBackup primary instance by entering the Hostname, Netmask interface, Default gateway, IP address, Tenant, disk size and NetBackup license key.
3. Click Next to deploy the instance. You can check the status and logs.
Once you’ve created a NetBackup primary and media service container, you need to use the NetBackup Java GUI to build the NetBackup domain. For details on building the domain, see the Veritas NetBackup Documentation.
18 19 MONITORING
AutoSupport and Call Home
Veritas AutoSupport is a set of infrastructures, processes and systems that enhance the support experience through proactive monitoring of Veritas Appliance hardware and software. AutoSupport also provides automated error reporting and support case creation. AutoSupport correlates the Call Home data with other site configuration data held by Veritas for technical support and error analysis. With AutoSupport, Veritas greatly improves the customer support experience.
Call Home provides information regarding appliance component states and status. Call Home is enabled by default. For more information about AutoSupport and Call Home, see the Veritas Appliance AutoSupport 2.0 Reference Guide .
Monitoring Hardware from the Flex Appliance Shell
Use the Flex Appliance Shell to obtain information about hardware components. This interface provides tab-completed items to monitor your Flex Appliance. Before you configure the appliance and your network, the system command is available to provide hardware monitoring information. For more information, see Accessing and using the Flex Appliance Shell.
CONCLUSION
Veritas Flex Appliances bring the agility, efficiency and security of container technology to NetBackup data protection. You can run multiple NetBackup deployments on a single Flex Appliance and create new deployments and upgrade in minutes. Flex Appliances efficiently tier to the cloud with NetBackup CloudCatalyst containers. Flex Appliances’ ease of use lets you quickly respond in a rapidly changing business environment.
REFERENCES
• Flex Appliance Product Documents: https://sort.veritas.com/DocPortal/pdf/130821112-136840843-1
• NetBackup Product Documents: https://sort.veritas.com/documents/doc_details/nbu/8.2/Windows%20and%20UNIX/Documentation
VERSIONSFlex Version Date Author Key Updates 1.4 May 2020 Rachel Zhu Original document 2.0 Oct 2020 Rachel Zhu • Immutable storage server • Application service monitoring and failover • Fibre Channel • Supportability—Easy access and log download
ABOUT VERITAS
Veritas Technologies is a global leader in data protection and availability. Over 50,000 enterprises—including 87 percent of the Fortune Global 500—rely on us to abstract IT complexity and simplify data management. The Veritas Enterprise Data Services Platform automates the protection and orchestrates the recovery of data everywhere it lives, ensures 24/7 availability of business-critical applications, and provides enterprises with the insights they need to comply with evolving data regulations. With a reputation for reliability at scale and a deployment model to fit any need, Veritas Enterprise Data Services Platform supports more than 800 different data sources, over 100 different operating systems, more than 1,400 storage targets, and more than 60 different cloud platforms. Learn more at www.veritas.com. Follow us on Twitter at @veritastechllc.
2625 Augustine Drive, Santa Clara, CA 95054 For specific country offices and contact +1 (866) 837 4827 numbers, please visit our website. www.veritas.com www.veritas.com/company/contact
Copyright © 2020 Veritas Technologies LLC. All rights reserved. Veritas, the Veritas Logo, and NetBackup are trademarks or registered trademarks of Veritas Technologies LLC or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. V1111 11/20