Disaster Recovery Data Center Setup for Airlines Client

Client Profile

The client is a major US airline with a market capitalization of over USD 20 billion. It operates a large domestic and international route network, employs over 80,000 people, and is a member of the Star Alliance, the world’s largest airline alliance.

Client Requirements

The client was looking to achieve a working disaster-proof business model and required the creation of a data center at one of their locations by cloning and replicating their existing data centers, which would support Disaster Recovery (DR) and ensure minimal downtime.

Challenges

1) Building a Disaster Recovery Data Center from Scratch

The client required a new disaster recovery (DR) data center to be built from the ground up, replicating their existing infrastructure. This task presented significant logistical and technical challenges, especially ensuring the new setup seamlessly replicates data and functions as an effective backup.

2) Network Performance Bottlenecks

Managing network performance between replication and production systems emerged as a challenge during the DR setup. The risk of bottlenecks during data transfer threatened the efficiency and reliability of the disaster recovery system, potentially leading to downtime or slower recovery processes.

3) Application Bugs with Replication Software

The use of Double-Take® Software, essential for creating a cost-effective DR solution, encountered several issues. Specifically, the software had compatibility problems with network bonding, causing periodic interruptions in the replication process and impacting overall system stability.

4) Testing Failover and Failback Mechanisms

Ensuring the disaster recovery system could handle real-time failovers and failbacks was critical. Any errors in these mechanisms could lead to data loss or extended downtime during a natural disaster. Thorough testing was required, but the process was complex and time-consuming, adding to the challenge.

Solutions

1) Strategic Design and Implementation of the Data Center

Veritis’ Linux engineers designed and built the data center from the ground up, aligning it with the client’s existing infrastructure. This included setting up both physical and virtual servers and installing necessary applications. The new data center mirrored the existing one, ensuring seamless data replication and robust disaster recovery capabilities.

2) Dedicated Network for Replication and Production

A dedicated network was created to address potential network bottlenecks for replication and production tasks. Veritis separated the data traffic between the original and replicated data centers, reducing the risk of performance issues and ensuring minimal downtime.

3) Patching and Bug Fixes for Double-Take® Software

Veritis resolved the issues with Double-Take® Software by applying targeted patches to address the network bonding compatibility problem. The team worked closely with the application developers to fix these bugs, ensuring the replication software operated efficiently and without interruptions.

4) Comprehensive Failover and Failback Testing

Veritis conducted rigorous DR testing, including failover and failback scenarios, to ensure the system’s reliability during a disaster. The testing process was followed by operational tasks such as patching and upgrading the servers, guaranteeing that the data center would perform flawlessly under real-world conditions.

Veritis Approach

The DR project comprised a Linux environment and involved building the data center from ground zero. The Veritis team of Linux engineers worked on designing and implementing networks within the data center.

In addition to setting up the physical and virtual servers, the team installed applications required to begin the replication of the existing data center. Double-Take^® Software was utilized to create a simplified and cost-effective DR architecture.

After installing and testing the applications on the servers, the team tested failovers and failbacks as part of the DR testing in addition to carrying out operational tasks such as patching and upgrading.

Environment / Technologies / Tools

Solaris 10, 11; RHEL 5 – 7; OEL (Oracle Enterprise Linux) Release 6, 7; Veritas Infoscale (VxVM, VCS, VVR, GC); HP c7000/Blade infrastructure, Double-Take^®, Zerto, Infrastructure – Server Hardware (T-Series, Superclusters T-M series, HP-c7000, BL460c)

Benefits to Client

The Veritis team took on both the operations and engineering roles, usually handled by separate teams, thereby enabling significant cost savings for the client.

Other benefits delivered by the project include:

• Faster service delivery
• Zero to minimal downtime
• Reduced roadblocks
• Automation of repetitive, time-intensive tasks
• Increased productivity and efficiency