Solutions

February 2005 - Data Center Relocation

Background:
In December of 2003, the United States General Services Administration Public Buildings Service (GSA/PBS) made a strategic decision to relocate its National Data Center outside of Washinton, DC. Its National Data Center houses the computer systems that are used to manage the Federal Government's real estate inventory, and it was decided that the data center should be moved in order to provide greater security to this environment.

Solution:
ECS Technology worked with GSA/PBS to identify a site for its new data center in Northern Virginia. In addition to finding a location, ECS also focused on facility design. ECS provided computer room and office space designs to architects and construction engineers and worked with them to finalize the specifications of the facility. Special attention was paid to high availability features in the design. This included power requirements, air conditioning and data communications capabilities.

Present and future power requirements were calculated in order to adequately size the UPS required to support the facility. A backup power generator was also incorporated into the design as an alternative power source. Air conditioning requirements were then calculted in order to properly cool the computer equipment. Air handlers were designed with N+1 capability menaing that one of the units could be taken out of service for maintenance and adequate cooling would still be provided by the others. Data communications services were provided by two different vendors in order to provide redundancy.

After the construction of the facility, ECS also coordinated the move of the data center from Washington, DC to its new home in Northern Virginia. The data center relocation was completed throughout the entire month of February 2005. The quality assurance and development labs were moved on subsequent weekends during the month. The approach for moving these labs was to simply power down the computer equipment and move it to its new location over the couse of the weekend. The production compter room was moved over final weekend of the month. EMC Time finder technology was used to replicate mission critical applications to the new data center in order to minimize downtime. Other applications were moved by powering down the servers and relocating them to the new data center over the weekend. After relocating the production computer room, GSA/PBS has conducted operations in its new data center since February 2005.

August 2004 - Enterprise Backup

Background:
The United States General Services Administration Public Buildings Service (GSA/PBS) previously used a de-centralized architecture for backing up all of the servers located in its Data Center. The architecture consisted of several backup servers with small DLT tape libraries connected to them. Each backup server was responsible for backing up a subset of servers within the Data Center. Full backups were performed twice weekly (one set for off-site) and incremental backups were performed on all other nights.

Growth in the Data Center, both in terms of the number of servers and the amount of data contained on each server, made this architecture increasingly difficult to manage. This growth began to significantly increase the amount of time required to perform the backup routine each night resulting in less time available to perfrom other operational duties. Also, having to perform the same managment tasks at multiple backup servers reduced staff efficiency by having to spend more time on these tasks than necessary.

Solution:
ECS Technology teamed with engineers from EMC to design an enterprise backup soltuion for GSA/PBS. In order to reduce the amount of time required for the backup process, a 'backup to disk' architecture was adopted. This architecture was also supplemented by a robotic tape library in order to handle off-site tape storage requirements and other data requests. An ADIC Pathlight server along with an EMC CLARiiON disk array were chosen to perform the 'backup to disk'. An ADIC Scalar i2000 LTO tape library was chosen as the tape library component and server backup software from Legato was chosen to manage the entire solution. Server backups are now performed directly to disk storage devices instead of slower traditional tape devices. This change significantly reduced that amount of time required to perform the backup routine each night. Further, off-site storage tapes are now created by moving data directly from the EMC CLARiiON to the ADIC tape library thereby eliminating the need to perform a second set of full backups during the week. By introducing speed and efficiency into their backup routine, GSA/PBS was able to increase server and application availability for their customers.

April 2002 - EMC TimeFinder Solution

Background:
GSA/PBS currently refreshes operational data from their production billing system to their decision support system each night by exporting data from the production billing server and, in turn, loading it into the decision support server. This export/import process already takes several hours during the night and, as the amount of data in the database grows, is only expected to increase. The problem is that the decision support server also acts as the gateway to provide billing data to other mission-critical GSA/PBS applications. Given the time it currently takes to refresh data on the decision support server, there is not enough time left during the night to provide data to supporting applications before start-of-business that morning.

Solution:
EMC TimeFinder tools were used to produce a mirrored copy of the disks that house the production billing database. The resulting set of mirrored disks is called a Business Continuance Volume, or BCV. As the database changes on the production disks during the day, only the corresponding changes need to be written to the BCV disks in order to keep the database current. After the BCV is synchronized each evening, it is immediately made available to the decision support server. From there, gateway processes can provide data to GSA/PBS supporting applications with plenty of time to spare before start-of-business that morning. This method provides a substantial time savings over exporting and importing the data in its entirety each night. The BCV architecture will refresh the decision support server in approximately eight minutes instead of the three hours that it currently takes.

December 2001 - Disaster Recovery / Business Continuance

Background:
GSA/PBS runs its production billing application from its Service Center located in Washington, DC. In an effort to continue business operations in the event that an emergency situation causes the Service Center to become unavailable, GSA/PBS has renewed its focus on providing adequate disaster recovery / business continuance services for its production billing application.

Solution:
GSA/PBS contracted NCR Corporation in Dayton, OH to provide the facility and servers that would be used to run the production billing application in the event of an emergency. ECS Technology develops, maintains, and tests the methodology required to rebuild and run the production billing application in Dayton if necessary. Mirrored boot disks with the operating system already preloaded were built to facilitate bringing the servers online much faster than loading the operating system from scratch. Data communication lines were preinstalled between the NCR facility in Dayton and the GSA data communications infrastructure. A tape backup rotation was established between the GSA Service Center in Washington, DC and the NCR backup facility in Dayton so that current versions of the application and database will be available if needed. Automated processes were developed to configure and format the disk space required to run the billing system. Additional procedures were developed to recover the application and its associated database from the tape backups to the appropriate file systems on the disk drives. Database log files were used to roll the data forward in time from the backup to provide a more current environment for users. ECS Technology also works with GSA/PBS to practice these procedures on a regular basis. System and database administrators perform a full run-through at least once every three months to ensure that GSA/PBS will be prepared to continue business operations in the event of an emergency.

June 2000 - Extranet Billing System

Background:
Each month GSA/PBS runs its production rent billing process, which essentially bills government agencies for space that they rent in federally-owned or-leased buildings. In the past GSA/PBS produced a printed hard copy of the rent bills and mailed them to each customer. This process was both expensive and time-consuming for PBS. Customers also found the hard-copy bills difficult to store, manage, and maintain over a long period of time.

Solution:
"Rent on the Web" is an Internet browser-based system that was developed for GSA/PBS to provide electronic rent bills to their customer agencies. Both Intranet and Extranet versions of the system were developed. The Intranet version of the system ran on the GSA network and gave GSA employees the ability to see and query rent bills for all of their customers. Additionally, the Extranet version of the system was opened to the Internet and allowed each customer agency to login and retrieve its own rent bill in electronic format. Customers could use the system to query specific portions of their rent bill based on various selection criteria. The system also provided customers with the raw data that was used to generate their rent bill. This gave them the ability to perform trend analysis on the cost of space and the amount of space being rented over time. GSA/PBS was also able to realize substantial time and cost savings by not having to produce and distribute hard-copy rent bills to each agency.

Comments