Safety First! Develop a backup strategy for your system
From "Special Report",  Access to Wang, August 1990
[ Prior Article ]     [ Return to the Catalog of articles ]     [ Next Article ]

Simply stated, backup consists of copies of critical system elements necessary to continue operation of the organization in the event that the normal processing channels are unavailable. This is typically interpreted to mean tape or disk copies of systems files that are made at predetermined times and set aside, but other interpretations are also valid - such as the disk mirroring options available to users of Local Area Networks or the complete system redundancy offered by high-end mainframes.

My purpose here is to outline some of the considerations involved in the backup process and the options available to implement that process. This necessarily involves issues of file retention, emergency processing, and the accessibility of the system to users - since the backup process typically requires exclusive access to the system. The goal of this analysis is the formulation of a backup plan.

In referring the frequency that these copies are made, I will mention days - the most common backup frequency - even though it is probable that more or less frequent backups are necessary for your conditions.

Goals of a Backup Plan

Before determining the backup process, the possible uses of that backup should be considered. Backup data can be used to provide data processing capability in the event of complete destruction of the computer facility, but many less severe conditions also require use of a backup. Production errors - such as an incorrect date entry in a purge process - are far more likely uses, as are temporary "simulations" of prior periods for reporting purposes. Routine disk initialization also requires complete and accurate backup copies.

While an exhaustive study of emergency processing issues is beyond the scope of this article, such considerations should be present when formulating a backup plan. How long will it take to restore the data? How long to resume normal processing after a crash? And how frequently should backup copies be made? How accessible are backup copies of the data? Are these copies complete and correct? All of these and other questions must be addressed before the backup plan is enacted.

The ideal backup plan should:

• Ensure data integrity by providing adequate retention for important files and testing the integrity of those files. It should also provide a means for disk maintenance and verification.

• Provide data for emergency processing, covering the data requirements for a recovery plan. The plan should model alternate processing approaches and provide test strategies.

• Improve disk performance by placing files in optimum positions on the disk and reorganizing disk catalogs.

Backup Administration

Once the plan is created, administrative elements must be considered. These include scheduling, staffing requirements, the timing and scheduling of routine maintenance, and the subsequent impact to system availability and service levels of all of this activity.

System administrators are often caught between the need for user access to the system and the goals of adequate backup and maintenance. Usually this means that the backup process is squeezed between times of user activity - traditionally, late at night. This type of schedule makes it difficult to hire, monitor, and retain good personnel. Fortunately, the time required for the backup process has been decreased and large-capacity backup media are making operator intervention during backup less necessary.

Types of Backup

Approaches to the backup process include full, incremental, and cycle plans. Each has advantages and drawbacks that must be considered in the complete backup plan.

With the full backup approach, all files are copied every time, regardless of whether they have been modified. Restoring data from this type of backup is the simplest, since every necessary file is present on any copy. The length of backup time and need for many volumes to contain the data are two principal drawbacks - though the emerging popularity of high-volume cartridge tape systems has negated most of these objections.

With the incremental approach, all files are copied at set intervals, with files changed since the last full backup copied every day. This lessens the time required for daily backup but makes restoration more difficult. To restore from an incremental backup, the last full backup must be copied and all daily incrementals overlaid up to the last valid day; this process can considerably increase the time required to restore data.

The cycle backup approach is similar to an incremental backup, except that files that change infrequently are divided into groups and backed up in a regular cycle. It eliminates the need to perform full backups by covering these static files over a predetermined backup cycle. Restoration from this backup approach amounts to successively restoring files from the beginning of the last cycle until the last valid backup - a process that depends on the number of days in the cycle. This process is best used in installations where the amount of time required for a full backup would significantly hinder production.

Most installations use a combination of the approaches mentioned to meet their organization's coverage needs. Such combinations might include: an incremental approach for most times, with full backups at the end or beginning of the accounting calendar; frequent incremental backups during the week and a separate weekend full backup, or other strategies.

Backup Strategies

I use the term backup strategy to refer to the approach you take to the backup process based on the usage of your system. The coverage you need depends on how much data on your system changes, how frequently it changes, and likely reasons for access to earlier versions of the data. By understanding the ways your system is used and the possible reasons for restoring data to it, you can select the manner in which the data is gathered and the medium used to record it.

Consider a simple example: a batch-oriented Purchasing system. In this hypothetical system, users enter information into transaction files and update master files every few days. There are no ties to other applications on the system. The only files that change daily (and, therefore, need daily backup) are the transaction files; all other files change less and might be backed up less frequently. Naturally, unchanged files must still be backed up on a known, consistent schedule. In this case, a sequential backup system (e.g. tape) and an incremental backup approach would be adequate.

Restoring this system might amount to copying the last full backup and overlaying the most recent incremental backup for recently-changed files. Since the full backup would contain the bulk of the files, the time required to restore from successive incremental backups would be less significant.

Most installations are more complex than this example. Systems are often integrated, so data changes made in a Purchasing system are likely to update Inventory and Accounts Payable files as well. In other words, an incremental backup is likely to pick up most of the data files on the system every time - perhaps most of the storage requirements of the system. In this case, full backups might be more viable than incrementals, since the time saved by not copying unchanged files might be overcome by the time required to determine whether the file is to be included in the incremental backup. Restoring data from a full backup would also be simpler and probably take less time. And sequential backup media would be suitable, since you would usually need to restore all of the system rather than portions of it.

Large document-processing shops have a very different situation: a multitude of files that change little, with a handful that change every day. Full backups might cover the large number of documents that have not changed, but changed documents are more likely to need restoration to cover user mistakes or other situations. In such conditions, the number of files that change and the likelihood that these files might need to be restored both go up. The backup frequency and speed of access become concerns, since the files must be accessed randomly for restoration.

The critical need for large Word Processing shops is rapid access to those documents most recently changed. This indicates that some part of the backup approach must be timely and access must be random. In addition, the entire catalog must be accessible relatively quickly. For such situations, a random-access medium (disk, optical disk) might be best, supplemented by routine full backups performed by lower-cost sequential media. Depending on the access speed deemed important, it may also be necessary for additional coverage - such as a disk-based document backup and archiving system that detects editing changes immediately and retains all prior versions.

Unfortunately, few systems fit into either of these extremes. It is more often necessary to tailor the backup approach and even the medium to specific systems. Conflicting access and service level goals between classes of users may mean that the best alternative is a compromise.

Implementing the Backup Plan

Once the goals and approach are decided, the plan must be implemented. For many shops, this amounts to careful planning and scheduling, the use of a appropriate backup medium, and Wang's BACKUP utility. If your needs are complicated, the traditional backup setup - Wang's BACKUP utility and reel-to-reel tape - may not be sophisticated enough. In any case, it is worthwhile to consider the tools of backup in relation to the goals of the backup plan.

BACKUP is by far the most common program used for the backup process. BACKUP offers universal access on all VS systems, preservation of File Descriptor data such as modification date, owner ID, and special access rights, and the ability to reliably span multiple disks or tapes. It uses a file orientation, assembling disparate portions of files into contiguous wholes during output, and ambiguous file elements (wild cards) can be used for the input or output specification. BACKUP produces a log file and an optional report, and either can be stored on the output volume or any other volume specified by the user. File conflicts are noted on the operator's screen and the backup log, and the files bypassed.

Drawbacks to BACKUP include its non-standard tape output (non-labelled tapes), tape headers that are large and space-consuming, and little internal verification of the file's integrity. While a log is produced by BACKUP, its use is limited to the BACKUP program itself; it cannot provide rapid answers to queries about a file's location. The skipping of open files is a valid approach, but BACKUP will not return later and try to get these files, as some other products do. BACKUP will copy files within similarly-named libraries that lie across multiple volumes, but cannot automatically restore these files to multiple volumes. Finally, BACKUP is considerably slower than some third-party offerings.

In response to these limitations, a small crop of third-party backup software has sprouted. Most of the claims made by these products center around transfer speed, but some also offer other features - such as the ability to restore to multiple output volumes or automatic maintenance of file location files for query purposes.

Actually, you may not have a choice of backup software if you use any of the new-generation backup devices, such as 8mm VCR tape cartridge units (the Exebyte format) or Digital Audio Tape (DAT). At present, all of these units use proprietary interfaces that require their own software for operation. This can be an issue if your emergency plan calls for your files to be restored at a site with another brand of tape cartridge unit. Be sure to consider this before purchasing such units.

Backup Hardware Choices

The types of equipment used to perform backups has changed radically in the last few years. The use of high-density 9-track (reel-to-reel) tape systems has increased, due to cost reductions and vendor competition. At the same time, removable magnetic disk options have remained stagnant: the only new removable drive of significance introduced recently is the 76-megabyte RSD, and 288-megabyte SMD drives remain the only large-capacity removable disk. Optical disks have made an appearance, but failed to gain wide acceptance due to slower transfer rates, non-standard data format, and limited availability. Finally, the appearance of an assortment of high-capacity cartridge tape systems has overtaken the market, causing many to reevaluate their backup approach.

Each medium has individual advantages and constraints, and must be evaluated in terms of meeting the goals of the backup process. Some of the considerations in choosing a medium include:

• Access type: The choice of random of sequential access devices depends on the possible reasons for a restoration of data. For last-resort backup coverage sequential units are fine, but random access is necessary if the files are be restored selectively and with great speed.

• Transfer rate: The speed of information transfer impacts the amount of time required to perform the backup and to restore from that backup. The medium must be able to perform both operations within the requirements of the backup plan and likely emergency processing scenarios.

• Storage capacity: The amount of information that can be retained on a single volume - and, thus, whether additional volumes must be used - is a critical element in staffing and scheduling. operator intervention may be required to mount additional volumes as each is filled.

• Special characteristics: Unique abilities, such as: use of WORM (Write Once Read Many) optical disk drives to prevent modification of historical data; conversion of documents to MS-DOS format and transmission to off-site storage for emergency processing via PCs; and other special considerations.

• Support: Vendor support for the device, including alternative support choices (if desired).

• Media compatibility: Ability for backup media to be read and used in other computing environments, as in the use of standard ANSI- or IBM-labelled tapes so other systems could read and use the data.

• Purchase and support costs: The purchase price and support costs of the medium, including periodic maintenance and likely service interval between failures.

• Media cost: The cost of individual volumes, coupled with the number required to provide adequate backup generations.

Other Items for Critical Attention

The backup plan must address many elements and strike a balance between complete system access and data safety. Other elements of the backup process not covered here include the specifics of retention strategies, including identification of critical information; file location logging and inquiry; the physical health of the storage systems, including bad block detection and VTOC verification; elimination of disk fragmentation through disk compression or routine full restores; detection of errors in indexes or alternate index chains within files; the storage conditions of the backup volumes; and the security of backup copies from unauthorized theft or destruction. Finally, the backup plan must be thoroughly tested and subjected to regular review for relevancy and completeness.

Careful attention to the backup process can help ensure that, as guardians of data integrity, we are not surprised by events beyond our reckoning without possibility of recovery.

Figure 1: Other Software for Backup

Here are some Wang utilities and Useraids that have some application to the backup process or data verification:

• VOLCOPY: Wang's fast backup utility that moves physical disk areas from one disk to another disk (not tape). Both disks must be of the same size, and should be error-free for best results. No files can be open on either volume. Offered as an option from the BACKUP utility - for the few shops that can take advantage of it.

• TAPECOPY: Another Wang utility that creates tape copies of files. Primarily used when a industry-standard recording format is needed, since it can produce labelled and non-labelled tapes using either the ASCII or EBCDIC character sets. Input files must be specified explicitly; i.e., no wild cards, volume or library references, or other selection criteria. TAPECOPY opens the files using INPUT mode, so the input files must also be opened INPUT or not at all.
  
  Files restored using TAPECOPY lose their modification history, owner ID, and special access privileges - unless these parameters are added back from a log file or other external source. Some third-party vendors of backup utilities and a few Useraids depend on TAPECOPY to perform the transfer to tape and construct front-end programs to perform the necessary selection, logging, and restoration processes.

• COPY: Wang's general disk copy utility is similar in capabilities to TAPECOPY, but it can accept full library and volume specifications. COPY has been enhanced to allow SHARED as well as INPUT access modes, and the file can be locked during the copy operation; these options must be stated before attempting to access the file or a correction screen will appear. Like TAPECOPY, COPY is used by some third-party software and Useraids.

• SELCOPY: A Useraid that allows selection of input files based on dates, file types, owner ID, and other parameters. Selected files are copied to the specified output volume (using COPY or TAPECOPY) and a report is prepared of the results. Could benefit from additional GETPARMs (the program is largely interactive) and provision for wild-card file or library specification.

• COMPARE: A variety of file comparison tools are available for those suspicious of the integrity of a copy. Among them, Wang's COMPARE is the best for interactive review. COMPARE offers several types of file comparison, including a SUMMARY option suited for quick review of backup copies.

• SUPCOPY: A Useraid that, like SELCOPY, allows selection of files or libraries and then uses COPY or TAPECOPY to perform the copies. Files or libraries are selected by name only, and the output files can be reorganized. The order of the output files can be specified, and a report is produced.

• XFERMANY: A Useraid that acts as a front-end to the Wang TRANSFER utility. Selected files or entire libraries are sent to a designated target system over a Wang network. The destination files cannot be renamed, and there is no verification that the file was properly received. A good introduction to the possibilities of this type of backup, but not practical for daily backup use due to a lack of GETPARMs and poor features.

• TAPEDUMP: A former Useraid, now supported by Wang Laboratories beginning with the 7.20 operating system. Dumps tape sectors in ASCII or EBCDIC for verification and review. Can also be used to check the format of unknown tapes created on other systems.

Figure 2: Glossary of Backup Terms

Term	Definition
Back up; backup	The process (back up; verb) or products (BACKUP; noun) of a routine procedure to copy vital information for safekeeping.
Contention	Access conflicts, as when two tasks attempt exclusive access to a file.
Disk and tape labels	Standard means of identifying a medium, recorded in the same manner as the data itself.
Disk mirroring	The use of redundant storage such as disks to simultaneously record information as it is changed on the primary storage system. Presently available in specific Local Area Network products, but the theory could be used on any system.
Generation	A set of backup volumes that represent a point in time. Usually used to refer to the number of back copies preserved.
Logs; logging	Reports or files constructed that show backup activity, usually including file statistics, input and output locations, date of processing, and any errors encountered.
Random access	Direct access to specific items. In data storage, a medium that quickly retrieves selected files. Opposite of sequential access.
Restore	The process of copying backup copies of data onto a system.
Retention	The amount of time a file is to be retained, based on its perceived usefulness to the organization.
Sequential access	Access to individual items along a specific path. In data storage, a medium that retrieves selected files by reading all files, bypassing all that do not apply. Opposite of random access.