Unix Automation Tools More on cron
From "Migration",  Access to Wang, March 1994
[ Prior Article ]     [ Return to the Catalog of articles ]     [ Next Article ]

Continuing the discussion from last month, let's look more at the tools and techniques for non-interactive (background) program operation under Unix.

More on cron

The Unix cron facility allows routine jobs to be set up and run in background. It is similar in many ways to job scheduling software available for the VS from third-party suppliers. Cron is not the actual queue, like the VS Non-Interactive Task Queue; it is merely a means of scheduling activities.

Like many other Unix tools, cron uses a text table (the crontab file) as its data file. This table lists the run time and date and the job to be executed at that time. Cron files are usually located in /usr/lib/cron or /usr/spool/cron.

The file or files involved with cron vary with different versions of Unix. Berkeley (BSD) Unix versions use a single cron table that is shared by all users; AT&T versions use a separate file for each user. The AT&T approach allows greater access to end users for background operations, since they can create and maintain their own background schedule.

Setting up cron jobs on AT&T-based Unix systems involves the following steps:

1. Assume the identity of the person who will run the background job. (AT&T Unix systems only.)

2. Create or modify a cron file.

3. Register the changed cron file with the system by running the crontab utility.

The first step is important but often overlooked: cron uses the profile of the owner of the crontab file to execute the specified job, and file or directory access problems may come up if the permissions are not correctly set. The crontab file should either be edited by the end user or a system administrator using the su (switch user) command. (BSD systems add the user ID to the crontab file as an additional field.)

Each line of a crontab file contains the minute, hour, day of the month, month, weekday, and the statements to be executed, with each element separated by one or more spaces. (On BSD systems a field containing the user ID the jobs is to be run under is added after the weekday and before the program statements.) Since there are no labels or other indications of which variables belong where, the field elements must be in exact order. Asterisks are used as wild card characters to indicate all values are acceptable. Multiple values can be used to allow more than one run per day, and ranges of values can be indicated as well.

Here are some examples of crontab files:

0,15,30,45 * * * * dsb /usr/bin/foo

(Runs foo every fifteen minutes using the profile of user dsb. (BSD example))

2 7-22 * * * /usr/local/bin/xyzzy -d

(Runs program xyzzy with the -d option at two minutes after every hour, beginning at 7:00 am and ending at 10:00 pm (22:00 using the 24-hour clock). (AT&T example))

15 7 * * 3 /bin/asdf

(Executes asdf at 7:15 am every Wednesday. (AT&T example))

The final step in creating a cron job is to run the program to register the changes, crontab. When executed, crontab will place a copy of your cron file in the system's own directory and update the internal list of jobs to be run. At the time you run crontab you should get a message indicating that the job will be run under the default shell for the system - typically, the Bourne shell. The change is immediate: after registering the new file, the jobs will execute as soon as they are supposed to according to the cron schedule.

On AT&T systems users can be specifically allowed access to cron or denied access to it through the presence of their initials in the cron.allow or cron.deny files. Only one of these files may exist. An empty cron.deny file means that all users may use cron. (As mentioned in last month, the at and batch commands have a similar approach for restricting access, using at.allow or at.deny files.)

Job sequencing

Cron takes responsibility for releasing jobs at the appointed time, but there are many times when jobs must be run in a series, not at the same time as other jobs (in parallel). Examples of this include update programs that make conflicting use of the same files or require a large amount of system resources whenever they are run. Program run sequence can be controlled in the VS Procedure Queue through careful use of background initiators: by setting up only one initiator with a given class, you can guarantee only one job of that class will be run at a time. In similar situations, Unix system administrators must set up the jobs with sufficient time under ordinary circumstances and hope nothing runs longer than anticipated.

A more complex requirement is conditional execution based on the results of a prior process. For example, say some reports should only be allowed to run after successful performance of a posting process. The report programs should wait until successful completion of the posting process before running. Most Unix system administrators set up the jobs and hope nothing fails. Another method is to launch following jobs from the scripts that must precede them, but this is not a flexible approach to batch scheduling.

No standard Unix utility provides such conditional execution functions, but several commercial vendors offer scheduling products that work well for these purposes. Most of these tools migrated from mainframe environments and still carry mainframe prices, so be prepared to spend a lot for a job control system. You might also want to look into the jobque system by Byron Ford, which sets up a series of jobs and runs them after checking return codes. Jobque is available from various sources on the Internet or through the Unix forum on CompuServe.

When things go wrong

It's relatively easy to get background jobs set up in Unix, but it's harder to tell if they performed as you expected. Some ways to get this visibility include logs and mail messages to the user or system administrator. As in every system environment, Unix job accounting information must be added to the process by the programmer.

Job logging is relatively easy to set up under AT&T Unix systems through cron logging. Such logs record the time and resources of processed jobs and must be reviewed and purged frequently.

User accounting systems can also give some of this visibility. All versions of Unix have an accounting system that records system resources and their use. Such systems log many aspects of jobs as they are run, but also produce large log files that must be processed and archived or removed. In commercial environments, the value of these systems must be weighed against the system resources they require in operation.

Neither of these logging systems give much information about a process when it has ended abnormally. In absence of the VS Operator's Console and its messages requesting assistance, Unix mail may be the next choice. Scripts can be set up to send mail to interested users whenever a job ends with an error or simply to notify the user whenever it has been run. Here's an example:

echo "The BIGJOB batch run FAILED" | mail dsb

This shows how a single-line message might be mailed notifying you of a program event. A more sophisticated approach might be as follows:

echo "Subject: BIGJOB run failure" > test.message echo "" >> test.message echo "BIGJOB failed during its run." >> test.message echo "Date and time: `date`" >> test.message mail dsb < test.message rm test.message

In this example, a small text file is created using the echo command and appending lines to a file. A subject line has been added, and the mail system will show this as the subject when the message is delivered. The date and time the message was created will be found in the second line by executing the date command. This produces a more readable message but requires more effort to set up. Mail messages also take time to be delivered and many users will not take the effort to look for them.

Add batch jobs to your environment

In spite of some of the limitations of its tools for batch scheduling, Unix administration requires more routine jobs that should be automated. Some of these routine log reporting, file purges, and many other maintenance tasks. Unix system administrators will find more reason to add non-interactive processes in their work to reduce some of the tedious and error- prone elements of their craft. Proficiency in the Unix batch tools is an important tool for programmers and system administrators.

Unix Bookshelf

UNIX Programmer's Reference
John J. Valley
Que Corporation, Carmel, Indiana; 1991
ISBN 0-88022-536-X

This book fits between a basic introduction to Unix and a full treatise on systems programming. It begins with simple shell programming examples and progresses through program maintenance, filters, serial communications, operating system internals, and multiprocessing programs. Good C and Unix shell reference sections.