Scheduling Jobs

Scheduling jobs allows you to create routine workflows to simplify data transfers between Agents. You can create jobs that run automatically in sequence by using the API to set job frequency, failure behavior, and priority.

Overview

When generating a job using the REST API, you can include Schedule job arguments that set when and how often a job runs.

Each Schedule attribute sets how the job processes:

• Schedule.timezone - which time zone to use for the scheduled job in tz format (Required)
• Schedule._sp_frequency - how often the job runs (See Job Frequency)
• Schedule._sp_start_at - when the job should start running in YYYY/MM/DD HH:MM:SS format
• Schedule._sp_interrupt_on_failure - sets the job's failure state (See Job Failure)
• Schedule.priority - sets job priority (See Job Priority)
• Schedule.finishBefore - sets granular job priority order (See Job Priority)

Include specific Schedule arguments within the jobArgs object within the call body. For example, the following job will start in the America/Toronto time zone on May 20, 2019, at 7:10 AM. It will run daily at 7:10 AM:

    "jobArgs" : {
                    "ObjectUploader.Schedule.timezone":      "America/Toronto",
                    "ObjectUploader.Schedule._sp_start_at":  "2019/05/20 07:10:00",
                    "ObjectUploader.Schedule._sp_frequency": "DAILY",
                    "ObjectUploader.Schedule._sp_interrupt_on_failure": "no",
                    "ObjectUploader.Schedule.priority": "",
                    "ObjectUploader.Schedule.finishBefore": "",
                    ...
                }

Job Frequency

Jobs can be configured to run at a specified time and frequency. Job runs use Schedule._sp_frequency and Schedule._sp_start_at job arguments to determine a job's first run and any subsequent runs.

Ordinals

You can specify jobs to run on a particular weekday within a specific month, such as the second Thursday or the last Friday of a month:

    "jobArgs" : {
                    ...
                    "ObjectUploader.Schedule._sp_frequency": "SECOND MONDAY",

                    ...
                }

You can set Schedule._sp_frequency to use:

• FIRST / 1ST
• SECOND/2ND
• THIRD/3RD
• FOURTH/4TH
• LAST

Intervals

You can specify that jobs run at specified intervals, such as every five hours, or every two weeks:

    "jobArgs" : {
                    ...
                    "ObjectUploader.Schedule._sp_frequency": "2 WEEKS",

                    ...
                }

You can set Schedule._sp_frequency to use:

• MONTHS / M
• MINUTES / MI
• HOURS / H
• DAYS / D
• WEEKS / W
• YEARS / Y

Job Failure

You can specify how a job behaves if it encounters an error or fails to complete using the Schedule._sp_interrupt_on_failure option.

Setting Schedule._sp_interrupt_on_failure to yes sets the job status to INTERRUPTED and will not continue, preserving the job environment, including workflow component inputs and outputs. Once resumed, the job restarts at the beginning of the workflow component that failed.

Setting Schedule._sp_interrupt_on_failure to no will end a job when it fails and display FAILED status.

Job Priority

Agents using concurrency resource controls limit the number of jobs running at a time, adding any additional jobs to a queue. The Schedule.priority and Schedule.finishBefore arguments determine how to order queued jobs, while jobs without a Schedule.priority or Schedule.finishbefore are processed in the order they are added to the queue.

The Schedule.priority argument sets a job's overall priority level in order to place it in sequence.

You can set Schedule.priority to use:

• LOW / 1
• MEDIUM / 2 (Default)
• HIGH / 3
• CRITICAL / 4
• IMMEDIATE / 5

Note: Schedule.priority accepts strings containing either the specific priority (e.g. LOW) or a numerical value (e.g. 1). Whenever information is retrieved from the Manager API, it is displayed as the specific priority level.

The Schedule.finishBefore argument determines how to queue jobs that have the same priority level, but may need to run in a specific sequence, or sooner than a previously queued job.

Set finishBefore using a date and time in YYYY/MM/DD HH:MM:SS format.

Example

As an example, two jobs are sent to the same resource controlled Agent or job group:

    "jobName" : "example one - without a finishBefore set"
    "jobArgs" : {
                    ...
                    "ObjectUploader.Schedule.priority": "MEDIUM",
                    "ObjectUploader.Schedule.finishBefore": "",
                    ...
                }

        "jobName" : "example two - with a finishBefore set"
        "jobArgs" : {
                    ...
                    "ObjectUploader.Schedule.priority": "MEDIUM",
                    "ObjectUploader.Schedule.finishBefore": "2019/05/21 07:10:00",
                    ...
                }

If both jobs are sent to the Manager, example two will be queued by the Manager first. It has a Schedule.finishBefore set, and so will take priority over the other job.

Job Statistics

After a job run starts, sending POST requests to the /jobs/stats/ endpoint provides specific job statistics for a job run.

Requests to /jobs/stats/{jobname}/{jobgroupname|jobgroupID} show the statistics for the most recent job run, or ​/jobs​/stats​/{jobname}​/{jobgroupname|jobgroupID}​/{runNumber} to request statistics from a specific run.

Note: To view the most recent job run, use 0 as the runNumber.

The request must include a JSON data body containing stats endpoint that includes the statistics to be returned.

Example Request Body

[
  "source_host",
  "target_host",
  "status",
  "file_count",
  "byte_count",
  "files_transferred",
  "files_skipped",
  "files_deleted",
  "failed_files",
  "transfer_rate",
  "transfer_rate_max",
  "transfer_rate_min",
  "file_data_bytes",
  "file_bytes_skipped",
  "total_errors"
]

Statistics Reference

Jobs include the following statistic types: