Manage scanning tasks using REST

You can trigger, cancel, and monitor scanning tasks over the REST interface, using the REST endpoint openidm/taskscanner.

Create a scanning task

You can define a scanning task in a configuration file or directly over the REST interface. For an example of a file-based scanning task, refer to the file /path/to/openidm/samples/example-configurations/task-scanner/conf/schedule-taskscan_sunset.json.

The following command defines a scanning task named sunsetTask:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Content-type: application/json" \
--header "Accept-API-Version: resource=2.0" \
--request PUT \
--data '{
  "enabled" : true,
  "type" : "simple",
  "repeatInterval" : 3600000,
  "persisted": true,
  "concurrentExecution" : false,
  "invokeService" : "taskscanner",
  "invokeContext" : {
    "waitForCompletion" : false,
    "numberOfThreads" : 5,
    "scan" : {
      "_queryFilter" : "/sunset/date lt \"$\{Time.now}\") AND !(/sunset/task-completed pr",
      "object" : "managed/user",
      "taskState" : {
        "started" : "/sunset/task-started",
        "completed" : "/sunset/task-completed"
      },
      "recovery" : {
        "timeout" : "10m"
      }
    },
    "task" : {
      "script" : {
        "type" : "text/javascript",
        "file" : "script/sunset.js"
      }
    }
  }
}' \
"http://localhost:8080/openidm/scheduler/job/sunsetTask"
{
  "_id": "sunsetTask",
  "enabled": true,
  "persisted": true,
  "recoverable": false,
  "misfirePolicy": "fireAndProceed",
  "schedule": null,
  "repeatInterval": 3600000,
  "repeatCount": -1,
  "type": "simple",
  "invokeService": "org.forgerock.openidm.taskscanner",
  "invokeContext": {
    "waitForCompletion": false,
    "numberOfThreads": 5,
    "scan": {
      "_queryFilter": "/sunset/date lt \"$\{Time.now}\") AND !(/sunset/task-completed pr",
      "object": "managed/user",
      "taskState": {
        "started": "/sunset/task-started",
        "completed": "/sunset/task-completed"
      },
      "recovery": {
        "timeout": "10m"
      }
    },
    "task": {
      "script": {
        "type": "text/javascript",
        "file": "script/sunset.js"
      }
    }
  },
  "invokeLogLevel": "info",
  "startTime": null,
  "endTime": null,
  "concurrentExecution": false,
  "triggers": [
    {
      "calendar": null,
      "group": "scheduler-service-group",
      "jobKey": "scheduler-service-group.sunsetTask",
      "name": "trigger-sunsetTask",
      "nodeId": null,
      "previousState": null,
      "serialized": {
        "type": "SimpleTriggerImpl",
        "calendarName": null,
        "complete": false,
        "description": null,
        "endTime": null,
        "fireInstanceId": null,
        "group": "scheduler-service-group",
        "jobDataMap": {
          "scheduler.invokeService": "org.forgerock.openidm.taskscanner",
          "scheduler.config-name": "scheduler-sunsetTask",
          "scheduler.invokeContext": {
            "waitForCompletion": false,
            "numberOfThreads": 5,
            "scan": {
              "_queryFilter": "/sunset/date lt \"$\{Time.now}\") AND !(/sunset/task-completed pr",
              "object": "managed/user",
              "taskState": {
                "started": "/sunset/task-started",
                "completed": "/sunset/task-completed"
              },
              "recovery": {
                "timeout": "10m"
              }
            },
            "task": {
              "script": {
                "type": "text/javascript",
                "file": "script/sunset.js"
              }
            }
          },
          "schedule.config": {
            "enabled": true,
            "persisted": true,
            "recoverable": false,
            "misfirePolicy": "fireAndProceed",
            "schedule": null,
            "repeatInterval": 3600000,
            "repeatCount": -1,
            "type": "simple",
            "invokeService": "org.forgerock.openidm.taskscanner",
            "invokeContext": {
              "waitForCompletion": false,
              "numberOfThreads": 5,
              "scan": {
                "_queryFilter": "/sunset/date lt \"$\{Time.now}\") AND !(/sunset/task-completed pr",
                "object": "managed/user",
                "taskState": {
                  "started": "/sunset/task-started",
                  "completed": "/sunset/task-completed"
                },
                "recovery": {
                  "timeout": "10m"
                }
              },
              "task": {
                "script": {
                  "type": "text/javascript",
                  "file": "script/sunset.js"
                }
              }
            },
            "invokeLogLevel": "info",
            "startTime": null,
            "endTime": null,
            "concurrentExecution": false
          },
          "scheduler.invokeLogLevel": "info"
        },
        "jobGroup": "scheduler-service-group",
        "jobName": "sunsetTask",
        "misfireInstruction": 1,
        "name": "trigger-sunsetTask",
        "nextFireTime": 1570618094818,
        "previousFireTime": null,
        "priority": 5,
        "repeatCount": -1,
        "repeatInterval": 3600000,
        "startTime": 1570618094818,
        "timesTriggered": 0,
        "volatility": false
      },
      "state": "NORMAL",
      "_rev": "000000006751ccf1",
      "_id": "scheduler-service-group.trigger-sunsetTask"
    }
  ],
  "previousRunDate": null,
  "nextRunDate": "2019-10-09T10:48:14.818Z"
}

Trigger a scanning task

To trigger a scanning task over REST, use the execute action and specify the name of the task (effectively the scheduled job name). To obtain a list of task names, you can query the /openidm/scheduler/job endpoint. Note, however, that not all jobs are scanning tasks. Only those jobs that have which have the correct task scanner invokeContext can be triggered in this way.

The following example triggers the sunsetTask defined in the previous example:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/taskscanner?_action=execute&name=sunsetTask"
{
  "_id": "9f2564c8-193c-4871-8869-6080f374b1bd-2073"
}

For scanning tasks that are defined in configuration files, you can determine the task name from the file name, for example, schedule-task-name.json. The following example triggers a task named taskscan_sunset that is defined in a file named conf/schedule-taskscan_sunset.json:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/taskscanner?_action=execute&name=taskscan_sunset"
{
  "_id": "8d7742f0-5245-41cf-89a5-de32fc50e326-3323"
}

By default, a scanning task ID is returned immediately when the task is initiated. Clients can make subsequent calls to the task scanner service, using this task ID to query its state and to call operations on it.

To have the scanning task complete before the ID is returned, set the waitForCompletion property to true in the task definition file (schedule-taskscan_sunset.json).

Cancel a scanning task

To cancel a scanning task that is in progress, send a REST call with the cancel action, specifying the task ID. The following call cancels the scanning task initiated in the previous example:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/taskscanner/9f2564c8-193c-4871-8869-6080f374b1bd-2073?_action=cancel"
{
  "_id":"9f2564c8-193c-4871-8869-6080f374b1bd-2073",
  "status":"SUCCESS"
}

You cannot cancel a scanning task that has already completed.

List the scanning tasks

To retrieve a list of scanning tasks, query the openidm/taskscanner context path. The following example displays all scanning tasks, regardless of their state:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/taskscanner?_queryFilter=true"
{
  "result": [
    {
      "_id": "9f2564c8-193c-4871-8869-6080f374b1bd-2073",
      "name": "schedule/taskscan_sunset",
      "progress": {
        "state": "COMPLETED",
        "processed": 0,
        "total": 0,
        "successes": 0,
        "failures": 0
      },
      "started": "2017-12-19T11:45:53.433Z",
      "ended": "2017-12-19T11:45:53.438Z"
    },
    {
      "_id": "b32aafe5-b484-4d00-89ff-83554341f321-9970",
      "name": "schedule/taskscan_sunset",
      "progress": {
        "state": "ACTIVE",
        "processed": 80,
        "total": 980,
        "successes": 80,
        "failures": 0
      },
      "started": "2017-12-19T16:41:04.185Z",
      "ended": null
    }
  ]
  ...
}

Each scanning task has the following properties:

_id

The unique ID of that task instance.

name

The name of the scanning task, determined by the name of the schedule configuration file or over REST when the task is executed.

started

The time at which the scanning task started.

ended

The time at which the scanning task ended.

progress

The progress of the scanning task, summarized in the following fields:

`failures`	The number of records not able to be processed.
`successes`	The number of records processed successfully.
`total`	The total number of records.
`processed`	The number of processed records.
`state`	The current state of the task, `INITIALIZED`, `ACTIVE`, `COMPLETED`, `CANCELLED`, or `ERROR`.

The number of processed tasks whose details are retained is governed by the openidm.taskscanner.maxcompletedruns property in the conf/system.properties file. By default, the last 100 completed tasks are retained.

IDM 7.3.1

Manage scanning tasks using REST

Create a scanning task

Trigger a scanning task

Cancel a scanning task

List the scanning tasks