Manage Scanning Tasks Over REST
You can trigger, cancel, and monitor scanning tasks over the REST interface, using the REST endpoint openidm/taskscanner:
You can define a scanning task in a configuration file or directly over the REST interface. For an example of a file-based scanning task, see the file /path/to/openidm/samples/example-configurations/task-scanner/conf/schedule-taskscan_sunset.json.
The following command defines a scanning task named sunsetTask over REST:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Content-type: application/json" \ --header "Accept-API-Version: resource=2.0" \ --request PUT \ --data '{ "enabled" : true, "type" : "simple", "repeatInterval" : 3600000, "persisted": true, "concurrentExecution" : false, "invokeService" : "taskscanner", "invokeContext" : { "waitForCompletion" : false, "numberOfThreads" : 5, "scan" : { "_queryFilter" : "((/sunset/date lt \"${Time.now}\") AND !(${taskState.completed} pr))", "object" : "managed/user", "taskState" : { "started" : "/sunset/task-started", "completed" : "/sunset/task-completed" }, "recovery" : { "timeout" : "10m" } }, "task" : { "script" : { "type" : "text/javascript", "file" : "script/sunset.js" } } } }' \ "http://localhost:8080/openidm/scheduler/job/sunsetTask"{ "_id": "sunsetTask", "enabled": true, "persisted": true, "recoverable": false, "misfirePolicy": "fireAndProceed", "schedule": null, "repeatInterval": 3600000, "repeatCount": -1, "type": "simple", "invokeService": "org.forgerock.openidm.taskscanner", "invokeContext": { "waitForCompletion": false, "numberOfThreads": 5, "scan": { "_queryFilter": "((/sunset/date lt \"${Time.now}\") AND !(${taskState.completed} pr))", "object": "managed/user", "taskState": { "started": "/sunset/task-started", "completed": "/sunset/task-completed" }, "recovery": { "timeout": "10m" } }, "task": { "script": { "type": "text/javascript", "file": "script/sunset.js" } } }, "invokeLogLevel": "info", "startTime": null, "endTime": null, "concurrentExecution": false, "triggers": [ { "calendar": null, "group": "scheduler-service-group", "jobKey": "scheduler-service-group.sunsetTask", "name": "trigger-sunsetTask", "nodeId": null, "previousState": null, "serialized": { "type": "SimpleTriggerImpl", "calendarName": null, "complete": false, "description": null, "endTime": null, "fireInstanceId": null, "group": "scheduler-service-group", "jobDataMap": { "scheduler.invokeService": "org.forgerock.openidm.taskscanner", "scheduler.config-name": "scheduler-sunsetTask", "scheduler.invokeContext": { "waitForCompletion": false, "numberOfThreads": 5, "scan": { "_queryFilter": "((/sunset/date lt \"${Time.now}\") AND !(${taskState.completed} pr))", "object": "managed/user", "taskState": { "started": "/sunset/task-started", "completed": "/sunset/task-completed" }, "recovery": { "timeout": "10m" } }, "task": { "script": { "type": "text/javascript", "file": "script/sunset.js" } } }, "schedule.config": { "enabled": true, "persisted": true, "recoverable": false, "misfirePolicy": "fireAndProceed", "schedule": null, "repeatInterval": 3600000, "repeatCount": -1, "type": "simple", "invokeService": "org.forgerock.openidm.taskscanner", "invokeContext": { "waitForCompletion": false, "numberOfThreads": 5, "scan": { "_queryFilter": "((/sunset/date lt \"${Time.now}\") AND !(${taskState.completed} pr))", "object": "managed/user", "taskState": { "started": "/sunset/task-started", "completed": "/sunset/task-completed" }, "recovery": { "timeout": "10m" } }, "task": { "script": { "type": "text/javascript", "file": "script/sunset.js" } } }, "invokeLogLevel": "info", "startTime": null, "endTime": null, "concurrentExecution": false }, "scheduler.invokeLogLevel": "info" }, "jobGroup": "scheduler-service-group", "jobName": "sunsetTask", "misfireInstruction": 1, "name": "trigger-sunsetTask", "nextFireTime": 1570618094818, "previousFireTime": null, "priority": 5, "repeatCount": -1, "repeatInterval": 3600000, "startTime": 1570618094818, "timesTriggered": 0, "volatility": false }, "state": "NORMAL", "_rev": "000000006751ccf1", "_id": "scheduler-service-group.trigger-sunsetTask" } ], "previousRunDate": null, "nextRunDate": "2019-10-09T10:48:14.818Z" }
To trigger a scanning task over REST, use the execute action and specify the name of the task (effectively the scheduled job name). To obtain a list of task names, you can query the /openidm/scheduler/job endpoint. Note, however, that not all jobs are scanning tasks. Only those jobs that have which have the correct task scanner invokeContext can be triggered in this way.
The following example triggers the sunsetTask defined in the previous example:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/taskscanner?_action=execute&name=sunsetTask"{ "_id": "9f2564c8-193c-4871-8869-6080f374b1bd-2073" }
For scanning tasks that are defined in configuration files, you can determine the task name from the file name, for example, schedule-task-name.json. The following example triggers a task named taskscan_sunset that is defined in a file named conf/schedule-taskscan_sunset.json:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/taskscanner?_action=execute&name=taskscan_sunset"{ "_id": "8d7742f0-5245-41cf-89a5-de32fc50e326-3323" }
By default, a scanning task ID is returned immediately when the task is initiated. Clients can make subsequent calls to the task scanner service, using this task ID to query its state and to call operations on it.
To have the scanning task complete before the ID is returned, set the waitForCompletion property to true in the task definition file (schedule-taskscan_sunset.json).
To cancel a scanning task that is in progress, send a REST call with the cancel action, specifying the task ID. The following call cancels the scanning task initiated in the previous example:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request POST \ "http://localhost:8080/openidm/taskscanner/9f2564c8-193c-4871-8869-6080f374b1bd-2073?_action=cancel"{ "_id":"9f2564c8-193c-4871-8869-6080f374b1bd-2073", "status":"SUCCESS" }
Note
You cannot cancel a scanning task that has already completed.
To retrieve a list of scanning tasks, query the openidm/taskscanner context path. The following example displays all scanning tasks, regardless of their state:
curl \ --header "X-OpenIDM-Username: openidm-admin" \ --header "X-OpenIDM-Password: openidm-admin" \ --header "Accept-API-Version: resource=1.0" \ --request GET \ "http://localhost:8080/openidm/taskscanner?_queryFilter=true"{ "result": [ { "_id": "9f2564c8-193c-4871-8869-6080f374b1bd-2073", "name": "schedule/taskscan_sunset", "progress": { "state": "COMPLETED", "processed": 0, "total": 0, "successes": 0, "failures": 0 }, "started": "2017-12-19T11:45:53.433Z", "ended": "2017-12-19T11:45:53.438Z" }, { "_id": "b32aafe5-b484-4d00-89ff-83554341f321-9970", "name": "schedule/taskscan_sunset", "progress": { "state": "ACTIVE", "processed": 80, "total": 980, "successes": 80, "failures": 0 }, "started": "2017-12-19T16:41:04.185Z", "ended": null } ] ... }
Each scanning task has the following properties:
_idThe unique ID of that task instance.
nameThe name of the scanning task, determined by the name of the schedule configuration file or over REST when the task is executed.
startedThe time at which the scanning task started.
endedThe time at which the scanning task ended.
progressThe progress of the scanning task, summarised in the following fields:
failures- the number of records not able to be processedsuccesses- the number of records processed successfullytotal- the total number of recordsprocessed- the number of processed recordsstate- the current state of the task,INITIALIZED,ACTIVE,COMPLETED,CANCELLED, orERROR
The number of processed tasks whose details are retained is governed by the openidm.taskscanner.maxcompletedruns property in the conf/system.properties file. By default, the last one hundred completed tasks are retained.