Configure the LiveSync Retry Policy

If a liveSync operation fails, IDM reattempts the change an infinite number of times until the change is successful. This behavior can increase data consistency in the case of transient failures (for example, when the connection to the database is temporarily lost). However, in situations where the cause of the failure is permanent (for example, if the change does not meet certain policy requirements) the change will never succeed, regardless of the number of attempts. In this case, the infinite retry behavior can effectively block subsequent liveSync operations from starting.

To avoid this, you can configure a liveSync retry policy to specify the number of times a failed modification should be reattempted, and what should happen if the modification is unsuccessful after the specified number of attempts.

Generally, a scheduled reconciliation operation will eventually force consistency. However, to prevent repeated retries that block liveSync, restrict the number of times that the same modification is attempted. You can then specify what happens to failed liveSync changes. The failed modification can be stored in a dead letter queue, discarded, or reapplied. Alternatively, an administrator can be notified of the failure by email or by some other means. This behavior can be scripted. The default configuration in the samples provided with IDM is to retry a failed modification five times, and then to log and ignore the failure.

The liveSync retry policy is configured in the connector configuration file (provisioner.openicf-*.json). The sample connector configuration files have a retry policy defined as follows:

"syncFailureHandler" : {
  "maxRetries" : 5,
  "postRetryAction" : "logged-ignore"
},

maxRetries

Specifies the number of attempts that IDM should make to process the failed modification.

The value of this property must be a positive integer, or -1. A value of zero indicates that failed modifications should not be reattempted. In this case, the post-retry action is executed immediately when a liveSync operation fails. A value of -1 (or omitting the maxRetries property, or the entire syncFailureHandler from the configuration) indicates that failed modifications should be retried an infinite number of times. In this case, no post retry action is executed.

The default retry policy relies on the scheduler, or whatever invokes liveSync. Therefore, if retries are enabled and a liveSync modification fails, IDM will retry the modification the next time that liveSync is invoked.

postRetryAction

Indicates what should happen if the maximum number of retries has been reached (or if maxRetries has been set to zero). The post-retry action can be one of the following:

logged-ignore
IDM should ignore the failed modification, and log its occurrence.
dead-letter-queue
IDM should save the details of the failed modification in a table in the repository (accessible over REST at repo/synchronisation/deadLetterQueue/provisioner-name).
script
Specifies a custom script that should be executed when the maximum number of retries has been reached. For information about using custom scripts in the configuration, see Scripting Function Reference. In addition to the regular objects described in that section, the following objects are available in the script scope:
syncFailure
Provides details about the failed record. The structure of the syncFailure object is as follows:
```
"syncFailure" :
  {
    "token" : the ID of the token,
    "systemIdentifier" : a string identifier that matches the "name" property in
                         provisioner.openicf.json,
    "objectType" : the object type being synced, one of the keys in the
                   "objectTypes" property in provisioner.openicf.json,
    "uid" : the UID of the object (for example uid=joe,ou=People,dc=example,dc=com),
    "failedRecord", the record that failed to synchronize
  },    
```
To access these fields, include syncFailure.fieldname in your script.
failureCause
Provides the exception that caused the original liveSync failure.
failureHandlers
Two synchronization failure handlers are provided by default:
loggedIgnore indicates that the failure should be logged, after which no further action should be taken.
deadLetterQueue indicates that the failed record should be written to a specific table in the repository, where further action can be taken.
To invoke one of the internal failure handlers from your script, use a call similar to the following (shown here for JavaScript):
```
failureHandlers.deadLetterQueue.invoke(syncFailure, failureCause);
```

The following sample provisioner configuration file extract shows a liveSync retry policy that specifies a maximum of four retries before the failed modification is sent to the dead letter queue:

...
    "syncFailureHandler" : {
        "maxRetries" : 4,
        "postRetryAction" : dead-letter-queue
    },
...

In the case of a failed modification, a message similar to the following is output to the log file:

INFO: sync retries = 1/4, retrying

IDM reattempts the modification the specified number of times. If the modification is still unsuccessful, a message similar to the following is logged:

INFO: sync retries = 4/4, retries exhausted
Jul 19, 2013 11:59:30 AM
    org.forgerock.openidm.provisioner.openicf.syncfailure.DeadLetterQueueHandler invoke
INFO: uid=jdoe,ou=people,dc=example,dc=com saved to dead letter queue

The log message indicates the entry for which the modification failed (uid=jdoe, in this example).

You can view the failed modification in the dead letter queue, over the REST interface, as follows:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/repo/synchronisation/deadLetterQueue/ldap?_queryFilter=true&_fields=_id"
{
  "result":
    [
      {
        "_id": "4",
        "_rev": "000000001298f6a6"
      }
    ],
  ...
}

To view the details of a specific failed modification, include its ID in the URL:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/repo/synchronisation/deadLetterQueue/ldap/4"
{
  "objectType": "account",
  "systemIdentifier": "ldap",
  "failureCause": "org.forgerock.openidm.sync.SynchronizationException:
            org.forgerock.openidm.objset.ConflictException:
            org.forgerock.openidm.sync.SynchronizationException:
            org.forgerock.openidm.script.ScriptException:
            ReferenceError: \"bad\" is not defined.
            (PropertyMapping/mappings/0/properties/3/condition#1)",
  "token": 4,
  "failedRecord": "complete record, in xml format"
  "uid": "uid=jdoe,ou=people,dc=example,dc=com",
  "_rev": "000000001298f6a6",
  "_id": "4"
}

Note

The repo endpoint is an internal interface, as described in "ForgeRock Stability Label Definitions". Although it is used in the preceding example for the purposes of demonstration, you should not rely on this endpoint in production.