Managing Reconciliation Over REST

To trigger, cancel, and monitor reconciliation operations over REST, use the openidm/recon REST endpoint. Note that you can perform most of these actions through the Admin UI.

Triggering a Reconciliation

The following example triggers a reconciliation operation over REST based on the systemLdapAccounts_managedUser mapping:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/recon?_action=recon&mapping=systemLdapAccounts_managedUser"

By default, a reconciliation run ID is returned immediately when the reconciliation operation is initiated. Clients can make subsequent calls to the reconciliation service, using this reconciliation run ID to query its state, and to call operations on it. For an example, see "Obtaining the Details of a Reconciliation".

The reconciliation run initiated previously would return something similar to the following:

{
  "_id": "05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144",
  "state": "ACTIVE"
}

To complete the reconciliation operation before the reconciliation run ID is returned, set the waitForCompletion property to true when the reconciliation is initiated:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/recon?_action=recon&mapping=systemLdapAccounts_managedUser&waitForCompletion=true"

Tip

To trigger this reconciliation through the Admin UI, select Configure > Mappings, select a mapping, then select Reconcile.

If you select Cancel Reconciliation before it is complete, you will need to start the reconciliation again.

Canceling a Reconciliation

You can cancel a reconciliation in progress by specifying the reconciliation run ID. The following REST call cancels the reconciliation run initiated in the previous section:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request POST \
"http://localhost:8080/openidm/recon/0890ad62-4738-4a3f-8b8e-f3c83bbf212e?_action=cancel"

The output for a reconciliation cancellation request is similar to the following:

{
    "status":"INITIATED",
    "action":"cancel",
    "_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e"
}

If the reconciliation run is waiting for completion before its ID is returned, obtain the reconciliation run ID from the list of active reconciliations, as described in the following section.

Tip

To cancel a reconciliation run in progress through the UI, select Configure > Mappings, click on the mapping whose reconciliation you want to cancel, and click Cancel Reconciliation.

Listing a History of Reconciliations

Display a list of reconciliation processes that have completed, and those that are in progress, by running a RESTful GET on "http://localhost:8080/openidm/recon".

The following example displays all reconciliation runs:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/recon"

The output is similar to the following, with one item for each reconciliation run:

"reconciliations": [
    {
      "_id": "05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144",
      "mapping": "systemLdapAccounts_managedUser",
      "state": "SUCCESS",
      "stage": "COMPLETED_SUCCESS",
      "stageDescription": "reconciliation completed.",
      "progress": {
        "source": {
          "existing": {
            "processed": 2,
            "total": "2"
          }
        },
        "target": {
          "existing": {
            "processed": 0,
            "total": "0"
          },
          "created": 2,
          "unchanged": 0,
          "updated": 0,
          "deleted": 0
        },
        "links": {
          "existing": {
            "processed": 0,
            "total": "0"
          },
          "created": 2
        }
      },
      "situationSummary": {
        "SOURCE_IGNORED": 0,
        "FOUND_ALREADY_LINKED": 0,
        "UNQUALIFIED": 0,
        "ABSENT": 2,
        "TARGET_IGNORED": 0,
        "MISSING": 0,
        "ALL_GONE": 0,
        "UNASSIGNED": 0,
        "AMBIGUOUS": 0,
        "CONFIRMED": 0,
        "LINK_ONLY": 0,
        "SOURCE_MISSING": 0,
        "FOUND": 0
      },
      "statusSummary": {
        "SUCCESS": 2,
        "FAILURE": 0
      },
      "durationSummary": {
        "sourceQuery": {
          "min": 42,
          "max": 42,
          "mean": 42,
          "count": 1,
          "sum": 42,
          "stdDev": 0
        },
        "auditLog": {
          "min": 0,
          "max": 1,
          "mean": 0,
          "count": 24,
          "sum": 15,
          "stdDev": 0
        },
        "linkQuery": {
          "min": 5,
          "max": 5,
          "mean": 5,
          "count": 1,
          "sum": 5,
          "stdDev": 0
        },
        "targetQuery": {
          "min": 3,
          "max": 3,
          "mean": 3,
          "count": 1,
          "sum": 3,
          "stdDev": 0
        },
        "targetPhase": {
          "min": 0,
          "max": 0,
          "mean": 0,
          "count": 1,
          "sum": 0,
          "stdDev": 0
        },
        "sourceObjectQuery": {
          "min": 6,
          "max": 34,
          "mean": 21,
          "count": 22,
          "sum": 474,
          "stdDev": 9
        },
        "postMappingScript": {
          "min": 0,
          "max": 1,
          "mean": 0,
          "count": 22,
          "sum": 17,
          "stdDev": 0
        },
        "defaultMappingScript": {
          "min": 0,
          "max": 4,
          "mean": 2,
          "count": 22,
          "sum": 48,
          "stdDev": 2
        },
        "sourcePhase": {
          "min": 490,
          "max": 490,
          "mean": 490,
          "count": 1,
          "sum": 490,
          "stdDev": 0
        }
      },
      "parameters": {
        "sourceQuery": {
          "resourceName": "system/ldap/account",
          "queryFilter": "true",
          "_fields": "_id"
        },
        "targetQuery": {
          "resourceName": "managed/user",
          "queryFilter": "true",
          "_fields": "_id"
        }
      },
      "started": "2020-05-07T09:14:57.740Z",
      "ended": "2020-05-07T09:14:58.325Z",
      "duration": 585,
      "sourceProcessedByNode": {}
    }
  ]

In contrast, the Admin UI displays the results of only the most recent reconciliation. For more information, see "Obtaining the Details of a Reconciliation in the Admin UI".

Each reconciliation run includes the following properties:

_id

The ID of the reconciliation run.

mapping

The name of the mapping.

state

The high-level state of the reconciliation run. Values can be as follows:

ACTIVE
The reconciliation run is in progress.
CANCELED
The reconciliation run was successfully canceled.
FAILED
The reconciliation run was terminated because of failure.
SUCCESS
The reconciliation run completed successfully.

stage

The current stage of the reconciliation run. Values can be as follows:

ACTIVE_INITIALIZED
The initial stage, when a reconciliation run is first created.
ACTIVE_QUERY_ENTRIES
Querying the source, target, and possibly link sets to reconcile.
ACTIVE_RECONCILING_SOURCE
Reconciling the set of IDs retrieved from the mapping source.
ACTIVE_RECONCILING_TARGET
Reconciling any remaining entries from the set of IDs retrieved from the mapping target, that were not matched or processed during the source phase.
ACTIVE_LINK_CLEANUP
Checking whether any links are now unused and should be cleaned up.
ACTIVE_PROCESSING_RESULTS
Post-processing of reconciliation results.
ACTIVE_CANCELING
Attempting to abort a reconciliation run in progress.
COMPLETED_SUCCESS
Successfully completed processing the reconciliation run.
COMPLETED_CANCELED
Completed processing because the reconciliation run was aborted.
COMPLETED_FAILED
Completed processing because of a failure.

stageDescription

A description of the stages described previously.

progress

The progress object has the following structure (annotated here with comments):

"progress":{
  "source":{             // Progress on set of existing entries in the mapping source
    "existing":{
      "processed":1001,
        "total":"1001"   // Total number of entries in source set, if known, "?" otherwise
    }
  },
  "target":{             // Progress on set of existing entries in the mapping target
    "existing":{
      "processed":1001,
        "total":"1001"     // Total number of entries in target set, if known, "?" otherwise
    },
    "created":0          // New entries that were created
  },
  "links":{              // Progress on set of existing links between source and target
    "existing":{
      "processed":1001,
        "total":"1001"     // Total number of existing links, if known, "?" otherwise
    },
  "created":0            // Denotes new links that were created
  }
},

You can adjust the number of reconciliation runs that are stored in IDM by adding the maxAnalysisRunsPerMapping and maxNonAnalysisRunsPerMapping properties to your mapping:

"reconAssociation" : {
    "maxAnalysisRunsPerMapping" : 1,
    "maxNonAnalysisRunsPerMapping" : 3
}

In this context, analysis refers to reconciliation runs that are triggered with the analyze=true parameter. These runs don't perform any actions, but determine which actions would be performed in a real reconciliation. Non-analysis refers to a normal reconciliation.

Obtaining the Details of a Reconciliation

Display the details of a specific reconciliation over REST, by including the reconciliation run ID in the URL. The following call shows the details of the reconciliation run initiated in "Triggering a Reconciliation".

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/recon/05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144"
{
  "_id": "05f63bce-4aaa-492e-9e86-a702d5c9d6c0-1144",
  "mapping": "systemLdapAccounts_managedUser",
  "state": "SUCCESS",
  "stage": "COMPLETED_SUCCESS",
  "stageDescription": "reconciliation completed.",
  "progress": {
    "source": {
      "existing": {
        "processed": 2,
        "total": "2"
      }
    },
    "target": {
      "existing": {
        "processed": 0,
        "total": "0"
      },
      "created": 2,
      "unchanged": 0,
      "updated": 0,
      "deleted": 0
    },
    "links": {
      "existing": {
        "processed": 0,
        "total": "0"
      },
      "created": 2
    }
  },
  "situationSummary": {
    "SOURCE_IGNORED": 0,
    "FOUND_ALREADY_LINKED": 0,
    "UNQUALIFIED": 0,
    "ABSENT": 2,
    "TARGET_IGNORED": 0,
    "MISSING": 0,
    "ALL_GONE": 0,
    "UNASSIGNED": 0,
    "AMBIGUOUS": 0,
    "CONFIRMED": 0,
    "LINK_ONLY": 0,
    "SOURCE_MISSING": 0,
    "FOUND": 0
  },
  "statusSummary": {
    "SUCCESS": 2,
    "FAILURE": 0
  },
  "durationSummary": {
    "sourceQuery": {
      "min": 42,
      "max": 42,
      "mean": 42,
      "count": 1,
      "sum": 42,
      "stdDev": 0
    },
    "auditLog": {
      "min": 0,
      "max": 1,
      "mean": 0,
      "count": 24,
      "sum": 15,
      "stdDev": 0
    },
    "linkQuery": {
      "min": 5,
      "max": 5,
      "mean": 5,
      "count": 1,
      "sum": 5,
      "stdDev": 0
    },
    "targetQuery": {
      "min": 3,
      "max": 3,
      "mean": 3,
      "count": 1,
      "sum": 3,
      "stdDev": 0
    },
    "targetPhase": {
      "min": 0,
      "max": 0,
      "mean": 0,
      "count": 1,
      "sum": 0,
      "stdDev": 0
    },
    "sourceObjectQuery": {
      "min": 6,
      "max": 34,
      "mean": 21,
      "count": 22,
      "sum": 474,
      "stdDev": 9
    },
    "postMappingScript": {
      "min": 0,
      "max": 1,
      "mean": 0,
      "count": 22,
      "sum": 17,
      "stdDev": 0
    },
    "defaultMappingScript": {
      "min": 0,
      "max": 4,
      "mean": 2,
      "count": 22,
      "sum": 48,
      "stdDev": 2
    },
    "sourcePhase": {
      "min": 490,
      "max": 490,
      "mean": 490,
      "count": 1,
      "sum": 490,
      "stdDev": 0
    }
  },
  "parameters": {
    "sourceQuery": {
      "resourceName": "system/ldap/account",
      "queryFilter": "true",
      "_fields": "_id"
    },
    "targetQuery": {
      "resourceName": "managed/user",
      "queryFilter": "true",
      "_fields": "_id"
    }
  },
  "started": "2020-05-07T09:14:57.740Z",
  "ended": "2020-05-07T09:14:58.325Z",
  "duration": 585,
  "sourceProcessedByNode": {}
}

Obtaining the Details of a Reconciliation in the Admin UI

You can display the details of the most recent reconciliation in the Admin UI. Select the mapping. In the page that appears, you'll see a message similar to:

Completed: Last reconciled November 20, 2019 15:28

Clicking on the reconciliation run date displays the details of the reconciliation run. Click Reconciliation Results for additional information.

If a reconciliation fails, select the Failure Summary tab to obtain information about the reasons for the failure.

To view reconciliation audit logs in the UI, add an Audit widget to your dashboard. The reconciliation Audit widget shows the same information that you get over REST.

Viewing Reconciliation Association Details

When performing a reconciliation run, information is reconciled between the source object and the target object. This creates an association between the two objects, which can be recorded in IDM by including the persistAssociations=true parameter when triggering a reconciliation. This information can then be retrieved by querying the recon/assoc endpoint.

To get a list of currently stored recon associations, run the following query:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/recon/assoc?_queryFilter=true"
     {
  "result": [
    {
      "_id": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230",
      "_rev": "1",
      "mapping": "managedUser_systemLdapAccounts",
      "sourceResourceCollection": "managed/user",
      "targetResourceCollection": "system/ldap/account",
      "isAnalysis": "false",
      "finishTime": "2019-05-01T23:36:24.434153Z"
    },
    {
      "_id": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-99638",
      "_rev": "1",
      "mapping": "systemLdapAccounts_managedUser",
      "sourceResourceCollection": "system/ldap/account",
      "targetResourceCollection": "managed/user",
      "isAnalysis": "true",
      "finishTime": "2019-05-06T21:31:42.140066Z"
    }
  ],
  "resultCount": 2,
  "pagedResultsCookie": null,
  "totalPagedResultsPolicy": "NONE",
  "totalPagedResults": -1,
  "remainingPagedResults": -1
}

You can also get information for a specific reconciliation by querying the recon ID:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/recon/assoc/da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230"
    {
  "_id": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230",
  "_rev": "1",
  "mapping": "managedUser_systemLdapAccounts",
  "sourceResourceCollection": "managed/user",
  "targetResourceCollection": "system/ldap/account",
  "isAnalysis": "false",
  "finishTime": "2019-05-01T23:36:24.434153Z"
}

It is possible to also get the specific association details of each entry in the reconciliation run by appending /entry to your query:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request GET \
"http://localhost:8080/openidm/recon/assoc/da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230/entry?_queryFilter=true"
    {
  "result": [
    {
      "_id": "400d40fd-da58-41f5-857b-71855eb97bd9",
      "_rev": "0",
      "mapping": "managedUser_systemLdapAccounts",
      "reconId": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230",
      "situation": "CONFIRMED",
      "action": "UPDATE",
      "linkQualifier": "default",
      "sourceObjectId": "07978ba5-b31d-4f8b-9f60-506c07f68495",
      "targetObjectId": "ca8abc7f-7b97-3e96-94fb-6b27b0ec5aed",
      "sourceResourceCollection": "managed/user",
      "targetResourceCollection": "system/ldap/account",
      "status": "SUCCESS",
      "exception": null,
      "message": null,
      "messageDetail": "null",
      "ambiguousTargetObjectIds": null
    },
    ...
    {
      "_id": "08ec633c-744f-4092-b88d-fe253b1d8e52",
      "_rev": "0",
      "mapping": "managedUser_systemLdapAccounts",
      "reconId": "da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230",
      "situation": "CONFIRMED",
      "action": "UPDATE",
      "linkQualifier": "default",
      "sourceObjectId": "ee2449a8-01e6-4c0b-84d3-e65e25c3e38c",
      "targetObjectId": "67a6596e-ebfc-3542-a664-1ab1610e082a",
      "sourceResourceCollection": "managed/user",
      "targetResourceCollection": "system/ldap/account",
      "status": "SUCCESS",
      "exception": null,
      "message": null,
      "messageDetail": "null",
      "ambiguousTargetObjectIds": null
    }
  ],
  "resultCount": 50,
  "pagedResultsCookie": null,
  "totalPagedResultsPolicy": "NONE",
  "totalPagedResults": -1,
  "remainingPagedResults": -1
}

Note

For particularly large reconciliations, the results returned can be quite substantial, since it includes the details of every object reconciled. We encourage using query filters to tune your queries to only return the subset of results you're looking for.

Purging Reconciliation Statistics From the Repository

When the number of completed reconciliation runs for a given mapping reaches the number specified by maxAnalysisRunsPerMapping or maxNonAnalysisRunsPerMapping, statistics are purged automatically. Statistics and reconciliation run information (such as recon associations) are purged chronologically by mapping, with the oldest reconciliation run for that mapping purged first.

You can also manually remove reconciliation statistics. To purge reconciliation statistics from the repository manually, run a DELETE command on the reconciliation run ID. For example:

curl \
--header "X-OpenIDM-Username: openidm-admin" \
--header "X-OpenIDM-Password: openidm-admin" \
--header "Accept-API-Version: resource=1.0" \
--request DELETE \
"http://localhost:8080/openidm/recon/da88b9a5-1fe5-4f8d-a6a8-7e0a2b4e136b-9230"