Best practice for liveSync when syncing Identity Cloud to DS

Last updated Jan 19, 2023

The purpose of this article is to provide best practice advice on using liveSync in ForgeRock Identity Cloud when syncing Identity Cloud to DS using the LDAP connector.

Warning

Do not compress, tamper with, or otherwise alter changelog database files directly unless specifically instructed to do so by a qualified ForgeRock technical support engineer. External changes to changelog database files can render them unusable by the server. By default, changelog database files are located under the /path/to/ds/changelogDb directory.

liveSync best practice

There are two approaches to using liveSync in Identity Cloud with DS:

Use the changelog. See Changelog for Notifications for further information on the changelog. This is the recommended setup.
Use the timestamp mechanism, although this has limitations as discussed in the Timestamp mechanism section.

Requirements

When using liveSync in Identity Cloud, you should observe the following requirements:

Do not have a load balancer in front of your DS servers

DS servers use replication, which means they may not be exactly in sync at any one time, which is why Identity Cloud should only communicate with a single DS server not a load balancer. See On Load Balancers for further information.

Configure your LDAP connector for high availability

The LDAP connector in Identity Cloud should point to the primary DS server. You can then configure secondary DS servers for failover as described here: How do I configure the LDAP connector in Identity Cloud or IDM (All versions) for LDAP failover?

Ensure you are running current supported versions of DS

Aside from it being good practice to stay on supported versions of software, there is a known issue with older DS versions and liveSync: OPENDJ-7286 (Changelog searches can start with incorrect cursors). One manifestation of this issue is that it can stop identities syncing when you have a mixture of Identity Cloud and IDM instances syncing with DS during a migration phase.

This issue is resolved in DS 6.5.4 and later, so it is recommended you are using at least DS 6.5.4. You should also check the EOSL Policy to ensure you remain supported: ForgeRock End of Service Life (EOSL) Policy and EOSL Dates | AM, DS and IDM.

If you cannot upgrade your DS servers, there are two potential workarounds to consider:

Use the timestamp mechanism instead for liveSync operations, but be aware that delete operations are not synchronized.
Add one or more DS 6.5.4 or later servers to the existing deployment and configure the LDAP connector to communicate with the newer DS servers. See When Adding New Servers (DS 7.x) or To Add a New Replica to an Existing Topology (DS 6.5.x) for further information. If you plan to have this mixed version deployment running for an extended period of time, it is preferable to add DS 6.5.x servers rather than DS 7.x servers due to the changes in replication functionality introduced in DS 7.

Timestamp mechanism

Timestamps are maintained per entry for create and modify operations; however, delete operations cannot be detected via timestamps. If delete synchronization is a high priority, you should continue to use the changelog for liveSync.

You can configure liveSync to use timestamps instead of the changelog as follows:

In the Identity Cloud admin UI, go to Native Consoles > Identity Management > Configure > Connectors > [LDAP Connector].
Expand the Additional Options section on the Details tab.
Enable the Use Timestamp Attributes for LiveSync option and click Save.
Reset the sync token using the REST API as described in How do I reset the liveSync syncToken in Identity Cloud or IDM (All versions)?

Caution

You should thoroughly test liveSync with the timestamp mechanism in your Staging environment first to ensure it meets your needs. Using liveSync with timestamps can potentially cause performance issues if the DS instance is managing millions of entries; the timestamp search can have a high cost when there are many entries.

liveSync retry policy

By default, Identity Cloud has a liveSync retry policy configured that retries the failed update five times, and then logs and ignores the failure. See Configure the liveSync retry policy for further information.

If required, you can adjust the retry policy as follows:

In the Identity Cloud admin UI, go to Native Consoles > Identity Management > Configure > Connectors > [LDAP Connector].
Navigate to the Sync tab and update the LiveSync retry attempt(s) and Action to perform after retry attempts fields as required.
Click Save.

Related Training

N/A