Generation IDs do not match error after restoring a DS (All versions) replica

Symptoms

When restoring a replica, the following error is shown in the DS error logs:

The restored replica is not synchronized with the other replication servers.

Recent Changes

Restored an old backup or imported LDIF data to a DS instance.

Causes

The generation ID contained in the restored data is not the same as the one in the current replication topology. The generation ID is a checksum of attributes from some entries and is used during replication to check that the suffix being updated is the same as the one offering the updates.

Solution

This issue can be resolved using one of the following approaches:

• Re-initialize the suffix from another replica, since the generation IDs will naturally be in sync.
• Restore a replica as follows if you are certain the data is correct:
  • DS 7 and later: Use the start-disaster-recovery and end-disaster-recovery commands as part of your restore process.
  • Pre-DS 7: Use the pre-external-initialization and post-external-initialization commands as part of your restore process.

Re-initialize the suffix

You can re-initialize the suffix as follows:

• DS 7.1 and later: $ ./dsrepl initialize --bindDN uid=admin --bindPassword password --baseDN dc=example,dc=com --toServer ds-1 --hostname localhost --port 4444 --trustStorePath /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --no-prompt
• DS 7: $ ./dsrepl initialize --bindDN uid=admin --bindPassword password --baseDN dc=example,dc=com --toServer ds-1 --hostname localhost --port 4444 --trustStorePath /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --no-prompt
• Pre-DS 7:  $ ./dsreplication initialize --adminUID admin --adminPassword password --baseDN dc=example,dc=com --hostSource ds1.example.com --portSource 4444 --hostDestination ds2.example.com --portDestination 5444 --trustAll --no-prompt

Use start-disaster-recovery and end-disaster-recovery (DS 7 and later)

If you are certain the data is correct, you can use start-disaster-recovery and end-disaster-recovery commands as part of your restore process. These commands ensure the generation ID of the replicated domain is updated. "Old" changes will not get replayed because they were targeting the data using the previous generation ID.  

You restore a replica as follows:

1. Run the following command to force all replicas' generation IDs for the affected data to an invalid state; this stops replication completely for the affected data:
   • DS 7.1 and later: $ dsrepl start-disaster-recovery --baseDN dc=example,dc=com --hostname ds1.forgerock.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --no-prompt
   • DS 7: $ dsrepl start-disaster-recovery --baseDN dc=example,dc=com --hostname ds1.forgerock.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --no-prompt
2. Restore or import from ldif to restore the data:
   • Restore
   • Import LDIF
3. Run the following command to recalculate all replicas' generation IDs for the affected data; this resumes replication. Ensure you use the same baseDN as in step 1:
   • DS 7.1 and later: $ dsrepl end-disaster-recovery --baseDN dc=example,dc=com --hostname ds1.forgerock.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --no-prompt
   • DS 7: $ dsrepl end-disaster-recovery --baseDN dc=example,dc=com --hostname ds1.forgerock.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --no-prompt

Use pre-external-initialization and post-external-initialization (pre-DS 7)

If you are certain the data is correct, you can use pre-external-initialization and post-external-initialization as part of your restore process. These commands ensure the generation ID of the replicated domain is updated. "Old" changes will not get replayed because they were targeting the data using the previous generation ID.  

You restore a replica as follows:

1. Prepare the domain on all servers for being externally initialized. You must specify the baseDN of the data you are going to be changing, for example:$ ./dsreplication pre-external-initialization --hostname ds1.forgerock.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt
2. Restore or import from ldif to restore the data:
   • Using the restore command, for example: $ ./restore --hostname ds1.forgerock.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backupID 20160614031032Z --backupDirectory /path/to/ds/backupfile
   • Using the import-ldif command, for example: $ ./import-ldif --hostname ds1.example.com --port 4444--includeBranch dc=example,dc=com --backendID userRoot --ldifFile /path/to/ldif_file
3. Enter the following command to set the new generation ID for the domain and broadcast it to all the servers, which allows them to replicate again. Ensure you use the same baseDN as in step 1: $ ./dsreplication post-external-initialization --hostname ds1.forgerock.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt

See Also

How do I restore old backup data to a DS 5.x or 6.x replication topology?

How do I roll back an entire network of DS 5.x or 6.x replicas to a previous backup?

How do I configure DS 5.x or 6.x to ensure accidentally deleted or changed data can be restored when replication is enabled?

Manual Initialization

Backup and Restore

Related Training

ForgeRock Directory Services Core Concepts (DS-400)

Related Issue Tracker IDs

N/A