How To
ForgeRock Identity Platform
Does not apply to Identity Cloud

How do I use the AWS snapshot feature to quickly create DS 6.x instances?

Last updated Jan 11, 2023

The purpose of this article is to provide information on using the Amazon Web Services™ (AWS™) snapshot feature to quickly provision new DS instances from an existing running server and add them to the replication topology. Although this article is specific to the AWS snapshot feature, you can use the same concepts with other instantaneous copy systems such as OpenZFS snapshot.

Prerequisites

The following process assumes you already have two DS instances installed and running (Master 1 and Master 2), where both instances were installed using the --instancePath setup option. This option allows the setup to separate the command line tools and runtime libraries from the instances data, instance libraries, configuration and log files. See How do I install DS (All versions) so that the instance files are separate to the install files? for further information.

Both Master 1 (opendj-source.forgerock.com) and Master 2 use the same setup install and instance paths:

Master 1 Installation Path: /opt/instances/opendj Instance Path: /opt/instances/opendjdata Master 2 Installation Path: /opt/instances/opendj Instance Path: /opt/instances/opendjdata

Using the AWS snapshot feature to create DS instances

Warning

You cannot use this process to upgrade a server to a new version; you must upgrade as detailed in the Installation Guide.

This example process refers to the following DS instances:

  • Master 1 - this is the source instance and has the following hostname: opendj-source.forgerock.com
  • Master 2 - this is the other existing instance.
  • Master 3 - this is the new instance you are creating and has the following hostname: opendj-new.forgerock.com

You can create new DS instances as follows:

  1. Select the Master 1 instance to use in the snapshot process; this is the snapshot source (opendj-source.forgerock.com).
  2. Stop the Master 1 instance; it is crucial that you stop the DS instance before using the AWS snapshot feature as this guarantees the backend databases have been properly shut down and the lock region/data is properly recorded; failing to do so can corrupt backend databases.
  3. Wait 5-10 seconds after the instance has shut down to allow the system to finish syncing the data to disk; failing to do so can corrupt the changelogDB file.
  4. Use the AWS snapshot feature to instantly copy the Master 1 instance, system. This snapshot is used as the template for the AWS newly provisioned (auto-scaled) system.
  5. Restart the Master 1 instance.
  6. Use the AWS auto-scaling system to provision the new system (Master 3) based on your snapshot; do not start this new Master 3 instance yet.
  7. Remove the entire installation path from the new Master 3 instance to prepare for the new configuration: $ rm -rf /opt/instances/opendj
  8. Remove all but the /opt/instances/opendjdata/db and /opt/instances/opendjdata/changelogDb directories from the instance path for the new Master 3 instance: $ cd /opt/instances/opendjdata $ rm -rf bak classes config import-tmp ldif legal-notices lib locks logsYour new Master 3 instance now only has the following elements leftover from the source instance (Master 1): opendj; instances/$ ls -la opendj ls: opendj: No such file or directory opendj; instances/$ ls -la opendjdata total 0 drwxr-xr-x 4 opendj opendj 136 Jun 9 11:51 . drwxr-xr-x 66 opendj opendj 2244 Jun 9 11:50 .. drwxr-xr-x 5 opendj opendj 170 Jun 9 10:53 changelogDb drwxr-xr-x 3 opendj opendj 102 Jun 9 09:49 dbYou are now ready to install and set up the new Master 3 instance.
  9. Install DS in the instances path of the new Master 3 instance, for example: $ cd /opt/instances $ unzip DS-5.5.0.zip $ cd opendj
  10. Set up the new Master 3 instance with no backend configuration using the setup command. For example:$ ./setup --instancePath /opt/instances/opendjdata --ldapPort 1389 --adminConnectorPort 4444 --rootUserDN "cn=Directory Manager" --rootUserPassword password --hostname opendj-new.forgerock.com --enableStartTLS --ldapsPort 1636 --acceptLicenseThe Master 3 instance is set up so that:
    • The /opt/instances/opendj directory contains all the command line tools and runtime libraries, for example: opendj; cd /opt/instances/opendj opendj; opendj/$ ls -l total 88 drwxr-xr-x@ 3 opendj opendj 102 Jan 8 16:30 QuickSetup.app -rw-r--r--@ 1 opendj opendj 1801 Jan 8 16:22 README drwxr-xr-x@ 3 opendj opendj 102 Jan 8 16:30 Uninstall.app drwxr-xr-x@ 31 opendj opendj 1054 Jan 8 16:30 bat drwxr-xr-x@ 35 opendj opendj 1190 Jun 9 12:38 bin -rw-r--r-- 1 opendj opendj 278 Jun 9 12:04 install drwxr-xr-x@ 5 opendj opendj 170 Jan 8 16:30 legal-notices drwxr-xr-x@ 66 opendj opendj 2244 Jan 8 16:30 lib -rw-r--r--@ 1 opendj opendj 6501 Jan 8 16:22 opendj_logo.png -rwxr-xr-x@ 1 opendj opendj 1838 Jan 8 16:22 setup -rw-r--r--@ 1 opendj opendj 2504 Jan 8 16:22 setup.bat drwxr-xr-x@ 3 opendj opendj 102 Jan 8 16:30 snmp drwxr-xr-x@ 11 opendj opendj 374 Jan 8 16:30 template -rwxr-xr-x@ 1 opendj opendj 1875 Jan 8 16:22 uninstall -rw-r--r--@ 1 opendj opendj 2109 Jan 8 16:22 uninstall.bat -rwxr-xr-x@ 1 opendj opendj 1754 Jan 8 16:22 upgrade -rw-r--r--@ 1 opendj opendj 1840 Jan 8 16:22 upgrade.bat
    • The /opt/instances/opendjdata directory contains the database backend, the changelogDb, instance libraries, logs etc, for example: opendj; cd /opt/instances/opendjdata opendj; opendjdata/$ ls -l total 0 drwxr-xr-x 2 opendj opendj 68 Jun 9 12:12 bak drwxr-xr-x 5 opendj opendj 170 Jun 9 10:53 changelogDb drwxr-xr-x 2 opendj opendj 68 Jun 9 12:12 classes drwxr-xr-x 26 opendj opendj 884 Jun 9 12:12 config drwxr-xr-x 3 opendj opendj 102 Jun 9 09:49 db drwxr-xr-x 2 opendj opendj 68 Jun 9 12:12 import-tmp drwxr-xr-x 2 opendj opendj 68 Jun 9 12:12 ldif drwxr-xr-x 3 opendj opendj 102 Jun 9 12:12 legal-notices drwxr-xr-x 3 opendj opendj 102 Jun 9 12:12 lib drwxr-xr-x 9 opendj opendj 306 Jun 9 12:12 locks drwxr-xr-x 7 opendj opendj 238 Jun 9 12:12 logs
Note

The Instance libraries (/opt/instances/opendjdata/lib) can be used for installing support patches that should only exist for this instance.

  1. Initialize (pre-warm) the EBS volume using the following command: $ fio --filename=/dev/xvdg --rw=randread --bs=256k --iodepth=64 --ioengine=libaio --direct=1 --name=volume-initialize --output=/tmp/prefetch-summary.logThis step can take a long time but is necessary to prevent performance issues in the new Master 3 instance due to the way in which AWS builds a snapshot. AWS builds a snapshot by copying the data to S3 behind the scenes; once the snapshot is applied to an EBS volume, the data in S3 is lazily loaded as each data block is loaded.
  2. Check the status of the new Master 3 instance; you will notice -No LDAP Databases Found- under Data Sources: $ ./status --bindDN "cn=Directory Manager" --bindPassword password --- Server Status --- Server Run Status: Started Open Connections: 1 --- Server Details --- Host Name: opendj-new.forgerock.com Administrative Users: cn=Directory Manager Installation Path: /opt/instances/opendj Instance Path: /opt/instances/opendjdata Version: ForgeRock Directory Services 5.5.0 Java Version: 1.8.0_45 Administration Connector: Port 4444 (LDAPS) --- Connection Handlers --- Address:Port : Protocol : State -------------:------------------------:--------- -- : LDIF : Disabled 0.0.0.0:161 : SNMP : Disabled 0.0.0.0:1389 : LDAP (allows StartTLS) : Enabled 0.0.0.0:1636 : LDAPS : Enabled 0.0.0.0:1689 : JMX : Disabled 0.0.0.0:8080 : HTTP : Disabled --- Data Sources --- -No LDAP Databases Found-
  3. Add any implementation-specific configuration such as password policies, timeouts and so on.
  4. Add schema to the instance either by using ldapmodify or copying the schema across.
  5. Create the backend configuration using the existing /opt/instances/opendjdata path: $ ./dsconfig create-backend --set base-dn:dc=forgerock,dc=com --set enabled:true --set db-directory:/opt/instances/opendjdata/db --type je --backend-name userRoot --hostname opendj-new.forgerock.com --port 4444 --trustAll --bindDN "cn=Directory Manager" --bindPassword password --no-prompt
Note

The backend database is immediately available for use, but is not yet replicated. Do not change database entries in the backend until replication has been fully enabled.

You may notice the following errors in the DS error log when the backend is created; these can be safely ignored:

[09/Jun/2016:12:24:16 -0600] category=PLUGGABLE severity=NOTICE msgID=org.opends.messages.backend.513 msg=The database backend userRoot containing 1000 entries has started [09/Jun/2016:12:24:16 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.724 msg=The search filter "(|(objectClass=subentry)(objectClass=ldapSubentry))" used by subentry manager is not indexed in backend userRoot. Backend initialization for subentry manager processing might take a very long time to complete [09/Jun/2016:12:24:17 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.660 msg=The search filter "(&(|(objectClass=groupOfNames)(objectClass=groupOfUniqueNames)(objectClass=groupOfEntries))(!(objectClass=ds-virtual-static-group)))" used by group implementation cn=Static,cn=Group Implementations,cn=config is not indexed in backend userRoot. Backend initialization for this group implementation might take a very long time to complete [09/Jun/2016:12:24:17 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.660 msg=The search filter "(objectClass=ds-virtual-static-group)" used by group implementation cn=Virtual Static,cn=Group Implementations,cn=config is not indexed in backend userRoot. Backend initialization for this group implementation might take a very long time to complete [09/Jun/2016:12:24:17 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.660 msg=The search filter "(objectClass=groupOfURLs)" used by group implementation cn=Dynamic,cn=Group Implementations,cn=config is not indexed in backend userRoot. Backend initialization for this group implementation might take a very long time to complete [09/Jun/2016:12:24:17 -0600] category=ACCESS_CONTROL severity=WARNING msgID=org.opends.messages.access_control.96 msg=Backend userRoot does not have a presence index defined for attribute type aci. Access control initialization may take a very long time to complete in this backend

  1. Create any implementation-specific indexes and index configuration.
  2. Check the status of the new Master 3 instance again; you will now see details under Data Sources: --- Server Status --- Server Run Status: Started Open Connections: 1 --- Server Details --- Host Name: opendj-new.forgerock.com Administrative Users: cn=Directory Manager Installation Path: /opt/instances/opendj Instance Path: /opt/instances/opendjdata Version: ForgeRock Directory Services 5.5.0 Java Version: 1.8.0_45 Administration Connector: Port 4444 (LDAPS) --- Connection Handlers --- Address:Port : Protocol : State -------------:------------------------:--------- -- : LDIF : Disabled 0.0.0.0:161 : SNMP : Disabled 0.0.0.0:1389 : LDAP (allows StartTLS) : Enabled 0.0.0.0:1636 : LDAPS : Enabled 0.0.0.0:1689 : JMX : Disabled 0.0.0.0:8080 : HTTP : Disabled --- Data Sources --- Base DN: dc=example,dc=com Backend ID: userRoot Entries: 1000 Replication:
  3. Restart the new Master 3 instance; this will clear up any errors previously seen in the DS error log.
  4. Add this new Master 3 instance to the replication topology:
    • Log in to the Master 1 system that was used as the snapshot source.
    • Enable replication from the Master 1 instance to the new Master 3 instance using the dsreplication command. For example:$ ./dsreplication configure --adminUid admin --adminPassword password --baseDn dc=example,dc=com --host1 opendj-source.forgerock.com --port1 4444 --bindDn1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 8989 --host2 opendj-new.forgerock.com --port2 6444 --bindDn2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 10989 --trustAll --no-promptThe newly provisioned DS instance (Master 3) is now ready to be placed into the load balancing pool.

Decommissioning a DS instance

The above process can be used to easily auto-provision new DS instances and add them to the replication topology. When removing an unused instance, you must first disable replication on the DS instance to be decommissioned before the AWS system can be deprovisioned. Failing to do so will leave configuration elements in the remaining servers which will cause errors for commands such as dsreplication status and may affect replication/replication performance.

You can disable replication as follows:$ ./dsreplication unconfigure --unconfigureAll --hostname opendj-newX.forgerock.com --port 4444 --adminUID admin --adminPassword password --trustAll --no-prompt

See Also

How do I install DS (All versions) so that the instance files are separate to the install files?

Creating an Amazon EBS Snapshot

Auto Scaling Groups

Scaling the Size of Your Auto Scaling Group

Initializing Amazon EBS Volumes

Related Training

N/A

Related Issue Tracker IDs

N/A
Copyright and Trademarks Copyright © 2023 ForgeRock, all rights reserved.