Installing and Administering DS/OpenDJ

This book provides information on installing and administering DS/OpenDJ, including frequently asked questions and known issues.


General


What versions of DS/OpenDJ are compatible with AM/OpenAM?

The purpose of this article is to provide compatibility information between DS/OpenDJ and AM/OpenAM versions. This includes the embedded DS/OpenDJ versions shipped with AM/OpenAM and, the supported Java® versions for each combination.

DS/OpenDJ and AM/OpenAM compatibility

Embedded DS/OpenDJ

Embedded DS/OpenDJ versions are shipped with AM/OpenAM and cannot be upgraded; however, when you upgrade to a newer version of AM/OpenAM, the embedded DS/OpenDJ instance is also upgraded. The following embedded DS/OpenDJ versions are included with AM/OpenAM:

AM/OpenAM versions Embedded DS/OpenDJ version
AM 6.5.1 DS 6.5.1
AM 6.5 DS 6.5
AM 6 DS 6
AM 5.5 and 5.5.1 DS 5.5
AM 5 and 5.1.x DS 5
OpenAM 13.5.2 OpenDJ 3.5.3
OpenAM 13.5.1 OpenDJ 3.5.2
OpenAM 13.5 OpenDJ 3.5
OpenAM 13.0 OpenDJ 3.0
OpenAM 12.0.4 OpenDJ 2.6.4
OpenAM 12.0.0 to 12.0.3 OpenDJ 2.6.2

External DS/OpenDJ

The following matrix indicates compatibility between AM/OpenAM and external DS/OpenDJ versions:

  DS 6.x DS 5.x OpenDJ 3.x OpenDJ 2.6.x
AM 6.x Yes Yes Yes --
AM 5.5 Yes Yes Yes --
AM 5 and 5.1.x Yes Yes Yes Partly (User Store only)
OpenAM 13.x -- -- Yes Yes
OpenAM 12.x -- -- -- Yes
Note

It is strongly recommended that you always upgrade to the latest maintenance releases for whichever versions of AM/OpenAM and DS/OpenDJ you have deployed.

Java Compatability

Java 11

  • AM 6.5 and later.
  • DS 6.5 and later.

Java 9

  • DS 6 only.

Java 8

  • AM 5 and later; OpenAM 12.0.0 and later
  • DS 5 and later; OpenDJ 2.6.2 and later

Java 7

  • AM 5 and 5.1.x; OpenAM 12.x and 13.x
  • OpenDJ 2.6.x and 3.x. Java 7 can be used in DS 5 but support for it is deprecated.

Java 6

  • OpenAM 12.x
  • OpenDJ 2.6.x

Software and Hardware requirements

AM/OpenAM

DS/OpenDJ

See Also

What operating systems are ForgeRock products supported on?

Checking your product versions are supported

FAQ: Installing AM/OpenAM

FAQ: Upgrading AM/OpenAM

FAQ: Installing and configuring DS/OpenDJ

Installing and configuring AM/OpenAM

Installing and Administering DS/OpenDJ

Related Training

N/A

Related Issue Tracker IDs

N/A


What versions of DS/OpenDJ are compatible with IDM/OpenIDM?

The purpose of this article is to provide compatibility information between DS/OpenDJ and IDM/OpenIDM versions. This includes connecting to DS/OpenDJ as an external resource (LDAP connector), the embedded DS repositories shipped with later versions of IDM and the supported Java® versions for each combination.

External resources compatibility

You can connect to DS/OpenDJ as an external resource using the LDAP connector (the LDAP connector works with any LDAPv3 compliant servers). The following matrix indicates compatibility between IDM/OpenIDM and DS/OpenDJ when using the LDAP connector:

  DS 6.x DS 5.x OpenDJ 3.x
IDM 6.x Yes Yes Yes
IDM 5.x Yes Yes Yes
OpenIDM 4.x Yes Yes Yes
Note

It is strongly recommended that you always upgrade to the latest maintenance releases for whichever versions of IDM/OpenIDM and DS/OpenDJ you have deployed.

Repositories compatibility

Embedded DS

Embedded DS repositories have been shipped with IDM since IDM 5.5; these repositories are for evaluation purposes only and cannot be used in production. The following embedded DS versions are included with IDM:

IDM versions Embedded DS version
IDM 6.5 DS 6.5
IDM 6 DS 6
IDM 5.5 DS 5.5

External DS

IDM 6 introduced a new feature where an external DS instance can now be used as a supported repository in production environments. The following matrix indicates compatibility between IDM and external DS repositories:

  DS 6.5 DS 6 DS 5.x OpenDJ 3.x
IDM 6.5 Yes -- -- --
IDM 6 -- Yes -- --
IDM 5.x -- -- -- --
OpenIDM 4.x -- -- -- --
Note

It is strongly recommended that you always upgrade to the latest maintenance releases for whichever versions of IDM/OpenIDM and DS/OpenDJ you have deployed.

Java Compatability

Java 11

  • IDM 6.5 and later.
  • DS 6.5 and later.

Java 9

  • DS 6 only.

Java 8

  • IDM 5 and later; OpenIDM 4.x
  • DS 5 and later; OpenDJ 3.x

Java 7

  • IDM 5; OpenIDM 4.x
  • OpenDJ 3.x. Java 7 can be used in DS 5 but support for it is deprecated.

Software and Hardware requirements

IDM/OpenIDM

DS/OpenDJ

See Also

What operating systems are ForgeRock products supported on?

Checking your product versions are supported

FAQ: Upgrading IDM/OpenIDM

FAQ: Installing and configuring IDM/OpenIDM

FAQ: Installing and configuring DS/OpenDJ

Administering and configuring IDM/OpenIDM

Installing and Administering DS/OpenDJ

Related Training

N/A

Related Issue Tracker IDs

N/A


Install


How do I install DS/OpenDJ (All versions) so that the instance files are separate to the install files?

The purpose of this article is to provide information on installing DS/OpenDJ so that the instance files (user data) are separate to the install files (binaries). This setup allows you to separate all your backend database files and configuration in a separate file system to your binaries.

Background

The instance.loc file, which is located at the top level of of your unzipped DS/OpenDJ binary directory, indicates where the instance files are stored. The instance.loc file is used by all DS/OpenDJ tools to locate the server data and configuration. How this file is created varies by version as follows:

  • DS 5 and later: the instance.loc file is created during setup if the --instancePath option is used. It must not exist prior to running the setup command, else setup will error. 
  • OpenDJ 3.x: a default file is not provided to prevent it overwriting an amended file; if you require this setup, you must create the instance.loc file.
  • OpenDJ 2.6.x: a default file is provided, which contains the following:
    .
    
    This indicates that the instance files are stored in the same location as the install files. You can amend it as needed.
Note

Any support patches received from ForgeRock will need to be installed in the instance.loc's classes directory rather than the /path/to/ds/classes/ directory.

Installing DS/OpenDJ so that the instance files are separate to the install files

To install DS/OpenDJ so that the binaries are in /opt/ds and the instance files are in /var/ds, you should do the following depending on which version you are using:

DS 5 and later

  1. Download the latest DS zip file from BackStage.
  2. Navigate to the /opt directory (where you want to install DS) and unzip the DS zip file (for example, DS-5.5.0.zip). This creates /opt/ds/...
  3. Run the setup command per Installation Guide › Setting Up a Directory Server. You must include the --instancePath option, which should point to the directory where you want the instance files located. For example:
    $ ./setup --instancePath /var/ds --ldapPort 1389 --adminConnectorPort 4444 --rootUserDN "cn=Directory Manager" --rootUserPassword password --hostname ds1.example.com --acceptLicense
    The setup tool creates the /var/ds directory if it does not already exist and populates this directory with all the data and configuration files needed by the server. Additionally, it creates the instance.loc file.

Pre-DS 5

  1. Download the latest OpenDJ zip file from BackStage.
  2. Navigate to the /opt directory (where you want to install OpenDJ) and unzip the OpenDJ zip file (for example, opendj-3.5.0.zip). This creates /opt/opendj/...
  3. Create the instance.loc file in the /opt/opendj/ directory if it does not exist, else update it and specify the directory where you want the instance files located. For example, enter:
    /var/ds
  4. Run the setup command per OpenDJ Installation Guide › To Install OpenDJ Directory Server From the Command-Line. For example:
    $ ./setup --ldapPort 1389 --adminConnectorPort 4444 --rootUserDN "cn=Directory Manager" --rootUserPassword password --hostname opendj1.example.com --acceptLicense
    The setup tool creates the /var/ds directory if it does not already exist and populates this directory with all the data and configuration files needed by the server. You should ensure the instance.loc file is readable by all the client users and the server.​

See Also

How do I relocate the backend database files in DS/OpenDJ (All versions) on to a separate file system?

FAQ: Installing and configuring DS/OpenDJ

Installation Guide

Reference › File Layout

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-3121 (Setup fails to create the lib/extensions directory in the instance.loc path, if a instance.loc file is used.)

OPENDJ-1069 (OpenDJ shouldn't deliver a default instance.loc file)


How do I use the AWS snapshot feature to quickly create DS/OpenDJ (All versions) instances?

The purpose of this article is to provide information on using the Amazon Web Services™ (AWS™) snapshot feature to quickly provision new DS/OpenDJ instances from an existing running server and add them to the replication topology. Although this article is specific to the AWS snapshot feature, you can use the same concepts with other instantaneous copy systems such as OpenZFS snapshot.

Prerequisites

The following process assumes you already have two DS 5.5 instances installed and running (Master 1 and Master 2), where both instances were installed using the --instancePath setup option. This option allows the setup to separate the command line tools and runtime libraries from the instances data, instance libraries, configuration and log files. See How do I install DS/OpenDJ (All versions) so that the instance files are separate to the install files? for further information. 

Both Master 1 (opendj-source.forgerock.com) and Master 2 use the same setup install and instance paths. You can verify this as shown in the following example:

master1/$ cat opendj/instance.loc 
/opt/instances/opendjdata

master1/$ ./status
...
...
Installation Path:        /opt/instances/opendj
Instance Path:            /opt/instances/opendjdata
Version:                  ForgeRock Directory Services 5.5.0


master2/$ cat opendj/instance.loc
/opt/instances/opendjdata

master2/$ ./status
...
...
Installation Path:        /opt/instances/opendj
Instance Path:            /opt/instances/opendjdata
Version:                  ForgeRock Directory Services 5.5.0

Using the AWS snapshot feature to create DS/OpenDJ instances

Warning

You cannot use this process to upgrade a server to a new version; you must upgrade as detailed in the Installation Guide.

This example process refers to the following DS/OpenDJ instances:

  • Master 1 - this is the source instance and has the following hostname: opendj-source.forgerock.com
  • Master 2 - this is the other existing instance.
  • Master 3 - this is the new instance you are creating and has the following hostname: opendj-new.forgerock.com

You can create new DS/OpenDJ instances as follows:

  1. Select the Master 1 instance to use in the snapshot process; this is the snapshot source (opendj-source.forgerock.com).
  2. Stop the Master 1 instance; it is crucial that you stop the DS/OpenDJ instance before using the AWS snapshot feature as this guarantees the backend databases have been properly shut down and the lock region/data is properly recorded; failing to do so can corrupt backend databases.
  3. Wait 5-10 seconds after the instance has shut down to allow the system to finish syncing the data to disk;  failing to do so can corrupt the changelogDB file.
  4. Use the AWS snapshot feature to instantly copy the Master 1 instance, system. This snapshot is used as the template for the AWS newly provisioned (auto-scaled) system.
  5. Restart the Master 1 instance.
  6. Use the AWS auto-scaling system to provision the new system (Master 3) based on your snapshot; do not start this new Master 3 instance yet.   
  7. Remove the entire installation path from the new Master 3 instance to prepare for the new configuration:
    $ rm -rf /opt/instances/opendj
    
  8. Remove all but the /opt/instances/opendjdata/db and /opt/instances/opendjdata/changelogDb directories from the instance path for the new Master 3 instance:
    $ cd /opt/instances/opendjdata
    $ rm -rf bak classes config import-tmp ldif legal-notices lib locks logs
    
    Your new Master 3 instance now only has the following elements leftover from the source instance (Master 1): 
    opendj; instances/$ ls -la opendj
    ls: opendj: No such file or directory
    
    opendj; instances/$ ls -la opendjdata
    total 0
    drwxr-xr-x   4 opendj  opendj   136 Jun  9 11:51 .
    drwxr-xr-x  66 opendj  opendj  2244 Jun  9 11:50 ..
    drwxr-xr-x   5 opendj  opendj   170 Jun  9 10:53 changelogDb
    drwxr-xr-x   3 opendj  opendj   102 Jun  9 09:49 db
    You are now ready to install and set up the new Master 3 instance.
  9. Install DS/OpenDJ in the instances path of the new Master 3 instance, for example:
    $ cd /opt/instances
    $ unzip DS-5.5.0.zip
    $ cd opendj
    
  10. In OpenDJ 2.6.x and 3.x only: Create a new instance.loc file in this location:
    $ echo /opt/instances/opendjdata > /opt/instances/opendj/instance.loc
    
  11. Set up the new Master 3 instance with no backend configuration using the setup command applicable to your version:
    • DS 5 and later:
      $ ./setup --instancePath /opt/instances/opendjdata --ldapPort 1389 --adminConnectorPort 4444 --rootUserDN "cn=Directory Manager" --rootUserPassword password --hostname opendj-new.forgerock.com --enableStartTLS --ldapsPort 1636 --acceptLicense
      
    • Pre-DS 5:
      $ ./setup --cli --ldapPort 1389 --adminConnectorPort 4444 --rootUserDN "cn=Directory Manager" --rootUserPassword password --hostname opendj-new.forgerock.com --enableStartTLS --ldapsPort 1636 --generateSelfSignedCertificate --noPropertiesFile --acceptLicense --no-prompt
      
    The Master 3 instance is set up so that:
    • the /opt/instances/opendj directory contains all the command line tools and runtime libraries, for example:
      opendj; cd /opt/instances/opendj
      
      opendj; opendj/$ ls -l
      total 88
      drwxr-xr-x@  3 opendj  opendj   102 Jan  8 16:30 QuickSetup.app
      -rw-r--r--@  1 opendj  opendj  1801 Jan  8 16:22 README
      drwxr-xr-x@  3 opendj  opendj   102 Jan  8 16:30 Uninstall.app
      drwxr-xr-x@ 31 opendj  opendj  1054 Jan  8 16:30 bat
      drwxr-xr-x@ 35 opendj  opendj  1190 Jun  9 12:38 bin
      -rw-r--r--   1 opendj  opendj   278 Jun  9 12:04 install
      -rw-r--r--   1 opendj  opendj    26 Jun  9 11:50 instance.loc
      drwxr-xr-x@  5 opendj  opendj   170 Jan  8 16:30 legal-notices
      drwxr-xr-x@ 66 opendj  opendj  2244 Jan  8 16:30 lib
      -rw-r--r--@  1 opendj  opendj  6501 Jan  8 16:22 opendj_logo.png
      -rwxr-xr-x@  1 opendj  opendj  1838 Jan  8 16:22 setup
      -rw-r--r--@  1 opendj  opendj  2504 Jan  8 16:22 setup.bat
      drwxr-xr-x@  3 opendj  opendj   102 Jan  8 16:30 snmp
      drwxr-xr-x@ 11 opendj  opendj   374 Jan  8 16:30 template
      -rwxr-xr-x@  1 opendj  opendj  1875 Jan  8 16:22 uninstall
      -rw-r--r--@  1 opendj  opendj  2109 Jan  8 16:22 uninstall.bat
      -rwxr-xr-x@  1 opendj  opendj  1754 Jan  8 16:22 upgrade
      -rw-r--r--@  1 opendj  opendj  1840 Jan  8 16:22 upgrade.bat
    • the /opt/instances/opendjdata directory contains the database backend, the changelogDb, instance libraries, logs etc, for example:
      opendj; cd /opt/instances/opendjdata
      
      opendj; opendjdata/$ ls -l
      total 0
      drwxr-xr-x   2 opendj  opendj   68 Jun  9 12:12 bak
      drwxr-xr-x   5 opendj  opendj  170 Jun  9 10:53 changelogDb
      drwxr-xr-x   2 opendj  opendj   68 Jun  9 12:12 classes
      drwxr-xr-x  26 opendj  opendj  884 Jun  9 12:12 config
      drwxr-xr-x   3 opendj  opendj  102 Jun  9 09:49 db
      drwxr-xr-x   2 opendj  opendj   68 Jun  9 12:12 import-tmp
      drwxr-xr-x   2 opendj  opendj   68 Jun  9 12:12 ldif
      drwxr-xr-x   3 opendj  opendj  102 Jun  9 12:12 legal-notices
      drwxr-xr-x   3 opendj  opendj  102 Jun  9 12:12 lib
      drwxr-xr-x   9 opendj  opendj  306 Jun  9 12:12 locks
      drwxr-xr-x   7 opendj  opendj  238 Jun  9 12:12 logs
      
Note

The Instance libraries (/opt/instances/opendjdata/lib) can be used for installing support patches that should only exist for this instance.

  1. Initialize (pre-warm) the EBS volume using the following command:
    $ fio --filename=/dev/xvdg --rw=randread --bs=256k --iodepth=64 --ioengine=libaio --direct=1 --name=volume-initialize --output=/tmp/prefetch-summary.log
    This step can take a long time but is necessary to prevent performance issues in the new Master 3 instance due to the way in which AWS builds a snapshot. AWS builds a snapshot by copying the data to S3 behind the scenes; once the snapshot is applied to an EBS volume, the data in S3 is lazily loaded as each data block is loaded.
  2. Check the status of the new Master 3 instance; you will notice -No LDAP Databases Found- under Data Sources:
    $ ./status --bindDN "cn=Directory Manager" --bindPassword password
    
              --- Server Status ---
    Server Run Status:        Started
    Open Connections:         1
    
              --- Server Details ---
    Host Name:                opendj-new.forgerock.com
    Administrative Users:     cn=Directory Manager
    Installation Path:        /opt/instances/opendj
    Instance Path:            /opt/instances/opendjdata
    Version:                  ForgeRock Directory Services 5.5.0
    Java Version:             1.8.0_45
    Administration Connector: Port 4444 (LDAPS)
    
              --- Connection Handlers ---
    Address:Port : Protocol               : State
    -------------:------------------------:---------
    --           : LDIF                   : Disabled
    0.0.0.0:161  : SNMP                   : Disabled
    0.0.0.0:1389 : LDAP (allows StartTLS) : Enabled
    0.0.0.0:1636 : LDAPS                  : Enabled
    0.0.0.0:1689 : JMX                    : Disabled
    0.0.0.0:8080 : HTTP                   : Disabled
    
              --- Data Sources ---
    -No LDAP Databases Found-
  3. Add any implementation specific configuration such as password policies, timeouts etc.
  4. Add schema to the instance either by using ldapmodify or copying the schema across.
  5. Create the backend configuration using the existing /opt/instances/opendjdata path:
    $ ./dsconfig create-backend --set base-dn:dc=forgerock,dc=com --set enabled:true --set db-directory:/opt/instances/opendjdata/db --type [je or "local-db"] --backend-name userRoot --hostname opendj-new.forgerock.com --port 4444 --trustAll --bindDN "cn=Directory Manager" --bindPassword password --no-prompt
    
    where --type is je for DS / OpenDJ 3.x or "local-db" for OpenDJ 2.6.x.
Note

The backend database is immediately available for use, but is not yet replicated. Do not change database entries in the backend until replication has been fully enabled.

You may notice the following errors in the OpenDJ error log when the backend is created; these can be safely ignored:

[09/Jun/2016:12:24:16 -0600] category=PLUGGABLE severity=NOTICE msgID=org.opends.messages.backend.513 msg=The database backend userRoot containing 1000 entries has started
[09/Jun/2016:12:24:16 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.724 msg=The search filter "(|(objectClass=subentry)(objectClass=ldapSubentry))" used by subentry manager is not indexed in backend userRoot. Backend initialization for subentry manager processing might take a very long time to complete
[09/Jun/2016:12:24:17 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.660 msg=The search filter "(&(|(objectClass=groupOfNames)(objectClass=groupOfUniqueNames)(objectClass=groupOfEntries))(!(objectClass=ds-virtual-static-group)))" used by group implementation cn=Static,cn=Group Implementations,cn=config is not indexed in backend userRoot. Backend initialization for this group implementation might take a very long time to complete
[09/Jun/2016:12:24:17 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.660 msg=The search filter "(objectClass=ds-virtual-static-group)" used by group implementation cn=Virtual Static,cn=Group Implementations,cn=config is not indexed in backend userRoot. Backend initialization for this group implementation might take a very long time to complete
[09/Jun/2016:12:24:17 -0600] category=CORE severity=WARNING msgID=org.opends.messages.core.660 msg=The search filter "(objectClass=groupOfURLs)" used by group implementation cn=Dynamic,cn=Group Implementations,cn=config is not indexed in backend userRoot. Backend initialization for this group implementation might take a very long time to complete
[09/Jun/2016:12:24:17 -0600] category=ACCESS_CONTROL severity=WARNING msgID=org.opends.messages.access_control.96 msg=Backend userRoot does not have a presence index defined for attribute type aci. Access control initialization may take a very long time to complete in this backend
  1. Create any implementation specific indexes and index configuration. 
  2. Check the status of the new Master 3 instance again; you will now see details under Data Sources:
              --- Server Status ---
    Server Run Status:        Started
    Open Connections:         1
    
              --- Server Details ---
    Host Name:                opendj-new.forgerock.com
    Administrative Users:     cn=Directory Manager
    Installation Path:        /opt/instances/opendj
    Instance Path:            /opt/instances/opendjdata
    Version:                  ForgeRock Directory Services 5.5.0
    Java Version:             1.8.0_45
    Administration Connector: Port 4444 (LDAPS)
    
              --- Connection Handlers ---
    Address:Port : Protocol               : State
    -------------:------------------------:---------
    --           : LDIF                   : Disabled
    0.0.0.0:161  : SNMP                   : Disabled
    0.0.0.0:1389 : LDAP (allows StartTLS) : Enabled
    0.0.0.0:1636 : LDAPS                  : Enabled
    0.0.0.0:1689 : JMX                    : Disabled
    0.0.0.0:8080 : HTTP                   : Disabled
    
              --- Data Sources ---
    Base DN:     dc=example,dc=com
    Backend ID:  userRoot
    Entries:     1000
    Replication: 
    
  3. Restart the new Master 3 instance; this will clear up any errors previously seen in the DS/OpenDJ error log.
  4. Add this new Master 3 instance to the replication topology:
    • Log in to the Master 1 system that was used as the snapshot source.
    • Enable replication from the Master 1 instance to the new Master 3 instance using the dsreplication command applicable to your version:
      • DS 5 and later:
        $ ./dsreplication configure --adminUid admin --adminPassword password --baseDn dc=example,dc=com --host1 opendj-source.forgerock.com --port1 4444 --bindDn1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 8989 --host2 opendj-new.forgerock.com --port2 6444 --bindDn2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 10989 --trustAll --no-prompt
        
      • Pre-DS 5:
        $ ./dsreplication enable --adminUID admin --adminPassword password --baseDN dc=example,dc=com --host1 opendj-source.forgerock.com --port1 4444 --bindDN1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 8989 --host2 opendj-new.forgerock.com --port2 6444 --bindDN2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 10989 --trustAll --no-prompt
        
    The newly provisioned DS/OpenDJ instance (Master 3) is now ready to be placed into the load balancing pool.

Decommissioning a DS/OpenDJ instance

The above process can be used to easily auto-provision new DS/OpenDJ instances and add them to the replication topology. When removing an unused instance, you must first disable replication on the DS/OpenDJ instance to be decommissioned before the AWS system can be deprovisioned. Failing to do so will leave configuration elements in the remaining servers which will cause errors for commands such as dsreplication status and may affect replication/replication performance.

You can disable replication as follows:

  • DS 5 and later:
    $ ./dsreplication unconfigure --unconfigureAll --port 4444 --hostname opendj-newX.forgerock.com --bindDn "cn=Directory Manager" --adminPassword password --trustAll --no-prompt
  • Pre-DS 5:
    $ ./dsreplication disable --disableAll --port 4444 --hostname opendj-newX.forgerock.com --bindDN "cn=Directory Manager" --adminPassword password --trustAll --no-prompt

See Also

How do I install DS/OpenDJ (All versions) so that the instance files are separate to the install files?

Creating an Amazon EBS Snapshot

Auto Scaling Groups

Scaling the Size of Your Auto Scaling Group

Initializing Amazon EBS Volumes

Related Training

N/A

Related Issue Tracker IDs

N/A


Administration


How do I change the hostname for DS/OpenDJ (All versions)?

The purpose of this article is to provide information on changing the hostname for a DS/OpenDJ server. This article covers both replicated and non-replicated servers.

Changing the hostname

This process uses the following example server hostnames:

  • Original hostname: dsA.example.com
  • New hostname: dsB.example.com
  • Hostname of another replicated server: dsZ.example.com

To change the server hostname:

  1. Direct client applications to other servers.
  2. Prevent the server from accepting updates from client applications using the following command:
    $ ./dsconfig set-global-configuration-prop --port 4444 --hostname dsA.example.com --bindDN "cn=Directory Manager" --bindPassword password --set writability-mode:internal-only --trustAll --no-prompt
    
  3. If the server is replicated, disable replication using the dsreplication command applicable to your version:
    • DS 5 and later:
      $ ./dsreplication unconfigure --unconfigureAll --port 4444 --hostname dsA.example.com --bindDN "cn=Directory Manager" --adminPassword password --trustAll --no-prompt
    • Pre-DS 5:
      $ ./dsreplication disable --disableAll --port 4444 --hostname dsA.example.com --bindDN "cn=Directory Manager" --adminPassword password --trustAll --no-prompt
      
  4. Change the hostname details in the /etc/hosts file and/or on the DNS.
  5. Change the server-fqdn in the DIGEST-MD5 entry using the following command:
    $ ./dsconfig set-sasl-mechanism-handler-prop --handler-name DIGEST-MD5 --port 4444 --hostname dsA.example.com --bindDN "cn=Directory Manager" --bindPassword password --set server-fqdn:dsB.example.com --trustAll
    
  6. Restart DS/OpenDJ:
    $ ./stop-ds
    $ ./start-ds
  7. Regenerate all self-signed certificates. See Administration Guide › Preparing For Secure Communications and Administration Guide › Changing Server Certificates for further information.
  8. Restart DS/OpenDJ:
    $ ./stop-ds
    $ ./start-ds
  9. If the server was replicated, enable replication on the new server using the dsreplication command applicable to your version:
    • DS 5 and later:
      $ ./dsreplication configure --adminUid admin --adminPassword password --baseDn dc=example,dc=com --host1 dsZ.example.com --port1 4444 --bindDn1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 8989 --host2 dsB.example.com --port2 4444 --bindDn2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 8989 --trustAll --no-prompt
    • Pre-DS 5:
      $ ./dsreplication enable --adminUID admin --adminPassword password --baseDN dc=example,dc=com --host1 dsZ.example.com --port1 4444 --bindDN1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 8989 --host2 dsB.example.com --port2 4444 --bindDN2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 8989 --trustAll --no-prompt
  10. If the server was replicated, initialize the new server to ensure it has all the changes that have occurred since you disabled replication:
    $ ./dsreplication initialize --adminUID admin --adminPassword password --baseDN dc=example,dc=com --hostSource dsZ.example.com --portSource 4444 --hostDestination dsB.example.com --portDestination 4444 --trustAll --no-prompt
  11. Re-enable the server to accept updates from client applications using the following command:
    $ ./dsconfig set-global-configuration-prop --port 4444 --hostname dsA.example.com --bindDN "cn=Directory Manager" --bindPassword password --set writability-mode:enabled --trustAll --no-prompt

How does DS/OpenDJ resolve the hostname?

DS/OpenDJ picks up what the JVM or operating system has recorded for the FQDN. If you have not changed the hostname correctly, you may see the following error (referencing the original hostname):

[27/Feb/2017:15:33:59 -0800] category=JVM severity=NOTICE msgID=18 msg=JVM Host: Unknown (java.net.UnknownHostException: dsA.example.com: dsA.example.com: Name or service not known), running Linux 2.6.32-573.3.1.el6.x86_64 amd64, 4017881088 bytes physical memory size, number of processors available 4

You can confirm that the JVM or operating system is broadcasting the wrong FQDN using the hostname command:

$ hostname
dsA.example.com

On a Linux® system, you can correct the FQDN being broadcast by updating the /etc/sysconfig/network file with the correct FQDN and restarting your system. Upon running the hostname command again, you should see the new hostname, which means DS/OpenDJ will use the correct one and you should no longer see this error.

See Also

Installing and Administering DS/OpenDJ

Administration Guide › Moving Servers

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I check if a backend is online in DS/OpenDJ (All versions)?

The purpose of this article is to provide information on checking if a backend is online in DS/OpenDJ. This can be a useful check as part of your DS/OpenDJ monitoring.

Checking if a backend is online (DS 6 and later)

You can perform a ldapsearch against the backend to check it is online. This example performs a check against the userRoot backend:

$ ./ldapsearch --port 389 --bindDN "cn=Directory Manager" --bindPassword password --baseDN "ds-cfg-backend-id=userRoot,cn=backends,cn=monitor" --searchScope base "(objectClass=*)"

Online backend

If the backend is online, you would get a response such as the following:

dn: ds-cfg-backend-id=userRoot,cn=backends,cn=monitor
objectClass: top
objectClass: ds-monitor
objectClass: ds-monitor-backend
objectClass: ds-monitor-backend-pluggable
objectClass: ds-monitor-backend-db
ds-mon-backend-is-private: false
ds-mon-backend-entry-count: 1511
ds-mon-backend-writability-mode: enabled
ds-mon-backend-degraded-index-count: 0
ds-mon-backend-ttl-is-running: false
ds-mon-backend-ttl-last-run-time: 20190131162956.795Z
ds-mon-backend-ttl-thread-count: 0
ds-mon-backend-ttl-queue-size: 0
ds-mon-backend-ttl-entries-deleted: {"count":0,"total":0.000,"mean_rate":0.000,"m1_rate":0.000,"m5_rate":0.000,"m15_rate":0.000}
ds-mon-backend-filter-use-start-time: 19700101000000Z
ds-mon-backend-filter-use-indexed: 0
ds-mon-backend-filter-use-unindexed: 0
ds-mon-db-version: 7.5.11
ds-mon-db-cache-evict-internal-nodes-count: 0
ds-mon-db-cache-evict-leaf-nodes-count: 0
ds-mon-db-cache-total-tries-internal-nodes: 1709
ds-mon-db-cache-total-tries-leaf-nodes: 1382
ds-mon-db-cache-misses-internal-nodes: 31
ds-mon-db-cache-misses-leaf-nodes: 438
ds-mon-db-cache-size-active: 5042101
ds-mon-db-log-size-active: 7550013
ds-mon-db-log-cleaner-file-deletion-count: 0
ds-mon-db-log-utilization-min: 34
ds-mon-db-log-utilization-max: 34
ds-mon-db-log-size-total: 7550013
ds-mon-db-log-files-open: 1
ds-mon-db-log-files-opened: 3
ds-mon-db-checkpoint-count: 0
ds-cfg-backend-id: userRoot

Offline backend

If the backend is offline, you would get a response such as the following:

# The LDAP search request failed: 32 (No Such Entry)
# Additional Information:  Entry ds-cfg-backend-id=userRoot,cn=backends,cn=monitor does not exist in the "monitor" backend
# Matched DN:  cn=backends,cn=monitor

Checking if a backend is online (Pre-DS 6)

You can perform a ldapsearch against the backend to check it is online. This example performs a check against the userRoot backend:

$ ./ldapsearch --port 389 --bindDN "cn=Directory Manager" --bindPassword password --baseDN "cn=userRoot Backend,cn=monitor" --searchScope sub "(objectClass=*)"

Online backend

If the backend is online, you would get a response such as the following:

dn: cn=userRoot Backend,cn=monitor
objectClass: top
objectClass: ds-monitor-entry
objectClass: ds-backend-monitor-entry
ds-backend-is-private: FALSE
ds-backend-writability-mode: enabled
cn: userRoot Backend
ds-backend-entry-count: 200002
ds-base-dn-entry-count: 200002 dc=forgerock,dc=com
ds-backend-id: userRoot
ds-backend-base-dn: dc=forgerock,dc=com

Offline backend

If the backend is offline, you would get a response such as the following:

SEARCH operation failed
Result Code:  32 (No Such Entry)
Additional Information:  Entry cn=userRoot Backend,cn=monitor does not exist in the memory-based backend
Matched DN:  cn=monitor

You would also see a corresponding error in the DS/OpenDJ log, for example:

[24/Mar/2015:14:41:59 -0600] category=CONFIG severity=SEVERE_ERROR msgID=3407988 msg=An error occurred while trying to initialize a backend loaded from class org.opends.server.backends.jeb.BackendImpl with the information in configuration entry ds-cfg-backend-id=userRoot,cn=Backends,cn=config:  The database environment could not be opened: (JE 5.0.73) Environment must be closed, caused by: com.sleepycat.je.EnvironmentFailureException: Environment invalid because of previous exception: (JE 5.0.73) /Users/jdoe/Software/db/userRoot com.sleepycat.je.log.ChecksumException: Incomplete log entry header, size=0 lsn=0x10/0x67b292 LOG_CHECKSUM: Checksum invalid on read, log is likely invalid. Environment is invalid and must be closed. fetchTarget of 0x10/0x67b292 parent IN=3 IN class=com.sleepycat.je.tree.BIN lastFullVersion=0x1a/0x9136a5 lastLoggedVersion=0x1f/0xc74a1 parent.getDirty()=false state=0 (BackendImpl.java:1754 BackendImpl.java:319 BackendConfigManager.java:1298 BackendConfigManager.java:279 DirectoryServer.java:2210 DirectoryServer.java:1397 DirectoryServer.java:9651).  This backend will be disabled

See Also

How do I perform a heartbeat check against DS/OpenDJ (All versions)?

FAQ: Monitoring DS/OpenDJ

How do I use cn=monitor entry in DS/OpenDJ (All versions) for monitoring?

How do I verify that a DS/OpenDJ (All versions) server is responding to LDAP requests without providing a password?

Administration Guide › Monitoring, Logging, and Alerts

Related Training

ForgeRock Directory Services Core Concepts

Related Issue Tracker IDs

N/A


How do I relocate the backend database files in DS/OpenDJ (All versions) on to a separate file system?

The purpose of this article is to provide information on relocating the backend database files in DS/OpenDJ to a different location, such as a separate file system.

Overview

When you change the current location of the backend database files in DS/OpenDJ, the database files are created in the new location going forward. The database files in the existing location are not automatically moved to the new location and will remain in the existing location until you remove or archive them off. The following process is used to manually relocate the database files.

Note

You must restart DS/OpenDJ after running the dsconfig command and then import the data as described in the steps below or the database files will not be created in the new location. 

When you run the dsconfig command, you will see the following message in the error log, which will be resolved when you restart:

[10/Nov/2015:12:21:08 -0700] category=CONFIG severity=SEVERE_WARNING msgID=3277447 msg=org.opends.server.admin.server.ConfigChangeListenerAdaptor.applyConfigurationChange indicated that administrative action is required for entry ds-cfg-backend-id=userRoot,cn=Backends,cn=config:  messages="The change to the DB directory will not take effect until the backend is restarted. The DB files from the previous directory db must be moved to the new directory /databasefiles/instancedb after shutting down the backend to retain the existing data"

Relocating the backend database files

This process uses the following example details:

  • old location - /path/to/ds/db 
  • new location - /opt/ds/db
  • backend being relocated - userRoot

You can relocate the backend database files to a new location as follows: 

  1. Export the ldif file for the old location, for example: 
    $ ./export-ldif --hostname localhost --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backendID userRoot --ldifFile /path/to/generated.ldif --start 0 --trustAll
  2. Relocate the required backend from the old location to the new location, for example: 
    $ ./dsconfig set-backend-prop --backend-name userRoot --set db-directory:/opt/ds/db --hostname localhost --port 4444 --trustAll --bindDN "cn=Directory Manager" --bindPassword password --no-prompt
    
  3. Restart DS/OpenDJ.
  4. Import the ldif file you created in step 1 to restore the old database files to the new location, for example:
    $ ./import-ldif --hostname localhost --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backendID userRoot --ldifFile /path/to/generated.ldif
    All the backend database files for userRoot are now located in /opt/ds/db and this is the location that will be used going forward.
Note

You can optionally use the DS/OpenDJ backup and restore commands to relocate the backend database files instead, where the backup command replaces export-ldif in step 1 and the restore command replaces import-ldif in step 4.

See Also

How do I install DS/OpenDJ (All versions) so that the instance files are separate to the install files?

FAQ: Backup and restore in DS/OpenDJ

How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

Installing and Administering DS/OpenDJ

Reference

Administration Guide › Backing Up and Restoring Data

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I convert a PDB backend to a JE backend in DS 5.x or OpenDJ 3.x?

The purpose of this article is to provide information on converting a PDB backend to a JE backend in DS/OpenDJ. You must convert your PDB backend before upgrading to DS 6.

Overview

The PDB backend was introduced in OpenDJ 3, deprecated in DS 5 and removed in DS 6. Before upgrading to DS 6, you must convert your PDB backend to a JE backend. See Release Notes › What's New for further information.

This does not affect the embedded DS/OpenDJ in AM/OpenAM as that uses the JE backend by default.

Converting a PDB backend to a JE backend

Note

We strongly recommend testing this conversion in your own staging environment first and ensuring you have up to date backups and recovery plans in case you encounter any issues.

The steps needed to convert your backend vary according to which version you are currently on:

See Also

Upgrading DS/OpenDJ

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I make DS/OpenDJ (All versions) listen on port 389 without being root?

The purpose of this article is to provide guidance on running DS/OpenDJ as an unprivileged user while listening on a privileged port: 389 (LDAP) and 636 (LDAPS).

Overview

It is considered best practice on Unix® based systems to avoid running services as root, as any security vulnerabilities in the service could then lead to the root account being compromised. DS/OpenDJ's startup script lets the service be run as an administrator defined user, for example "opendj".

Unix operating systems prevent non-root users from listening on TCP/IP sockets below 1024. It is desirable to run DS/OpenDJ listening on the standard ports of 389 (LDAP) and 636 (LDAPS), but running as a non-root user would seem to prevent this.

This article offers some solutions for different supported operating systems: Linux® and Solaris®.

Note

Microsoft® Windows® does not restrict which processes can listen on TCP/IP sockets below 1024, so no special configuration is needed for DS/OpenDJ on Microsoft Windows servers.

Linux

There are two ways to solve this problem on Linux:

  • Using capabilities(7)
  • Using port forwarding

Using capabilities(7)

This technique lets you grant a given executable program additional privileges. In the case of DS/OpenDJ, the executable program is Java®.

Caution

Often Linux systems have multiple JVMs installed. Make sure you are configuring the JVM that DS/OpenDJ is actually using.

In the following steps, a 64-bit version of Java 8u172 is being used; update the paths as appropriate if you are using a different version:

  1. Add the net_bind_service capability to the Java executable:
    # setcap cap_net_bind_service+epi /usr/java/jdk1.8.0_172/jre/bin/java
    This change causes the operating system to treat the Java executable as "trusted". The runtime linker on Linux will prevent trusted executables from dynamically loading libraries from unknown locations, which will cause Java to fail to start.
  2. Configure the runtime linker to allow dynamic libraries to be loaded from a particular JVM directory and then to update its cache:
    # echo /usr/java/jdk1.8.0_172/jre/lib/amd64/jli > /etc/ld.so.conf.d/java.conf
    # ldconfig
  3. Reboot to force the runtime linker to reload its configuration.
  4. Verify that the runtime linker settings have been updated by running ldconfig again:
    # ldconfig -p | grep jli
    	libjli.so (libc6,x86-64) => /usr/java/jdk1.8.0_172/jre/lib/amd64/jli/libjli.so
  5. Configure DS/OpenDJ to listen on a privileged port using the dsconfig command applicable to your version:
    • DS 6 and later:
      $ ./dsconfig set-connection-handler-prop --handler-name LDAP --set listen-port:389 --hostname ds1.example.com --port 4444 --trustAll --bindDN "cn=Directory Manager" --bindPassword password --no-prompt
    • Pre-DS 6:
      $ ./dsconfig set-connection-handler-prop --handler-name "LDAP Connection Handler" --set listen-port:389 --hostname ds1.example.com --port 4444 --trustAll --bindDN "cn=Directory Manager" --bindPassword password --no-prompt
      
Warning

Any Java program will now be able to listen on a privileged port. Consider using port forwarding instead if this is a concern.

Using Port Forwarding

With this configuration, you redirect the standard port (for example, 389) to the non-privileged port that DS/OpenDJ is actually listening on (for example, 1389). The details for achieving this will depend on the Linux distribution; the examples below are from Red Hat® Enterprise Linux 6:

  1. Enable IP forwarding by editing /etc/sysctl.conf and setting the net.ipv4.ip_forward variable to 1:
    net.ipv4.ip_forward = 1
  2. Reload the changed file:
    # sysctl -p /etc/sysctl.conf
  3. Update the iptables configuration to allow connections to both ports (389 and 1389), and lastly to add REDIRECT rules to the "nat" table. The rules to allow the connections are:
    -A INPUT -m state --state NEW -m tcp -p tcp --dport 389 -j ACCEPT
    -A INPUT -m state --state NEW -m tcp -p tcp --dport 1389 -j ACCEPT
    
    The rules to redirect the port are:
    -A PREROUTING -t nat -p tcp --dport 389 -j REDIRECT --to-ports 1389
    -A OUTPUT -t nat -p tcp -d 127.0.0.1 --dport 389 -j REDIRECT --to-ports 1389
    The resulting /etc/sysconfig/iptables file might look like this:
    *filter
    :INPUT ACCEPT [0:0]
    :FORWARD ACCEPT [0:0]
    :OUTPUT ACCEPT [0:0]
    -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
    -A INPUT -p icmp -j ACCEPT
    -A INPUT -i lo -j ACCEPT
    -A INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
    -A INPUT -m state --state NEW -m tcp -p tcp --dport 389 -j ACCEPT
    -A INPUT -m state --state NEW -m tcp -p tcp --dport 1389 -j ACCEPT
    -A INPUT -j REJECT --reject-with icmp-host-prohibited
    -A FORWARD -j REJECT --reject-with icmp-host-prohibited
    COMMIT
    *nat
    -A PREROUTING -p tcp --dport 389 -j REDIRECT --to-ports 1389
    -A OUTPUT -p tcp -d 127.0.0.1 --dport 389 -j REDIRECT --to-ports 1389
    COMMIT
  4. Restart iptables using:
    # service iptables restart

DS/OpenDJ should now be accessible on port 389, even though it is configured to listen on port 1389.

Solaris

On Solaris 10 and above, you can use the privileges(5) mechanism to change the privileges for a specific user.

To allow the "opendj" account to listen on privileged network ports, do this:

# usermod -K defaultpriv=basic,net_privaddr opendj

The "opendj" user will then be able to listen on privileged network ports.

See Also

How do I prevent anonymous access in DS/OpenDJ (All versions)?

How do I prevent the use of weak SSL cipher suites on DS/OpenDJ (All versions) administration port?

How do I prevent the use of weak SSL cipher suites on DS/OpenDJ (All versions) replication port?

Installing and Administering DS/OpenDJ

Security Guide › Securing Directory Administration

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I stop DS/OpenDJ (All versions) to perform a clean shutdown?

The purpose of this article is to provide information on stopping DS/OpenDJ in order to perform a clean shutdown. It also highlights the importance of shutting down DS/OpenDJ properly.

Shutting down DS/OpenDJ

DS/OpenDJ needs to properly shut down to release any internal locks and write database specific information to the applicable files. The files in question differ depending on the type of backend in use (the PDB backend was deprecated in DS 5 and removed in DS 6):

  • JE backend files:
    • je.config.csv
    • je.info.0
    • je.stat.csv
  • PDB backend files:
    • dj
    • dj.lck
    • dj.log
    • dj_journal.000000000001
    • dj_journal.000000000002

If DS/OpenDJ is not shut down properly, it runs a recovery process upon startup; this process takes time and can also cause strange errors to be seen in your logs during startup. In extreme circumstances, failing to shut down DS/OpenDJ properly can corrupt the backend databases.

Note

If you use AM/OpenAM with an embedded DS/OpenDJ, you must ensure you shut down AM/OpenAM properly for the same reasons prior to restarting the server. 

The correct process for shutting down DS/OpenDJ is documented in Administration Guide › Stopping a Server. You should ensure your init script achieves a proper shutdown. Similarly, the Microsoft® Windows® service can also be used to shut down DS/OpenDJ properly.

See Also

How do I manage DS/OpenDJ (All versions) as a service on Microsoft Windows?

Administration Guide › Managing Server Processes

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-2818 (The "Managing server processes" guide needs a proper shutdown process)


How do I configure DS/OpenDJ (All versions) to be stopped and started as a service using systemd and systemctl?

The purpose of this article is to provide assistance with configuring DS/OpenDJ to be stopped and started as a service using systemd and systemctl if you are using a Linux® system. This method uses init scripts created with create-rc script.

Overview

systemd is backwards compatible with Linux initialization scripts as detailed in: How To Configure a Linux Service to Start Automatically After a Crash or Reboot – Part 1: Practical Examples.

The key thing to be aware of when using systemd is that you must create your own init scripts and enable the services to start automatically. This article describes how you can do this with DS/OpenDJ.

Configuring DS/OpenDJ

  1. Generate a DS/OpenDJ init script by running the create-rc-script tool as detailed in Administration Guide › Managing Server Processes. For example:
    $ ./create-rc-script --userName root --outputFile opendj.init
    
    You can view the attached example init script that was generated for OpenDJ 3.5.
  2. Create a service file and enable that service using the guidelines given in Auto-starting Services with systemd
    1. Create a service file; this example demonstrates creating the service file by copying the VirtualBox® service file:
      $ cd /usr/lib/systemd/system
      $ cp vboxadd.service opendj.service
      You should ensure Restart=always in this file to enable the service to respawn after a crash. You can view the attached example service file that was created in this way for OpenDJ 3.5.
    2. Enable the service file:
      $ sudo systemctl enable opendj.service
      
    3. Reload the daemon:
      $ sudo systemctl daemon-reload	
  3. Stop and restart DS/OpenDJ:
    $ ./stop-ds
    $ ./start-ds
  4. Verify the service is online as follows:
    $ systemctl is-enabled opendj.service
    enabled
    
    $ systemctl status opendj.service
    opendj.service - ForgeRock OpenDJ 3.5.0
       Loaded: loaded (/opt/opendj/bin; enabled)
       Active: active (exited) since Fri 2017-09-29 14:17:41 GMT; 7min ago
      Process: 650 ExecStart=/opt/opendj/bin/opendj.init start (code=exited, status=0/SUCCESS)
    
    Sep 29 14:17:20 localhost.localdomain su[665]: (to root) root on none
    Sep 29 14:17:41 opendj.forgerock.com systemd[1]: Started ForgeRock OpenDJ 3.5.0.
    

See Also

Administration Guide › Managing Server Processes

How To Configure a Linux Service to Start Automatically After a Crash or Reboot – Part 1: Practical Examples

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-3810 (Add Linux systemd support for OPENDJ startup)


How do I manage DS/OpenDJ (All versions) as a service on Microsoft Windows?

The purpose of this article is to provide guidance on starting and stopping the DS/OpenDJ service on Microsoft® Windows®.

Managing DS/OpenDJ as a service

DS/OpenDJ can be configured during setup to run as a service on Microsoft Windows; you will find the option to Run the server as a Windows Service at the end of the installation procedure just after the initial configuration step. At this point, the installer presents a Review configuration screen where you can verify your settings and enable this option before clicking Finish.

Microsoft Windows services (including DS/OpenDJ) can be started and stopped using the Windows Services MMC snap-in.

The DS/OpenDJ control panel can also start and stop DS/OpenDJ if it is running as a service; the control panel is removed in DS 6.

You can download the latest version from BackStage.

See Also

Administration Guide › Managing Server Processes

Installation Guide

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-617 (Fix Windows services)

OPENDJ-1322 (Control-Panel.bat can not start and stop the OpenDJ server when running as a windows service)


How do I change DS/OpenDJ (All versions) to use a different JDK version?

The purpose of this article is to provide information on changing DS/OpenDJ to use a different version of the JDK, if for example, you have upgraded your Java® environment. This article assumes you have already successfully upgraded your Java environment.

Changing JDK version

You can change which JDK DS/OpenDJ is using as follows:

  1. Update the default.java-home property in the java.properties file (located in the /path/to/ds/config directory). You can either specify an exact version or latest:
    • Exact version - specify the path of the correct JDK version. For example, if you are using Java 8 Update 45, the updated property would look similar to this:
      default.java-home=/usr/lib/jvm/jdk1.8.0_45/jre
      If you take this approach, you will need to update this property each time you upgrade your Java environment.
    • Latest - specify latest so that the latest version of the JDK is always used:
      default.java-home=/usr/java/latest
      If you take this approach, you will only need to restart DS/OpenDJ to take account of an upgraded Java environment in the future.
  2. Pre-DS 5 only: Run the bin/dsjavaproperties command to apply the changes you have made to the java.properties file:
    $ ./dsjavaproperties
    This command has been removed in DS 5.
  3. Restart the DS/OpenDJ server.

See How do I ensure DS/OpenDJ (All versions) uses the Java settings from java.properties file when DS/OpenDJ is started? for further information on the java.properties file.

See Also

How do I ensure DS/OpenDJ (All versions) uses the Java settings from java.properties file when DS/OpenDJ is started?

How do I collect JVM data for troubleshooting DS/OpenDJ (All versions)?

How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache?

How do I enable Garbage Collector (GC) Logging for DS/OpenDJ (All versions)?

OpenDJ Reference › Tools Reference › dsjavaproperties

Related Training

ForgeRock Directory Services Core Concepts 

Related Issue Tracker IDs

OPENDJ-3205 (Control-panel: missing java-settings and manage tasks screen)


How do I ensure DS/OpenDJ (All versions) uses the Java settings from java.properties file when DS/OpenDJ is started?

The purpose of this article is to provide information on ensuring DS/OpenDJ uses the Java® settings in the java.properties file in preference to any environment specific settings. Out of the box, the DS/OpenDJ server and DS/OpenDJ tools run with the default Java Virtual Machine and Java arguments. This article demonstrates how to override the defaults and ensure your new JVM settings are picked up by the java.properties file when DS/OpenDJ or its tools are started.

Overview

The java.properties file contains all the Java properties that are used when DS/OpenDJ is launched. The file contains an overview of the properties that can be changed as well as different example scenarios to help you understand what the properties are used for.

The java.properties file is located in the /path/to/ds/config directory.

You must run the bin/dsjavaproperties command to apply any changes you make to the java.properties file in OpenDJ; this command has been removed in DS 5.

See How do I change DS/OpenDJ (All versions) to use a different JDK version? for information on what needs changing in this file when you upgrade your Java environment.

Ensuring DS/OpenDJ uses the java.properties file

  1. Update the overwrite-env-java-args property in the java.properties file to true:
    overwrite-env-java-args=true
  2. Apply this change as follows depending on your version:
    • DS 5 and later: Restart the DS server or tool; DS will now always use the properties in the java.properties file.
    • Pre-DS 5: Run the bin/dsjavaproperties command to apply the changes you have made to the java.properties file:
      $ ./dsjavaproperties
      OpenDJ will now always use the properties in the java.properties file.

Checking your work

DS/OpenDJ and many of the DS/OpenDJ tools that would usually be used with updated properties, print out the JVM details and arguments to both the the terminal window and log files on startup; look for the following in /path/to/ds/logs/server.out or /path/to/ds/logs/errors to verify that the correct properties are being used:

[22/Jun/2015:16:39:59 +0100] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381713 msg=JVM Information: 1.7.0_55-b13 by Oracle Corporation, 64-bit architecture, 954728448 bytes heap size
[22/Jun/2015:16:39:59 +0100] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381714 msg=JVM Host: ds1.example.com, running Linux 2.6.32-504.16.2.el6.x86_64 amd64, 1967665152 bytes physical memory size, number of processors available 8
[22/Jun/2015:16:39:59 +0100] category=RUNTIME_INFORMATION severity=NOTICE msgID=20381715 msg=JVM Arguments: "-Xms1G", "-Xmx1G", "-XX:+UseConcMarkSweepGC", "-XX:MaxTenuringThreshold=1", "-XX:+UseCompressedOops", "-Dorg.opends.server.scriptName=start-ds"

See Finding current JVM settings section in How do I collect JVM data for troubleshooting DS/OpenDJ (All versions)? for further information on checking your JVM settings.

See Also

How do I change DS/OpenDJ (All versions) to use a different JDK version?

How do I collect JVM data for troubleshooting DS/OpenDJ (All versions)?

How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache?

How do I enable Garbage Collector (GC) Logging for DS/OpenDJ (All versions)?

OpenDJ Reference › Tools Reference › dsjavaproperties

Related Training

 ForgeRock Directory Services Core Concepts 

Related Issue Tracker IDs

OPENDJ-3205 (Control-panel: missing java-settings and manage tasks screen)


How do I set advanced properties with dsconfig in DS/OpenDJ (All versions)?

The purpose of this article is to provide assistance with setting advanced properties with dsconfig in DS/OpenDJ. Advanced properties are not shown by default.

Setting advanced properties with dsconfig

There are a number of DS/OpenDJ configuration settings that you can set with dsconfig that are considered advanced properties. These advanced properties are not shown by default.

For example, if you want to permit multiple password values in a password policy using the dsconfig set-password-policy-prop command and know from looking at Configuration Reference › Password Policy that the property you want to set is advanced, you just need to include --advanced before the sub-command. For example:

$ ./dsconfig set-password-policy-prop --bindDN "cn=Directory Manager" --bindPassword password --policy-name "Multi Value Policy" --advanced --set allow-multiple-password-values:true --trustAll --no-prompt

Similarly, you can use --advanced with a get command to show all properties, including the advanced ones.

For example, if you want to check the current property settings for the Configuration Reference › LDAP Connection Handler using the dsconfig get-connection-handler-prop command, you would use a command such as the following depending on your version:

  • DS 6 and later:
    $ ./dsconfig get-connection-handler-prop --port 4444 --bindPassword password --handler-name LDAP --advanced --trustAll --no-prompt
  • Pre-DS 6:
    $ ./dsconfig get-connection-handler-prop --port 4444 --bindPassword password --handler-name "LDAP Connection Handler" --advanced --trustAll --no-prompt

See Also

Reference › dsconfig

Configuration Reference

Related Training

ForgeRock Directory Services Core Concepts

Related Issue Tracker IDs

N/A


Best practice for managing groups in DS/OpenDJ (All versions)

The purpose of this article is to provide best practice advice on managing groups in DS/OpenDJ.

Overview

DS/OpenDJ utilizes three different types of groups to help manage data:

  • Static Groups: these groups have a list of members.
  • Dynamic Groups: these groups look up membership based on an LDAP filter.
  • Virtual Static Groups: these groups use dynamic group style definition but allow applications to list group members as if they were static.

See Developer's Guide › Working With Groups of Entries for more detail on how to create/manage each type of group.

Managing groups

When deciding how to architect groups for your organization, it is important to note there are some performance implications with:

  • Large numbers of groups.
  • Large numbers of entries in a static group.

Large numbers of groups

When you do a search that returns the isMemberOf attribute, it has to scan every group. If you have 10,000+ groups, this can be quite an expensive operation. 

Note

Using isMemberOf as a search filter will not have this performance impact as it is searching the group you specify for members.

Large numbers of entries in a static group

When you do a replace modification on a static group, it has to read/edit all members. if you have 30,000 + members in a static group, this can be quite an expensive operation.

Mitigation

You can use the following methods to mitigate these performance impacts: 

  • Use indexed searches for (member=...). Indexes are very efficient, although this will not find members of nested groups. 
  • Implement entry caches for large static groups; this will put the entries into memory so read operations will be much quicker. See Administration Guide › Caching Large, Frequently Used Entries for further information. Caching small groups is a waste of memory as the memory devoted to the entry cache is not available for other purposes.
  • Consider using modify-add operations instead of modify-replace operations. See Developer's Guide › Modify: Adding Attributes for further information.

You should ensure you fully test performance using the addrate, authrate, modrate, searchrate tools described in the Administration Guide › Testing Performance.

See Also

FAQ: General DS/OpenDJ

How do I know what index types are needed for search filters in DS/OpenDJ (All versions)?

How do I troubleshoot issues with my indexes in DS/OpenDJ (All versions)?

How do I use the Support Extract tool in DS 5.x, 6 and OpenDJ 3.x to capture troubleshooting data?

How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache?

Performance tuning and monitoring ForgeRock products

Installing and Administering DS/OpenDJ

Administration Guide › Indexing Attribute Values

Configuration Reference › Entry Cache

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I append data to an existing user store in DS/OpenDJ (All versions)?

The purpose of this article is to provide assistance in adding bulk user data to an existing user data store in DS/OpenDJ.

Overview

The Administration guide discusses using import-ldif to add a bulk load of data to a user store. However, you should be aware that import-ldif removes all existing data before importing the new data, which is problematic if you have existing data that needs to remain in place after the import.

If you want to add bulk data, there are two approaches you can take depending on your environment and the amount of data you want to add:

  • For small amounts of data, you can use ldapmodify, which allows you to use an ldif file to append entries to an existing user store. However, ldapmodify runs each entry in the ldif as an individual modification, which can have a bigger impact on performance compared to using import-ldif with very large files. See the Using ldapmodify to append data section for details.
  • For large amounts of data, you should use import-ldif. To ensure you don't lose any data, you can export the existing entries, merge them with your new entries and then import the resulting merged ldif. See the Using import-ldif to append data section for details.
Caution

There are no definitive limits to the amount of data and likely impact on performance; this will depend on your environment, so you should performance test and choose the most appropriate option.

Using ldapmodify to append data

You can use ldapmodify to append data as follows:

  1. Create an ldif file containing the data you want to append (called entries.ldif in this example).
  2. Apply the changes using the following ldapmodify command depending on your version:
    • DS 5 and later:
      $ ./ldapmodify --hostname ds1.example.com --port 1389 --bindDN "cn=Directory Manager" --bindPassword password /path/to/entries.ldif
    • Pre-DS 5:
      $ ./ldapmodify --hostname ds1.example.com --port 1389 --bindDN "cn=Directory Manager" --bindPassword password --filename /path/to/entries.ldif

See Developer's Guide › Modifying Entry Attributes for further information.

Note

The ldapmodify operation will fail if any of your ldif entries match existing entries; you can use the --continueOnError option to continue even if an error is encountered.

Using import-ldif to append data

You can use import-ldif to append data as follows:

  1. Create an ldif file containing the data you want to append (called newentries.ldif in this example).
  2. Export the existing entries using export-ldif, for example (while the server is online): 
    $ ./export-ldif --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backendID userRoot --includeBranch dc=example,dc=com --ldifFile existing.ldif --start 0 --trustAll
    
  3. Merge the entries in the ldif files created in steps 1 and 2 using the appropriate ldifmodify command depending on your version, for example:
    • DS 5 and later:
      $ ./ldifmodify --outputLDIF merged.ldif existing.ldif newentries.ldif
      
    • Pre-DS 5:
      $ ./ldifmodify --sourceLDIF existing.ldif --changesLDIF newentries.ldif --targetLDIF merged.ldif
      
  4. Import the resulting ldif file using import-ldif, for example: 
    $. /import-ldif --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backendID userRoot --includeBranch dc=example,dc=com --ldifFile merged.ldif --trustAll
    
Note

You can add the --continueOnError option when you run the ldifmodify command if you have duplicate entries in your new ldif file. Attributes in duplicate entries will not be updated as these entries are ignored when merging the files. 

See Also

Installing and Administering DS/OpenDJ

Administration Guide › Importing and Exporting Data

Administration Guide › Other Tools For Working With LDIF Data

OpenDJ 3 › Release Notes › Important Changes to Existing Functionality › import-ldif 

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I configure mapping for Pass Through Authentication (PTA) in DS/OpenDJ (All versions) to Active Directory?

The purpose of this article is to provide information on configuring mapping for Pass Through Authentication (PTA) in DS/OpenDJ to Active Directory®.

Overview

In DS / OpenDJ 3.5, you can map different attributes, such as uid in DS/OpenDJ to sAMAccountName in Active Directory.

In OpenDJ 3.0 and earlier, you must use the same mapping attribute in both OpenDJ and Active Directory; you cannot map different attributes, such as uid in OpenDJ to sAMAccountName in Active Directory.

Configuring mapping for PTA

PTA is configured using dsconfig as described in Administration Guide › Configuring Pass-Through Authentication to set the required mapped-attributes.

DS / OpenDJ 3.5

You can set the mapped-search-filter-template to a value such as "(samAccountName=%s)" and the mapped-attribute to "uid". During a search, the local entry's value for the mapped-attribute replaces %s in the mapped-search-filter-template, meaning you can map samAccountName to uid.

OpenDJ 3.0 and earlier

OpenDJ uses the attribute value specified for the ds-cfg-mapped-attribute configuration element to search AD with. You can use either mapped-search or mapped-bind for the ds-cfg-mapping-policy configuration element.

The mapped-search or mapped-bind mapping-policy works providing the following requirements are met:

  • The PTA configuration has a ds-cfg-mapped-attribute with a value. For example:
    ds-cfg-mapped-attribute: cn
  • The OpenDJ user must have the mapped attribute with a value that matches the attribute:value pair on the AD side. For example, both the OpenDJ user and AD user have:
    cn: OpenDJ User

For example, the OpenDJ User could have:

dn: uid=opendj.sustaining,ou=People,dc=forgerock,dc=com 
cn: OpenDJ Sustaining

or

dn: uid=opendj.sustaining,ou=People,dc=forgerock,dc=com 
sAMAccountName: OpenDJ Sustaining

And the AD User would have:

dn: CN=OpenDJ Sustaining,CN=Users,DC=forgerock,DC=com 
cn: OpenDJ Sustaining 
sAMAccountName: OpenDJ Sustaining

The following examples show how this mapping works for OpenDJ 3.0 and earlier.

Example 1

In this example, ds-cfg-mapped-attribute: cn will use the OpenDJ's users "cn" attribute:value pair to search against AD. Providing AD has the same attribute:value pair and the search returns result=0 (success) then the user can authenticate to AD.

OpenDJ PTA Configuration

dn: cn=AD PTA Policy,cn=Password Policies,cn=config
objectClass: top
objectClass: ds-cfg-authentication-policy
objectClass: ds-cfg-ldap-pass-through-authentication-policy
ds-cfg-mapped-search-bind-password: password
ds-cfg-mapped-search-base-dn: CN=Users,DC=example,DC=com
ds-cfg-mapped-attribute: cn
ds-cfg-mapped-search-bind-dn: CN=Administrator,CN=Users,DC=example,DC=com
cn: AD PTA Policy
ds-cfg-java-class: org.opends.server.extensions.LDAPPassThroughAuthenticationPolicyFactory
ds-cfg-mapping-policy: mapped-search
ds-cfg-use-password-caching: false
ds-cfg-primary-remote-ldap-server: ad.example.com:389

DJ User (with cn: OpenDJ User)

dn: uid=opendj.user,ou=People,dc=forgerock,dc=com
objectClass: person
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: top
mail: opendj.user@forgerock.com
givenName: OpenDJ
uid: opendj.user
cn: OpenDJ User

AD User (with cn: OpenDJ User)

dn: CN=OpenDJ User,CN=Users,DC=example,DC=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: OpenDJ User
sn: User
givenName: OpenDJ
distinguishedName: CN=OpenDJ User,CN=Users,DC=example,DC=com
name: OpenDJ User
sAMAccountName: OpenDJ User

When a user BINDs to OpenDJ, it will use the value defined in the ds-cfg-mapped-attribute in the config.ldif. In this case, it will retrieve the OpenDJ users "cn" attribute and issue a search against AD with the same attribute:value pair:

  1. User "uid=opendj.user" BINDs to OpenDJ.
  2. OpenDJ retrieves the cn value from the user uid: opendj.user.
  3. OpenDJ searches AD with --baseDN <suffix> "(cn=OpenDJ User)"
  4. If the search return is 0 (success) the PTA Plugin authenticates the user to AD.

Providing the OpenDJ user has the same attribute:value pair that is contained on the AD side, the PTA Plugin can utilize that attribute:value combination.

Example 2

In this example, the following PTA configuration will work as long as the attribute specified for ds-cfg-mapped-attribute (sAMAccountName) is available and contains the same value on both the OpenDJ server as well as the AD sever as seen in the following configurations.

OpenDJ PTA Configuration

dn: cn=AD PTA Policy,cn=Password Policies,cn=config
objectClass: top
objectClass: ds-cfg-authentication-policy
objectClass: ds-cfg-ldap-pass-through-authentication-policy
ds-cfg-mapped-search-bind-password: password
ds-cfg-mapped-search-base-dn: CN=Users,DC=example,DC=com
ds-cfg-mapped-attribute: sAMAccountName
ds-cfg-mapped-search-bind-dn: CN=Administrator,CN=Users,DC=example,DC=com
cn: AD PTA Policy
ds-cfg-java-class: org.opends.server.extensions.LDAPPassThroughAuthenticationPolicyFactory
ds-cfg-mapping-policy: mapped-search
ds-cfg-use-password-caching: false
ds-cfg-primary-remote-ldap-server: ad.example.com:389

OpenDJ User (with sAMAccountName: OpenDJ User)

dn: uid=opendj.user,ou=People,dc=forgerock,dc=com
objectClass: person
objectClass: inetOrgPerson
objectClass: organizationalPerson
objectClass: top
mail: opendj.user@forgerock.com
givenName: OpenDJ
uid: opendj.user
cn: OpenDJ User
sn: User
sAMAccountName: OpenDJ User

AD User (with sAMAccountName: OpenDJ User)

dn: CN=OpenDJ User,CN=Users,DC=example,DC=com
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: user
cn: OpenDJ User
sn: Sustaining
givenName: OpenDJ
distinguishedName: CN=OpenDJ User,CN=Users,DC=example,DC=com
displayName: OpenDJ User
name: OpenDJ User
sAMAccountName: OpenDJ User

See Also

Administration Guide › Configuring Pass-Through Authentication

Configuration Reference › LDAP Pass Through Authentication Policy

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-1103 (Pass Through Authentication mapped-search-filter-template)

OPENDJ-1626 (Mapping attributes for passthrough authentication)


How do I configure DS (All versions) and OpenDJ 3.x to use the Syslog audit event handler?

The purpose of this article is to provide information on configuring DS/OpenDJ to use the Syslog audit event handler; it covers both an external and a local configuration. This article assumes you already have a working Syslog server configured.

Overview

You can configure the Syslog audit event handler as either an external or local configuration as detailed in the following sections:

Note

You must ensure Syslog is correctly set up before configuring DS/OpenDJ to use the Syslog audit event handler. See RSyslog Documentation for further information, ensuring that you restart the Syslog server after making any changes to the Syslog configuration.

DS 5 and later

In DS 5 and later, example configuration files can be found in the /path/to/ds/config/audit-handlers directory; they have an -example suffix and can be used as the basis of your configuration files.

You should be aware of the following limitation noted in Administration Guide › Configuring Access Logging to Syslog:

The implementation currently only supports writing access messages to Syslog, rather than error messages. As a result, this feature is of limited use in most deployments. 

Using an external Syslog audit event handler

You can configure DS/OpenDJ to use an external Syslog audit event handler as follows:

  1. Create a JSON configuration file for the Syslog audit event handler in the /path/to/ds/config/audit-handlers directory (DS) or the /path/to/opendj/config directory (OpenDJ 3.x). For example, your configuration file may look similar to this:
    {
       "class": "org.forgerock.audit.handlers.syslog.SyslogAuditEventHandler",
       "config": {
          "name": "External Syslog Event Handler",
          "enabled" : true,
          "topics": [
             "ldap-access"
          ],
          "protocol": "UDP",
          "host": "syslogd.forgerock.com",
          "port": 514,
          "connectTimeout": 60,
          "facility": "SYSLOG",
          "buffering": {
             "enabled": false
          }
       }
    }
    
  2. Configure DS/OpenDJ to use the external Syslog audit handler using the dsconfig command, for example:
    $ ./dsconfig create-log-publisher --port 4444 --hostname ds1.example.com --bindDN "cn=Directory Manager" --bindPassword password --publisher-name "External Syslog Publisher Access" --type external-access --set enabled:true --set config-file:/path/to/ds/config/audit-handlers/syslog-handler.json --trustAll --no-prompt 
  3. Restart the DS/OpenDJ server.

The audit logs should now be written to the /var/log/msgs directory on the remote syslogd server. 

Using a local Syslog audit event handler

You can configure DS/OpenDJ to use a local Syslog audit event handler as follows:

  1. Create a JSON configuration file for the Syslog audit event handler in the /path/to/ds/config/audit-handlers directory (DS) or the /path/to/opendj/config directory (OpenDJ 3.x). For example, your configuration file may look similar to this (observe the host is now set to localhost):
    {
       "class": "org.forgerock.audit.handlers.syslog.SyslogAuditEventHandler",
       "config": {
          "name": "Local Syslog Event Handler",
          "enabled" : true,
          "topics": [
             "ldap-access"
          ],
          "protocol": "UDP",
          "host": "localhost",
          "port": 514,
          "connectTimeout": 60,
          "facility": "SYSLOG",
          "buffering": {
             "enabled": false
          }
       }
    }
    
  2. Configure DS/OpenDJ to use the local Syslog audit handler using the dsconfig command, for example:
    $ ./dsconfig create-log-publisher --port 4444 --hostname ds1.example.com --bindDN "cn=Directory Manager" --bindPassword password --publisher-name "Local Syslog Publisher Access" --type external-access --set enabled:true --set config-file:/path/to/ds/config/audit-handlers/local-syslog-handler.json --trustAll --no-prompt 
  3. Restart the DS/OpenDJ server.

The audit logs should now be written to the /var/log/messages directory on the local server where DS/OpenDJ is running. 

See Also

Configuration Reference › create-log-publisher

RFC 5424 - The Syslog Protocol

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-4295 (Syslog data is not fully RFC compliant)


How do I change the location of log files for DS/OpenDJ (All versions)?

The purpose of this article is to provide information on changing the location and/or name of log files for DS/OpenDJ. By default, DS/OpenDJ stores the log files in the /path/to/ds/logs directory.

Changing the location of log files

DS/OpenDJ uses log publishers to define log details for each log type, including the log location and name. 

You can change the log location and/or log name by using dsconfig and updating the value of the log-file property. For example, this property defaults to logs/access for the access log, which is a relative path to the DS/OpenDJ directory.

If you run dsconfig in interactive mode, you can navigate to: 21) Log Publisher > 3) View and edit > [Log File Type] > 5) log-file and amend the name and location. You can enter a full path, such as /logs/ds, rather than one that is relative to the DS/OpenDJ directory if required.

Alternatively, you can run the dsconfig command. For example:

$ ./dsconfig set-log-publisher-prop --publisher-name [publisherName] --set log-file [filePath] --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password

Where [publisherName] is one of the following:

  • "File-Based Access Logger" - access log
  • "File-Based Audit Logger" - audit log
  • "File-Based Debug Logger" - debug log
  • "File-Based Error Logger" - errors log
  • "File-Based HTTP Access Logger" - http-access log 
Note

You can also configure the log retention and log rotation options in a similar way by navigating to 22) Log Retention Policy and 23) Log Rotation Policy when using dsconfig in interactive mode or by using the dsconfig set-log-retention-policy-prop and dsconfig set-log-rotation-policy-prop commands respectively.

See Also

Administration Guide › Monitoring, Logging, and Alerts › Server Logs

Reference › Tools Reference › dsconfig

Configuration Reference › set-log-publisher-prop

Related Training

N/A

Related Issue Tracker IDs

N/A


Patches


How do I check what patches are installed for ForgeRock products?

The purpose of this article is to provide details for checking what patches are installed for AM/OpenAM, DS/OpenDJ, IDM/OpenIDM and IG/OpenIG.

Overview

This article provides details for checking what patches are installed by product:

You can also use the patchinfo utility in AM/OpenAM and IG/OpenIG as detailed in How do I use the patchinfo utility to check what patches are installed for AM/OpenAM (All versions) or IG/OpenIG (All versions)? 

Checking for patches in AM/OpenAM

Providing you have unzipped your patches directly in the openam directory as recommended, you can check what patches are installed as follows:

$ cd /path/to/tomcat/webapps/openam/WEB-INF
$ ls README.*

README.1.12345    README.201608

Where each README represents a set of installed patches. READMEs are named to indicate either the ticket number where the patch was given (README.1.12345) or the security advisory ID (README.201608).

Embedded DS/OpenDJ

You can check what patches are installed using the status -V command as follows:

$ cd $HOME/[am_instance]/opends/bin
$ ./status -V

Wed Nov 29 11:27:09 MST 2017
ForgeRock Directory Services 5.5.0+OPENDJ-1234,OPENDJ-5678
Build 20171019141329

Any installed patches will be listed after the product name and version as an issue tracker ID (JIRA).

Checking for patches in Policy Agents

You can check the debug log to see what patch version you have installed, for example:

######################################################
 # OpenAM Web Agent                                   #
 # Version: 4.1.0-27                                  #
 # Revision: 39e5be3                                  #
 # Container: IIS 7.5, 8.x WINNT 32/64bit/WINNT       #
 # Build date: Nov  2 2017 13:42:43                   #
 ######################################################

For Web policy agents 4.x, you can also check the version using the agentadmin --v command as follows:

$ ./agentadmin --v

OpenAM Web Agent for Apache Server 2.2.x 64bit 
Version: 4.1.0-25 
Revision: 42ff535 
Build machine: delacroix 
Build date: Sep 15 2017 16:44:11

Checking for patches in DS/OpenDJ

You can check what patches are installed using the status -V command as follows:

$ cd /path/to/ds/bin
$ ./status -V

Wed Nov 29 11:27:09 MST 2017
ForgeRock Directory Services 5.5.0+OPENDJ-1234,OPENDJ-5678
Build 20171019141329

Any installed patches will be listed after the product name and version as an issue tracker ID (JIRA).

Checking for patches in IDM/OpenIDM

You can check what patches are installed using the PS command when running IDM/OpenIDM interactively, for example:

ps
START LEVEL 12
   ID   State         Level  Name
[   0] [Active     ] [    0] System Bundle (4.2.1)
[   1] [Resolved   ] [    1] javax.transaction API v.1.1 (3.1.1)
[   2] [Active     ] [    1] OpenIDM System Bundle (4.5.0)
...
[ 135] [Active     ] [   10] OpenIDM Core Bundle (4.5.0.OPENIDM-1234-5678)
...

Any installed patches will be listed after a product bundle as an issue tracker ID (JIRA). Patches only apply to a bundle (not the whole product), so you will need to check all bundles for patches, focusing on any that start OpenIDM, ForgeRock or OpenICF. You can search for version.JIRA_prefix to easily find the patched bundles, for example:

(4.5.0.OPENIDM
(4.5.0.OPENICF
(4.5.0.CREST

Checking for patches in IG/OpenIG

Providing you have unzipped your patches directly in the top-level IG/OpenIG WAR directory as recommended, you can check what patches are installed as follows:

$ cd /path/to/jetty/webapps/root/WEB-INF
$ ls README.*

README.1.12345    README.201606

Where each README represents a set of installed patches. READMEs are named to indicate either the ticket number where the patch was given (README.1.12345) or the security advisory ID (README.201606).

See Also

How do I install an AM/OpenAM patch (All versions) supplied by ForgeRock support?

FAQ: Patches in AM/OpenAM

How do I install a DS/OpenDJ patch (All versions) supplied by ForgeRock support?

How do I install an IDM/OpenIDM patch (All versions) supplied by ForgeRock support?

How do I install an IG/OpenIG patch (All versions) supplied by ForgeRock support?

Related Training

N/A

Related Issue Tracker IDs

OPENAM-2651 (Formalized patching mechanism)


How do I install a DS/OpenDJ patch (All versions) supplied by ForgeRock support?

The purpose of this article is to provide information on installing a DS/OpenDJ patch that has been supplied by ForgeRock support. The instructions differ depending on whether you are using a standalone DS/OpenDJ or an embedded DS/OpenDJ (in AM/OpenAM or IDM 6 and later).

Overview

This article covers installing DS/OpenDJ patches in the following setups:

Installing a patch for standalone DS/OpenDJ

The following process to install a DS/OpenDJ patch includes steps to create a Backout file. Having a Backout file means you can reverse the patch changes if needed and restore the previous /classes directory. 

Note

If you have installed the instance files separately to the install files (How do I install DS/OpenDJ (All versions) so that the instance files are separate to the install files?), the instance.loc file indicates where the instance files are stored.​ You should follow these instructions but substitute the directory specified in the instance.loc file for the /path/to/ds directory to ensure the patch is installed in the instance.loc's classes directory. 

You can install a DS/OpenDJ patch as follows:

  1. Create a Backout file as follows:
    $ cd /path/to/ds
    $ tar cf backout-patch.tar classes
    $ rm -fr classes
  2. Copy the supplied patch zip to the /tmp directory.
  3. Extract the patch zip:
    $ cd /path/to/ds
    $ unzip /tmp/OpenDJ-x.x.x-OPENDJ-xxxx.zip
  4. Stop and restart the DS/OpenDJ server:
    $ ./stop-ds
    $ ./start-ds

Verify the patch installation

You can verify the patch has been installed correctly as follows:

$ ./status -V

Example response, where DS/OpenDJ version and patch ID corresponds to the patch you just installed:

OpenDJ 3.0.0+OPENDJ-1234
Build 20160501120000Z

Back out the patch

You can back out a patch as follows:

$ cd /path/to/ds
$ rm -fr classes
$ tar xf backout-patch.tar
$ ./stop-ds
$ ./start-ds

Installing a patch for embedded DS/OpenDJ in AM/OpenAM

Apache Tomcat™

If you use the dsconfig tool with your embedded DS/OpenDJ, you must apply the patch in two places in order that the dsconfig tool continues to work after installing the patch.

  1. Back up the /path/to/tomcat/webapps/openam/WEB-INF/classes directory where AM/OpenAM is deployed. If you use the dsconfig tool, you should also back up the $HOME/[am_instance]/opends/classes directory.
  2. Check the /path/to/tomcat/webapps/openam/WEB-INF/classes directory where AM/OpenAM is deployed to ensure that the classes contained in the patch do not already exist in the directory. If one or more classes do already exist, it may mean you have a conflicting patch installed. If this is the case, you should seek advice from ForgeRock support prior to applying the patch. 
  3. Stop the web application container in which AM/OpenAM runs. 
  4. Go to the /path/to/tomcat/webapps/openam/WEB-INF directory where AM/OpenAM is deployed:
    $ cd /path/to/tomcat/webapps/openam/WEB-INF
  5. Extract the patch zip:
    $ unzip /path/to/patch/X.zip
  6. If you use the dsconfig tool, you should go to the $HOME/[am_instance]/opends directory and extract the patch zip:
    $ cd $HOME/[am_instance]/opends
    $ unzip /path/to/patch/X.zip
  7. Restart the web application container in which AM/OpenAM runs.
Note

You can remove the patch by replacing the /classes directory with your backup.

Oracle® WebLogic Server

 The patch must be applied to both the AM/OpenAM war file on the WebLogic web application container and the embedded DS/OpenDJ directory ($HOME/[am_instance]/opends). The same steps apply regardless of whether AM/OpenAM has been deployed on a Weblogic admin server or a Weblogic managed server.

  1. Stop the web application container in which AM/OpenAM runs.
  2. Copy the supplied patch zip to the /tmp directory.
  3. Make a backup of the original openam.war file:
    $ cp /path/to/weblogic/openam.war /path/to/weblogic/openam.war.original
  4. Create a working directory called ToBePatched:
    $ mkdir /path/to/weblogic/ToBePatched
  5. Copy the openam.war file to the ToBePatched directory:
    $ cp openam.war /path/to/weblogic/ToBePatched/openam.war
  6. Go to the ToBePatched directory:
    $ cd /path/to/weblogic/ToBePatched
  7. Expand the openam.war file:
    $ JAVA_HOME/bin/jar xvf openam.war
    This creates a directory structure with the contents of the openam.war file.
  8. Delete the openam.war file from the ToBePatched directory:
    $ rm /path/to/weblogic/ToBePatched/openam.war
  9. Extract the patch zip into the expanded war file directory:
    $ cd /path/to/weblogic/ToBePatched/WEB-INF/ 
    $ unzip /tmp/OpenDJ-x.x.x-OPENDJ-xxxx.zip
  10. Delete the original openam.war file:
    $ rm /path/to/weblogic/openam.war
    
  11. Go to the ToBePatched directory and repack the new openam.war file with the patch included:
    $ cd /path/to/weblogic/ToBePatched/
    $ $JAVA_HOME/bin/jar cvf /path/to/weblogic/openam.war *
    
  12. Go to the embedded DS/OpenDJ directory ($HOME/[am_instance]/opends) and extract the patch zip:
    $ cd $HOME/[am_instance]/opends 
    $ unzip /tmp/OpenDJ-x.x.x-OPENDJ-xxxx.zip
  13. Restart the web application container in which AM/OpenAM runs.
  14. Verify the patch installation using the status command:
    $ $HOME/[am_instance]/opends/bin/status -V
    Example response, where DS/OpenDJ version and patch ID corresponds to the patch you just installed:
    OpenDJ 3.0.0+OPENDJ-1234
    Build 20160501120000Z

Installing a patch for embedded DS in IDM 6 and later

The following process to install a patch for embedded DS includes steps to create a Backout file. Having a Backout file means you can reverse the patch changes if needed and restore the previous /classes directory. 

You can install a patch as follows:

  1. Create a Backout file as follows:
    $ cd ./db/openidm/opendj
    $ tar cf backout-patch.tar classes
    $ rm -fr classes
  2. Copy the supplied patch zip to the /tmp directory.
  3. Extract the patch zip:
    $ cd ./db/openidm/opendj
    $ unzip /tmp/OpenDJ-x.x.x-OPENDJ-xxxx.zip
  4. Shutdown and restart the IDM instance.
    $ cd /path/to/idm
    $ ./shutdown.sh
    $ ./startup.sh
    

Verify the patch installation

You can verify the patch has been installed correctly as follows:

$ cd ./db/openidm/opendj/bin
$ ./status -V

Example response, where DS version and patch ID corresponds to the patch you just installed:

ForgeRock Directory Services 6.0.0+OPENDJ-1234,OPENDJ-5678
Build 20180626201741

Back out the patch

You can back out a patch as follows:

$ cd ./db/openidm/opendj
$ rm -fr classes
$ tar xf backout-patch.tar
$ cd /path/to/idm
$ ./shutdown.sh
$ ./startup.sh

See Also

Maintenance and Patch availability policy

How do I upgrade DS/OpenDJ if I have support patches installed (All versions)?

How do I install an AM/OpenAM patch (All versions) supplied by ForgeRock support?

How do I check what patches are installed for ForgeRock products?

Related Training

N/A

Related Issue Tracker IDs

N/A


Access Controls


How do I know what the default Global ACIs are used for in DS/OpenDJ (All versions)?

The purpose of this article is to provide information on what the default Global ACIs are used for in DS/OpenDJ and whether they can be removed or modified.

Default Global ACIs

Caution

Some Global ACIs must not be removed or modified, whereas some may be removed and others may be removed but their removal is not recommended. In all cases, you must test your changes in a pre-production environment first to ensure there is no adverse impact.

The default Global ACIs have been categorized below according to whether they can be removed or not:

Modification or removal is permitted; must be tested

The following default Global ACIs exist in this category:

  • Anonymous "Read" access - allows anonymous read access to most user data attributes:
    ds-cfg-global-aci: (targetattr!="userPassword||authPassword||changes||changeNumber||changeType||changeTime||targetDN||newRDN||newSuperior||deleteOldRDN")(version 3.0; acl "Anonymous read access"; allow (read,search,compare) userdn="ldap:///anyone";)
    
  • Self-entry "Write" access - allows authenticated users to modify each of these attributes on their own entry:
    ds-cfg-global-aci: (targetattr="audio||authPassword||description||displayName||givenName||homePhone||homePostalAddress||initials||jpegPhoto||labeledURI||mobile||pager||postalAddress||postalCode||preferredLanguage||telephoneNumber||userPassword")(version 3.0; acl "Self entry modification"; allow (write) userdn="ldap:///self";)
    
  • Self-entry Password value "Read" access - allows authenticated users to read password values on their own entries after binding; password values are hashed by default:
    ds-cfg-global-aci: (targetattr="userPassword||authPassword")(version 3.0; acl "Self entry read"; allow (read,search,compare) userdn="ldap:///self";)

Modification or removal may affect applications; must be tested

The following default Global ACIs exist in this category:

  • Anonymous Extended Operations "Read" access - allows anonymous and authenticated users to request the LDAP extended operations that are specified by OID: See Reference › LDAP Extended Operations for further information.
    ds-cfg-global-aci: (extop="1.3.6.1.4.1.26027.1.6.1 || 1.3.6.1.4.1.26027.1.6.3 || 1.3.6.1.4.1.4203.1.11.1 || 1.3.6.1.4.1.1466.20037 || 1.3.6.1.4.1.4203.1.11.3") (version 3.0; acl "Anonymous extended operation access"; allow(read) userdn="ldap:///anyone";)
  • Anonymous LDAP Control "Read" access - allows anonymous and authenticated users to use the LDAP controls that are specified by OID: See Reference › LDAP Controls for further information.
    ds-cfg-global-aci: (targetcontrol="2.16.840.1.113730.3.4.2 || 2.16.840.1.113730.3.4.17 || 2.16.840.1.113730.3.4.19 || 1.3.6.1.4.1.4203.1.10.2 || 1.3.6.1.4.1.42.2.27.8.5.1 || 2.16.840.1.113730.3.4.16 || 1.2.840.113556.1.4.1413") (version 3.0; acl "Anonymous control access"; allow(read) userdn="ldap:///anyone";)
  • Authenticated users LDAP Control "Read" access - allows authenticated users to use the LDAP controls that are specified by OID: See Reference › LDAP Controls for further information.
    ds-cfg-global-aci: (targetcontrol="1.3.6.1.1.12 || 1.3.6.1.1.13.1 || 1.3.6.1.1.13.2 || 1.2.840.113556.1.4.319 || 1.2.826.0.1.3344810.2.3 || 2.16.840.1.113730.3.4.18 || 2.16.840.1.113730.3.4.9 || 1.2.840.113556.1.4.473 || 1.3.6.1.4.1.42.2.27.9.5.9") (version 3.0; acl "Authenticated users control access"; allow(read) userdn="ldap:///all";)
    
  • Anonymous Schema-related Operational Attributes "Read" access - allows anonymous and authenticated users to read LDAP schema definitions:
    ds-cfg-global-aci: (target="ldap:///cn=schema")(targetscope="base")(targetattr="objectClass||attributeTypes||dITContentRules||dITStructureRules||ldapSyntaxes||matchingRules||matchingRuleUse||nameForms||objectClasses")(version 3.0; acl "User-Visible Schema Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)
    
  • Anonymous User-visible Operational Attributes "Read" access - allows anonymous and authenticated users to read operational attributes related to entry updates and entry identification:
    ds-cfg-global-aci: (targetattr="createTimestamp||creatorsName||modifiersName||modifyTimestamp||entryDN||entryUUID||subschemaSubentry||etag||governingStructureRule||structuralObjectClass||hasSubordinates||numSubordinates")(version 3.0; acl "User-Visible Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)

Modification or removal may affect applications and is not recommended; must be tested

The following default Global ACI exists in this category:

  • Anonymous root DSE attributes "Read" access - allows anonymous and authenticated users to read attributes that describe what features the server supports:
    ds-cfg-global-aci: (target="ldap:///")(targetscope="base")(targetattr="objectClass||namingContexts||supportedAuthPasswordSchemes||supportedControl||supportedExtension||supportedFeatures||supportedLDAPVersion||supportedSASLMechanisms||supportedTLSCiphers||supportedTLSProtocols||vendorName||vendorVersion")(version 3.0; acl "User-Visible Root DSE Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)
    

Must not be modified or deleted

The following default Global ACI exists in this category:

  • Denied "Replication backend" data access:
    ds-cfg-global-aci: (target="ldap:///dc=replicationchanges")(targetattr="*")(version 3.0; acl "Replication backend access"; deny (all) userdn="ldap:///anyone";)
    

See Also

How do I only allow selected users to search, update and delete LDAP entries in DS/OpenDJ (All versions)?

How do I prevent anonymous access in DS/OpenDJ (All versions)?

Installing and Administering DS/OpenDJ

Administration Guide › Configuring Privileges and Access Control

Related Training

ForgeRock Directory Services Core Concepts

Related Issue Tracker IDs

N/A


How do I prevent anonymous access in DS/OpenDJ (All versions)?

The purpose of this article is to show the most effective way of preventing anonymous access from LDAP clients in DS/OpenDJ.

Preventing anonymous access

A common security requirement is that all LDAP clients connecting to your server must authenticate, which allows for proper auditing. However the default configuration of DS/OpenDJ allows unauthenticated or anonymous connections.

A very common strategy for LDAP clients is to connect to a server, retrieve information about the server’s capabilities from the root entry, and then authenticate to the server based on the server's capabilities. If a client is not allowed to retrieve this information it will usually just fail. Additionally, clients doing heartbeat-style checks on the server often do unauthenticated reads of the root entry, so completely disabling unauthenticated connections is not recommended.

The recommended approach is to allow unauthenticated connections (that is, the default) and to use access controls to permit access to the root entry, and only the root entry. To achieve this, you need to grant access to the root entry in the server’s global-aci using the dsconfig command, where the ACI value after --add is a single long line with no line breaks. For example:

$ ./dsconfig set-access-control-handler-prop --add 'global-aci:(target="ldap:///")(targetscope="base")(targetattr="objectClass||namingContexts||supportedAuthPasswordSchemes||supportedControl||supportedExtension||supportedFeatures||supportedLDAPVersion||supportedSASLMechanisms||vendorName||vendorVersion")(version 3.0; acl "User-Visible Root DSE Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)' --hostname server.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --trustAll --no-prompt
Note

The attributes listed in the targetattr are ones that are widely used when checking server capabilities. You might find you need to add additional attributes to this list; searching the DS/OpenDJ access logs for searches to base="" will show the list of attributes that the LDAP clients are trying to return.

You must also remove the default access granted to anonymous users using a similar dsconfig command. A new attribute was added in OpenDJ 3 (debugsearchindex), which changes the command slightly. For example:

  • DS / OpenDJ 3.x:
    $ ./dsconfig set-access-control-handler-prop --remove 'global-aci:(targetattr!="userPassword||authPassword||debugsearchindex||changes||changeNumber||changeType||changeTime||targetDN||newRDN||newSuperior||deleteOldRDN")(version 3.0; acl "Anonymous read access"; allow (read,search,compare) userdn="ldap:///anyone";)' --hostname server.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --trustAll --no-prompt
    
  • OpenDJ 2.6.x:
    $ ./dsconfig set-access-control-handler-prop --remove 'global-aci:(targetattr!="userPassword||authPassword||changes||changeNumber||changeType||changeTime||targetDN||newRDN||newSuperior||deleteOldRDN")(version 3.0; acl "Anonymous read access"; allow (read,search,compare) userdn="ldap:///anyone";)' --hostname server.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --trustAll --no-prompt
    

If you have other custom ACIs that affect anonymous users, you will also need to review them and update as necessary.

Caution

Don't forget that any changes to the configuration using dsconfig will need to be repeated on all appropriate DS/OpenDJ instances.

Note

You can also achieve this by globally disabling unauthenticated connections using set-global-configuration-prop. However, this approach is not recommended because it prevents access to the root entry, which can cause other processes to fail, including connectors from OpenAM when they are configured to use the heartbeat mechanism (default).

See Also

How do I know what the default Global ACIs are used for in DS/OpenDJ (All versions)?

How do I only allow selected users to search, update and delete LDAP entries in DS/OpenDJ (All versions)?

Administration Guide › Configuring Privileges and Access Control › Configuring ACIs

Administration Guide › Configuring Privileges and Access Control › ACI Targets

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I only allow selected users to search, update and delete LDAP entries in DS/OpenDJ (All versions)?

The purpose of this article is to provide information on using ACIs to allow selected users to search, update and delete LDAP entries in DS/OpenDJ.

Controlling access

You can configure DS/OpenDJ to allow selected users to search, update and delete LDAP entries using access controls (ACIs). By default, there are a number of global ACIs configured that allow generic anonymous access, and authenticated read and search access. Global ACIs affect access across all backend databases. See How do I know what the default Global ACIs are used for in DS/OpenDJ (All versions)? for further information. You can modify or remove these ACIs, if required, to prevent this access.

Caution

You should test any changes you make to the default global ACIs first in a pre-production environment to ensure there is no adverse impact.

A typical default global ACI to remove when configuring access is the one that permits anonymous read access. You can remove this with one of the following dsconfig commands; a new attribute was added in OpenDJ 3 (debugsearchindex), which changes the command slightly. For example:

  • DS / OpenDJ 3.x:
    $ ./dsconfig set-access-control-handler-prop --remove 'global-aci:(targetattr!="userPassword||authPassword||debugsearchindex||changes||changeNumber||changeType||changeTime||targetDN||newRDN||newSuperior||deleteOldRDN")(version 3.0; acl "Anonymous read access"; allow (read,search,compare) userdn="ldap:///anyone";)' --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --trustAll --no-prompt
    
  • OpenDJ 2.6.x:
    $ ./dsconfig set-access-control-handler-prop --remove 'global-aci:(targetattr!="userPassword||authPassword||changes||changeNumber||changeType||changeTime||targetDN||newRDN||newSuperior||deleteOldRDN")(version 3.0; acl "Anonymous read access"; allow (read,search,compare) userdn="ldap:///anyone";)' --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --trustAll --no-prompt
    

You can remove other ACIs as needed using the above dsconfig commands, where you just replace this ACI with the one you want to remove.

Allowing selected access

When granting access to selected users, it is preferable to use local ACIs (that is, for specific backends) rather than global ACIs as this gives greater control over access. Granting access to all backends via a global ACI may also be considered a security risk.

Note

It is good practice to put the ACI in the actual backend, as this means it will be replicated and also included when you export the data. Global ACIs are not replicated and are easy to forget when you're creating new instances.

You can add local ACIs using the ldapmodify command. The following example command provides the admin user (uid=admin) with access to search, update and delete LDAP entries under dc=example,dc=com:

dn: dc=example,dc=com
objectClass: domain
objectClass: top
dc: example
aci: (target="ldap:///dc=example,dc=com")(targetattr = "* || +")(version 3.0; acl "Admin user access to suffix"; allow (all, export, import)(userdn="ldap:///uid=admin,ou=People,dc=example,dc=com");)

You can add a global ACI that achieves the same thing as above but for all backends, using the dsconfig command:

$ ./dsconfig set-access-control-handler-prop --add 'global-aci:(target="ldap:///dc=example,dc=com")(targetscope="subtree")(targetattr="* || +")(version 3.0; acl "Admin user access to suffix"; allow (all, export, import) userdn="ldap:///uid=admin,ou=People,dc=example,dc=com ";)' --hostname ds1.example.com --port 4444 --trustAll --bindDN "cn=Directory Manager" --bindPassword password --no-prompt

See Also

How do I know what the default Global ACIs are used for in DS/OpenDJ (All versions)?

How do I prevent anonymous access in DS/OpenDJ (All versions)?

Administration Guide › Configuring Privileges and Access Control

Administration Guide › Configuring Privileges and Access Control › How ACI is Evaluated

Reference › Tools Reference › dsconfig

Configuration Reference › set-access-control-handler-prop

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I create a new admin user in DS 5.x and OpenDJ 3.x?

The purpose of this article is to provide information on creating a new administrator in DS/OpenDJ. You can either give admin privileges to a regular user or create a new admin user in the RootDN with the same privileges as the Directory Manager user.

Creating a new admin user

Note

This article only applies to DS 5.x and OpenDJ 3.x. Changes were introduced in DS 6 where the root DN users no longer belong to a special group or have alternate names, nor are their accounts stored in the configuration file, config.ldif. Instead, directory superusers, such as cn=Directory Manager, are now stored in their own, separate backends whose base DN is the user DN. See DS 6 › Release Notes › Important Changes to Existing Functionality for further information. 

As a result of this change, the procedure to create a new admin user has changed. See the following documentation for DS 6.x instructions:

There are three approaches you can take to create an admin user:

Create a new admin user in the RootDN

You can use the following process to create a new admin user:

  1. Create an admin.ldif file with the new admin user's details, ensuring they are below cn=Root DNs,cn=config and you specify: objectClass: ds-cfg-root-dn-user. For example:
    dn: cn=newAdminUser,cn=Root DNs,cn=config
    changetype: add
    objectClass: top
    objectClass: person
    objectClass: organizationalPerson
    objectClass: inetOrgPerson
    objectClass: ds-cfg-root-dn-user
    cn: newAdminUser
    givenName: newAdmin
    sn: User1
    ds-pwp-password-policy-dn: cn=Root Password Policy,cn=Password Policies,cn=config
    userPassword: password
    ds-cfg-alternate-bind-dn: cn=newAdminUser
    ds-rlim-size-limit: 0
    ds-rlim-time-limit: 0
    ds-rlim-idle-time-limit: 0
    ds-rlim-lookthrough-limit: 0
  2. Add the new admin user using ldapmodify, for example:
    $ ./ldapmodify --port 1389 --bindDN "cn=Directory Manager" --bindPassword password admin.ldif
    
    In OpenDJ 3.5.x and earlier, you must specify the -f option before the filename.

Your new admin user will be created with the same privileges as the Directory Manager user.

See Also

How do I change the admin account password used for replication in DS/OpenDJ (All versions)?

How do I know what the default Global ACIs are used for in DS/OpenDJ (All versions)?

Installing and Administering DS/OpenDJ

Security Guide › Managing Administrator Accounts

Security Guide › Assigning Administrative Privileges

Security Guide › Using ACIs or Global Access Policies

Related Training

N/A

Related Issue Tracker IDs

N/A


Back Up and Restore


How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

The purpose of this article is to provide information to help you design and implement your backup and restore strategies in DS/OpenDJ.

Backup Strategies

Each DS/OpenDJ integration is different and the backup strategies must be evaluated based on your business requirements.

Full Backups

Full backups are normally done on a scheduled basis, but they are considered a Point In Time (PIT) backup that is not 100% up to date.

Considerations:

  • Take the DS/OpenDJ instance out of the load balancer until the backup is complete.
  • Include the Other Required Files (see Other Required Files section below) by manually copying these files to a safe location.
  • Ensure you have enough disk space to store sufficient backup snapshots to adhere to your business requirements.
  • Consider splitting the suffixes into separate backend databases if you have multiple suffixes and you expect your database to be massive. Each backup will be smaller and take less time to complete. It will also allow restoration and maintenance of individual backends without affecting the rest.
  • Consider adding Incremental Backups (see the Incremental Backups sub-section below) to augment the full backup. This will allow for a more complete restore point, but can take more time as each incremental backup is restored.

Frequency:

  • It is recommended to create full backups at least once a week with daily incremental backups.
  • It is recommended to create full backups at least once per purge delay interval with incremental backups more frequently.

Incremental Backups

Incremental backups can be used exclusively to back up data over time, but each and every incremental backup from the point of the last full backup must be applied on top of each other to fully restore the database to a usable state.

Considerations:

  • Consider using a daily full backup followed by incremental backups every six hours or less.

Frequency:

  • Daily incremental backups are recommended.
  • The frequency can be increased if the business requirements stipulate more up to date backups are needed.

Expectations:

When entry counts are small, incremental backups may appear to look like full backups in the logs. 

In the following example, the backup contained only one database file and it was thought the incremental backup looked more like a full backup. In truth, the entry counts were small and only one database jdb file was being used:

ds-task-backup-incremental: true
ds-task-log-message: [19/Dec/2018:10:00:00 +0000] severity="NOTICE" msgCount=0 msgID=9896349 message="Backup task BackupTask-3294216c-2ed8-4f08-98b9-757ac
a98fcf1-20150319100000000 started execution"
ds-task-log-message: [19/Dec/2018:10:00:00 +0000] severity="NOTICE" msgCount=1 msgID=10944792 message="Starting backup for backend userRoot"
ds-task-log-message: [19/Dec/2018:10:00:00 +0000] severity="NOTICE" msgCount=2 msgID=8847442 message="Not changed: 00000000.jdb"
ds-task-log-message: [19/Dec/2018:10:00:00 +0000] severity="NOTICE" msgCount=3 msgID=10944795 message="The backup process completed successfully"

The number of jdb files present can vary depending on the DS/OpenDJ version you are using, the number of entries in the database and the size of the entries. The default database size (db-log-file-max) varies by version:

  • DS 6 and later - default database size 1 GB.
  • DS 5.x / OpenDJ 2.6.2 and later - default database size 100 MB.
  • OpenDJ 2.6.0 and 2.6.1 - default database size 10 MB.

The following test was completed on non-replicated DS 5 instance with no special configuration (default database size of 100 MB). When example template entries are added with this database file size, a new database file will be created after approximately 56800+ entries. In tests, the second jdb file was created when between 56802 and 56902 entries were imported:

-rw-r--r--  1 opendj  opendj  99999153 Dec 20 08:17 db/userRoot/00000000.jdb
-rw-r--r--  1 opendj  opendj     31697 Dec 20 08:17 db/userRoot/00000001.jdb

See Administration Guide › Database Cache Settings for further information.

System Backups

Considerations:

  • Full system level backups can be used in place of full backups.
  • It is best to stop the DS/OpenDJ instance before doing a full system backup. This ensures the database and relevant files have been properly closed at the OS level.

Frequency:

  • It is recommended you create full system level backups at least once a week.

Backup Commands

DS/OpenDJ comes with a two commands that can be used individually to backup data from the DS/OpenDJ instance: backup and export-ldif. Both can be used for normal every day backups, but they do miss a few data points that need to be considered.

backup

The DS/OpenDJ backup command can be used:

  • For ad hoc or scheduled backups.
  • To backup individual or all backends.
  • To backup data in full or incrementally.

export-ldif

The export-ldif command allows administrators to extract a text based copy of the backend userRoot database in the form of an LDIF (Lightweight Data Interchange Format) file.

See Administration Guide › Backing Up Directory Data for further information.

Files Backed Up

backup

Files captured with the backup command can include jdb database files, schema files and task ldif files.

Note

backup --backUpAll will backup all the following each time this command option is used.

  • Database files: userRoot backend:
    -rw-r--r--   1 opendj  opendj  99999973 Jan 19 14:54 00000000.jdb
    -rw-r--r--   1 opendj  opendj  99999797 Jan 19 14:55 00000001.jdb
    -rw-r--r--   1 opendj  opendj  99999864 Jan 19 14:55 00000002.jdb
    -rw-r--r--   1 opendj  opendj  99997251 Jan 19 14:55 00000003.jdb
  • Schema files: schema backend:
    -rw-r--r--   1 opendj  opendj   43693 Jan 19 11:04 00-core.ldif
    -rw-r--r--   1 opendj  opendj    6596 Jan 19 11:04 01-pwpolicy.ldif
    -rw-r--r--   1 opendj  opendj  194661 Jan 19 11:04 02-config.ldif
    -rw-r--r--   1 opendj  opendj    5058 Jan 19 11:04 03-changelog.ldif
    -rw-r--r--   1 opendj  opendj    1227 Jan 19 11:04 03-pwpolicyextension.ldif
    -rw-r--r--   1 opendj  opendj    3561 Jan 19 11:04 03-rfc2713.ldif
    -rw-r--r--   1 opendj  opendj    2133 Jan 19 11:04 03-rfc2714.ldif
    -rw-r--r--   1 opendj  opendj    3235 Jan 19 11:04 03-rfc2739.ldif
    -rw-r--r--   1 opendj  opendj    3272 Jan 19 11:04 03-rfc2926.ldif
    -rw-r--r--   1 opendj  opendj    1710 Jan 19 11:04 03-rfc3112.ldif
    -rw-r--r--   1 opendj  opendj   12652 Jan 19 11:04 03-rfc3712.ldif
    -rw-r--r--   1 opendj  opendj   15858 Jan 19 11:04 03-uddiv3.ldif
    -rw-r--r--   1 opendj  opendj   12207 Jan 19 11:04 04-rfc2307bis.ldif
    -rw-r--r--   1 opendj  opendj    6339 Jan 19 11:04 05-rfc4876.ldif
    -rw-r--r--   1 opendj  opendj   12223 Jan 19 11:04 05-samba.ldif
    -rw-r--r--   1 opendj  opendj   14141 Jan 19 11:04 05-solaris.ldif
    -rw-r--r--   1 opendj  opendj    1125 Jan 19 11:04 06-compat.ldif
    -rw-r--r--   1 opendj  opendj  122837 Jan 19 12:09 99-user.ldif
  • Scheduled Tasks: tasks backend:
    -rw-------   1 opendj  opendj  572 Jan 19 11:05 tasks.ldif

export-ldif

The export-ldif command is used to export entry data from the DS/OpenDJ database backend into a text based file called LDIF. The export-ldif command extracts the binary data from the above *.jdb database files and saves it as an LDIF file.

Format of Files Backed Up

Each backup command saves the files backed up in various ways.

backup

All files with each backend backup are contained within a compressed .zip file. With the exception of a schema backend backup, all files within the backup itself use the original file names.

  • Database files: userRoot backend:
    backup-userRoot-20180204231456Z: Zip archive data, at least v2.0 to extract
    
    userRoot/$ unzip -l backup-userRoot-20180204231456Z
    Archive:  backup-userRoot-20180204231456Z
    OpenDJ backup 20180204231456Z of backend userRoot
      Length     Date    Time    Name
     --------    ----    ----    ----
      6849913  02-04-18  16:14   00000000.jdb
     --------                    -------
      6849913                    1 file
  • Schema files: schema backend:
    schema-backup-20180204231456Z: Zip archive data, at least v2.0 to extract
    
    schema/$ unzip -l schema-backup-20180204231456Z
    Archive:  schema-backup-20180204231456Z
    OpenDJ schema backup 20180204231456Z
      Length     Date    Time    Name
     --------    ----    ----    ----
            0  02-04-18  16:14   schema.comment
        43693  02-04-18  16:14   00-core.ldif.instance
         6596  02-04-18  16:14   01-pwpolicy.ldif.instance
       194661  02-04-18  16:14   02-config.ldif.instance
         5058  02-04-18  16:14   03-changelog.ldif.instance
         1227  02-04-18  16:14   03-pwpolicyextension.ldif.instance
         3561  02-04-18  16:14   03-rfc2713.ldif.instance
         2133  02-04-18  16:14   03-rfc2714.ldif.instance
         3235  02-04-18  16:14   03-rfc2739.ldif.instance
         3272  02-04-18  16:14   03-rfc2926.ldif.instance
         1710  02-04-18  16:14   03-rfc3112.ldif.instance
        12652  02-04-18  16:14   03-rfc3712.ldif.instance
        15858  02-04-18  16:14   03-uddiv3.ldif.instance
        12207  02-04-18  16:14   04-rfc2307bis.ldif.instance
         6339  02-04-18  16:14   05-rfc4876.ldif.instance
        12223  02-04-18  16:14   05-samba.ldif.instance
        14141  02-04-18  16:14   05-solaris.ldif.instance
         1125  02-04-18  16:14   06-compat.ldif.instance
     --------                    -------
       339691                    18 files
  • Scheduled Tasks: tasks backend:
    tasks-backup-20180204231456Z: Zip archive data, at least v2.0 to extract
    
    tasks/$ unzip -l tasks-backup-20180204231456Z 
    Archive: tasks-backup-20180204231456Z 
    OpenDJ tasks backup 20180204231456Z 
      Length     Date    Time    Name
     --------    ----    ----    ----
         1277  02-04-18  16:14   tasks.ldif
     --------                    -------
         1277                    1 file

ldif-export

As mentioned, export-ldif saves a copy of the database in a text based file in LDIF format:

bin/$ file userRoot-export.ldif
userRoot-export.ldif: ASCII English text

-rw-------  1 opendj  opendj  18053 Feb 12 16:21 userRoot-export.ldif

Example of an LDIF export (truncated):

dn: dc=example,dc=com
objectClass: domain
objectClass: top
dc: example
entryUUID: 5340ce76-ad75-3e8a-98a0-6e7674680f45

dn: ou=People,dc=example,dc=com
objectClass: organizationalunit
objectClass: top
ou: People
entryUUID: bbc3fc6f-7476-3428-9b3e-dd0b264720da

dn: uid=user.0,ou=People,dc=example,dc=com
objectClass: person
objectClass: inetorgperson
objectClass: organizationalperson
objectClass: top
postalAddress: Aaccf Amar$01251 Chestnut Street$Panama City, DE  50369
postalCode: 50369
uid: user.0
description: This is the description for Aaccf Amar.
userPassword: {SSHA}XOkWvX53G4YBu7KtLBi3pIIcQiJ2uzMeqh5XBg==
...
...
...

bin/$ file userRoot-export.ldif*
userRoot-export.ldif:      ASCII English text
userRoot-export.ldif.gzip: gzip compressed data, from FAT filesystem (MS-DOS, OS/2, NT)

Other Required Files

While the backup and export-ldif commands are good for backing up database backends, they miss crucial data points that are required to fully restore instances to their working state. It is recommended that you include each of the following (as needed) that are most relevant to your integration.

Critical files

  • config.ldif - contains the main configuration for the DS/OpenDJ instance, including replication data. This file is located in /path/to/ds/config.
  • admin-backend.ldif - contains the certificate for each replicated instance and the admin entry used for replication. The admin backend "cn=admin data" is created at startup using this file. This file is located in /db/adminRoot in DS 6.5 and later, /db/admin in DS 6 or /config in pre-DS 6.
  • keystore/truststores - contains client (truststore and keystore), replication (ads-truststore) and administration (admin-truststore and admin-keystore) key pairs. These files are located in /path/to/ds/config; as of DS 6 and later, ads-truststore is located in /path/to/ds/db/ads-truststore for new installs.
  • java.properties - contains the java properties used by the various command lines when executed. This file is located in /path/to/ds/config.

Non-critical files

  • http-config.json - contains the REST 2 LDAP configuration. This file is located in /path/to/ds/config.
  • wordlist.txt - contains all the "words" that all password policies use to match against when adding or changing passwords. This file is located in /path/to/ds/config.
  • Init scripts - all init.d scripts used to start and stop DS/OpenDJ instances when the system boots or shuts down.

Restoration Strategies

Replicated Servers

Special considerations must be taken when restoring replicated servers.

See How do I restore old backup data to a DS/OpenDJ (All versions) replication topology? and Administration Guide › To Restore a Replica for further information on the process of restoring replicated servers.

Restore Considerations for Replicated Servers: (*)

For all types of restores for a Replicated server, the following considerations must be adhered to:

  • Use a backup that is not older than the replication-purge-delay.
  • If the available backups are older than the purge delay, initialization from an up-to-date Master is a better option.
  • Restores using backups that are older than the purge delay will succeed but there will be a gap in the replicated changes as discussed in the documentation link above and the server will need to be initialized.

Full Restores

Considerations:

  • With the exception of the Purge Delay restriction above (*), no special considerations are needed.
  • Offline (server stopped) restores will complete faster than online (server started) restores.

Incremental Restores

Incremental Restores must be completed in the following way:

Example: If a full backup was taken on Sunday and incremental backups were taken daily, Monday through Saturday, the administrator would have 7 backups to restore the instance up to the latest restore point. This should be done as follows:

  1. Restore the latest full backup.
  2. Restore each incremental backup starting with Monday and ending with Saturday's backup.

Considerations:

  • With the exception of the Purge Delay restriction above (*), no special considerations are needed.
  • Offline (server stopped) restores will complete faster than online (server started) restores.

System Restores

Considerations:

  • With the exception of the Purge Delay restriction above (*), no special considerations are needed.

Bare Metal or Ground Zero Restores

When restoring to a newly installed instance, the instance must have been configured with the ./setup command before the restore takes place. Failing to first configure the new unzipped instance and issuing the restore command will result in the following error and the restore will fail:

[root@bin]# ./restore --backupDirectory /opt/opendj/bak/userRoot/ --backupID BackupTask-97308e2e-30b2-487f-8ee5-0eae8e411128-20150129020000000
ERROR
An error occurred while trying to load the Directory Server schema: An error
occurred at or near line 1700 while trying to parse the configuration from
LDIF file /opt/opendj/config/config.ldif:
org.opends.server.util.LDIFException: Entry cn=Character Set,cn=Password
Validators,cn=config read from LDIF starting at line 1700 includes a duplicate
attribute ds-cfg-character-set with value 1:ABCDEFGHIJKLMNOPQRSTUVWXYZ. The
second occurrence of that attribute value has been skipped

The following process must be adhered to if you need to build a new server or restore a server from scratch:

  1. Install and configure the instance to a basic level, that is, using the setup command.
  2. Execute the restore command with the appropriate --backupID.

Considerations:

  • With the exception of the Purge Delay restriction above (*), no special considerations are needed.

Point In Time Restores

If you need to restore all servers to a certain point in time you must do so as described in How do I roll back an entire network of DS/OpenDJ (All versions) replicas to a previous backup?

Considerations:

  • With the exception of the Purge Delay restriction above (*), no special considerations are needed.

See Also

How do I configure DS/OpenDJ (All versions) to ensure accidentally deleted or changed data can be restored when replication is enabled?

Generation IDs do not match error after restoring a DS/OpenDJ (All versions) replica

FAQ: Backup and restore in DS/OpenDJ

Installing and Administering DS/OpenDJ

Administration Guide › Backing Up and Restoring Data

Related Training

ForgeRock Directory Services Core Concepts

Related Issue Tracker IDs

N/A


How do I roll back an entire network of DS/OpenDJ (All versions) replicas to a previous backup?

The purpose of this article is to provide guidance on rolling back an entire network of DS/OpenDJ replication servers to a previous backup. This approach can be used if you want to be rid of undesirable changes that have occurred and are now being replicated across all servers.

Overview

Accidental deletions of data in DS/OpenDJ can be reverted in two ways:

  • The first way, described in How do I configure DS/OpenDJ (All versions) to ensure accidentally deleted or changed data can be restored when replication is enabled?, configures the replication changelog to record additional information about each change. This allows changes to be reverted at a very fine-grained level and with very little impact to the servers in the replication topology. However, reverting each change requires several manual steps.
  • The second way, described in this article, uses the backup and restore tools. This is comparatively coarse as you can only restore up until a given backup and it does require that every replicating server is reinitialized.

Rolling back an entire network of DS/OpenDJ replicas

To roll back an entire network of DS/OpenDJ replicas to a previous backup, you must restore the same backup to every replica and use pre-external-initialization and post-external-initialization as follows:

  1. Enter the following command on one of the servers to prepare the domain on all servers for being externally initialized: You must specify the baseDN of the data you are going to be changing, for example, dc=example,dc=com.
    $ ./dsreplication pre-external-initialization --hostname ds1.example.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt
    
  2. Enter the following command to restore the backup to each server (this command performs an online restore, so you do not need to stop the server first):
    $ ds1/bin/restore --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backupID [backupid] --backupDirectory /path/to/ds/bak
    $ ds2/bin/restore --hostname ds2.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backupID [backupid] --backupDirectory /path/to/ds/bak
    [...]
  3. Enter the following command on one of the servers to set the new generation ID for the entire domain. Ensure to use the same baseDN as the first step:
    $ ./dsreplication post-external-initialization --hostname ds1.example.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt
    

The above steps alter the generation ID of the replicated domain. "Old" changes will not get replayed because they were targeting the data using the previous generation ID. The final step calculates a new generation ID for the domain and broadcasts it to all the servers, which allows them to replicate again.

Replication will now proceed as normal, but from the restored point in time.

See Also

How do I configure DS/OpenDJ (All versions) to ensure accidentally deleted or changed data can be restored when replication is enabled?

How do I restore old backup data to a DS/OpenDJ (All versions) replication topology?

How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

FAQ: Backup and restore in DS/OpenDJ

FAQ: General DS/OpenDJ

Administration Guide › Backing Up and Restoring Data › Backing Up Directory Data

Administration Guide › Backing Up and Restoring Data › Restoring Directory Data From Backup

Reference › Tools Reference › dsreplication

Reference › Tools Reference › restore

Related Training

ForgeRock Directory Services Core Concepts

Related Issue Tracker IDs

N/A


How do I restore DS/OpenDJ from another DS/OpenDJ instance (All versions)?

The purpose of this article is to provide information on rebuilding DS/OpenDJ using data from another DS/OpenDJ server. It is always preferable to restore DS/OpenDJ from a backup, but you can use this approach if the server you want to restore does not have a current backup and the restore happens within the changelog purge delay window (default 3 days).

Rebuilding DS/OpenDJ

You can rebuild a DS/OpenDJ server (server1) using data from another DS/OpenDJ server (server2) as follows:

  1. Create a backup of the good server (server2) using the backup command, for example:
    $ ./backup --hostname ds2.example.com --port 4444  --bindDN "cn=Directory Manager" --bindPassword password --backUpAll --backupDirectory /path/to/ds/bak --start 0
    
  2. Copy the backup file(s) to the server you want to rebuild (server1).
  3. Stop the server you are rebuilding (server1).
  4. Restore the data to the server using the restore command. Each backend needs to be restored individually using a separate restore command, where you specify the backend to restore as part of the path. The restore command is different depending on your version; these examples show the userRoot backend being restored:
    • DS 5 and later (needs the --offline option when the server is stopped):
      $ ./restore --offline --backupID 20161019102414Z --backupDirectory /path/to/ds/bak/userRoot
    • OpenDJ 3.5.x and earlier:
      $ ./restore --backupID 20161019102414Z --backupDirectory /path/to/ds/bak/userRoot
  5. Start the server.

See Also

FAQ: Backup and restore in DS/OpenDJ

How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

How do I restore old backup data to a DS/OpenDJ (All versions) replication topology?

How do I roll back an entire network of DS/OpenDJ (All versions) replicas to a previous backup?

How do I control how long replication changes are retained in DS/OpenDJ (All versions)?

Administration Guide › Backing Up and Restoring Data › Backing Up Directory Data

Administration Guide › Backing Up and Restoring Data › Restoring Directory Data From Backup

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I restore old backup data to a DS/OpenDJ (All versions) replication topology?

The purpose of this article is to provide information on restoring old backup data to a DS/OpenDJ replication topology and using this old data to initialize replication. In this way, you can restore good data from an old backup and ensure it replicates across all servers. This article assumes replication has already been enabled.

Restoring old backup data and initializing replication

Note

If you are restoring a binary backup, you must ensure that the backup file exists on the local server on which you want to run the restore command. You cannot restore a backup file from a remote instance.

  • A binary backup - created using the DS/OpenDJ backup --backUpAll command.
  • In LDIF format - created using the DS/OpenDJ export-ldif command.

You can restore old backup data to a DS/OpenDJ replication topology and initialize replication as follows:

  1. Enter the following command on one of the servers (server1 for purposes of example) to prepare the domain on all servers for being externally initialized: You must specify the baseDN of the data you are going to be changing, for example, dc=example,dc=com.
    $ ./dsreplication pre-external-initialization --hostname ds1.example.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --no-prompt
  2. Restore the backup data to server1 as follows, depending on what type it is:
    • Binary backup - run the restore command on server1, for example:
      $ ./restore --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backupID 20161019100952Z --backupDirectory /path/to/ds/binaryBackup_bak
    • LDIF format - use the import-ldif command, for example:
      $ ./import-ldif --hostname ds1.example.com --port 4444 --baseDN dc=example,dc=com --backendID userRoot --ldifFile /path/to/backupfile.ldif
      
  3. Back up server1 (which now includes your restored backup data) using the backup command, for example:
    $ ./backup --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backendID userRoot --backupDirectory /path/to/ds/server1_bak --start 0
    
  4. Copy the backup file you created in the previous step and the accompanying backup.info file to each server you want to restore (server2 and server3 in this example).
  5. Restore this backup to all the other servers by running the restore command locally on each server in the topology, for example:
    $ server2/bin/restore --hostname ds2.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backupID 20161019102414Z --backupDirectory /path/to/ds/server1_bak
    $ server3/bin/restore --hostname ds3.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backupID 20161019102414Z --backupDirectory /path/to/ds/server1_bak
    [...]
  6. Enter the following command on server1 to set the new generation ID for the entire domain. Ensure you use the same baseDN as in step 1:
    $ ./dsreplication post-external-initialization --hostname ds1.example.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --no-prompt
Note

Instead of backing up and restoring (steps 3 and 5), you could use the dsreplication initialize-all command to initialize all servers via the network. This command Initializes the contents of the data under the specified base DN on all the servers with the contents on the specified server. See Reference › dsreplication for further details.

The above steps alter the generation ID of the replicated domain. "Old" changes will not get replayed because they were targeting the data using the previous generation ID. The final step calculates a new generation ID for the domain and broadcasts it to all the servers, which allows them to replicate again.

Replication will now proceed as normal, but from the restored point in time.

See Also

Generation IDs do not match error after restoring a DS/OpenDJ (All versions) replica

How do I roll back an entire network of DS/OpenDJ (All versions) replicas to a previous backup?

How do I configure DS/OpenDJ (All versions) to ensure accidentally deleted or changed data can be restored when replication is enabled?

How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

FAQ: Backup and restore in DS/OpenDJ

How do I quickly create a new DS/OpenDJ (All versions) replica?

Administration Guide › Backing Up and Restoring Data

Administration Guide › Managing Data Replication › Initializing Replicas

Related Training

ForgeRock Directory Services Core Concepts

Related Issue Tracker IDs

N/A


Frequently Asked Questions


FAQ: Installing and configuring DS/OpenDJ

The purpose of this FAQ is to provide answers to commonly asked questions regarding installing and configuring DS/OpenDJ.

Frequently asked questions

Q. Is DS/OpenDJ compatible with Oracle Java Development Kit (JDK) 9 or later?

A. Yes, Oracle JDK 9 is supported in DS 6 and JDK 11 is supported as of DS 6.5. You should only use supported versions to prevent compatibility issues.

Q. Where can I download the latest Enterprise or maintenance release of DS/OpenDJ?

A. The current supported releases of DS/OpenDJ are available for download from BackStage.

Q. Where can I find the DS/OpenDJ installation requirements?

A. The Release Notes provide the Hardware and Software requirements for installing DS/OpenDJ.

See the release notes for the latest version of DS: Release Notes › Before You Install.

Q. Can I install DS/OpenDJ so that the instance files are separate to the install files?

A. Yes you can. This setup allows you to separate all your backend database files and configuration in a separate file system to your binaries.

See How do I install DS/OpenDJ (All versions) so that the instance files are separate to the install files? for further details.

Q. How do I integrate IDM, AM and DS?

A. You should refer to ForgeRock Identity Platform Version 6: Integrating IDM, AM, and DS, which collects the information you need to set up integration in one place. For additional features, refer to the following chapters from IDM documentation: Samples Guide › Integrating IDM With the ForgeRock Identity Platform and the Integrator's Guide › Setting Up User-Managed Access (UMA), Trusted Devices, and Privacy.

Q. Is it possible to clone a copy of an existing DS/OpenDJ between servers?

A. You can move DS/OpenDJ data between servers and operating systems. Most of the configuration is portable.

The following two aspects of the configuration are not portable:

  • Server certificates - they contain the host name of the system.
  • Replication configuration - this includes the host name and administrative port numbers.

The procedure and considerations are discussed in detail Moving Servers and Changing Server Certificates.

Q. What application servers are compatible for installing DS/OpenDJ DSML gateway or DS/OpenDJ REST LDAP gateway?

A. If you are planning to install the DS/OpenDJ DSML gateway or the DS/OpenDJ REST LDAP gateway, you need to have the appropriate compatible Application Servers installed.

DS/OpenDJ REST to LDAP gateway, and DS/OpenDJ DSML gateway run on Apache Tomcat™ and Jetty®. ForgeRock supports only stable container releases. See the Tomcat and Jetty documentation for details on which container version supports the Java version used in your deployment.

Note

DS/OpenDJ directory server runs as a standalone JVM service and does not depend on an application server.

Q. Where can I find available options to install or remove DS/OpenDJ?

A. You can install or remove the DS/OpenDJ software using the Install Wizard or through the command line. Detailed steps can be found in the Installation Guide:

Q. Where can I find an example DS/OpenDJ installation script?

A. The Installation Guide › Installing a Directory Server provides examples of the available install options including scripted (Silent) Install.

Q. Can I create a new base DN using the command line?

A. Yes you can. It is a two stage process because creating the backend using the command line only creates the backend database and adds its configuration to the instance. It does not create the base entry for you. The backend is only created with the base entry if you do it via the control panel; the control panel is removed in DS 6.

To create a base DN:

  1. Create the backend configuration and database using the dsconfig command. See Administration Guide › Creating a New Database Backend for further information.
  2. Add the base DN entry to the backend using the ldapmodify command or import the base DN entry into the backend using the import-ldif command. See Administration Guide › Importing and Exporting Data for further information.

Q. Is there a configuration reference for DS/OpenDJ?

A. Yes, the Configuration Reference is provided, which details the configuration components available in DS/OpenDJ.

Q. How can I configure my local system to make administering DS/OpenDJ server easier?

A. You can eliminate the need to provide defined arguments when using command line utilities by creating a personal tools.properties file to store commonly entered parameters such as bind DN, host name, and port number.

See Developer's Guide › Configuring Default Settings for a tools.properties example.

After saving the changes to your new tools.properties file, perform a query of your suffix as follows:

$ ./ldapsearch "(objectclass=*)" 

Which gives an output similar to this (where the example suffix is dc=example,dc=com):

dn: dc=example,dc=com 
dc: example 
objectclass: domain 
objectclass: top
Note

The tools.properties file found in the user’s .opendj directory takes the highest precedence for the executing user. If you need different parameters for different DS/OpenDJ instances installed on the same system, create a tools.properties file in the config directory for each individual instance rather than in the user’s directory. 

See Also

FAQ: Passwords in DS/OpenDJ

FAQ: DS/OpenDJ compatibility with third-party products

FAQ: General DS/OpenDJ

FAQ: Upgrading DS/OpenDJ

Installing and Administering DS/OpenDJ

Deployment Guide

Installation Guide

Administration Guide

Related Training

ForgeRock Directory Services Core Concepts 


FAQ: Backup and restore in DS/OpenDJ

The purpose of this FAQ is to provide answers to commonly asked questions regarding backing up and restoring DS/OpenDJ.

Frequently asked questions

Q. Do I have to do a full backup each time?

A. You should ideally use a mixture of full and incremental backups to ensure you have a comprehensive backup strategy in place.

See How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)? for further information on designing your backup and restore strategy.

Q. Can I use the backup and restore commands to import directory data into a new server instance?

A. You can providing the DS/OpenDJ major versions are the same (for example, 3.0 to 3.5). However, this won't work if you want to import directory data from OpenDJ 2.6.x to OpenDJ 3.x.

OpenDJ 3 uses a pluggable backend/abstraction layer, which supports JE and PDB backends. The JE backend used in OpenDJ 3 is different to the old 'local-db' JE format, so binary backups are not compatible between OpenDJ 2.6.x and later releases. See OpenDJ 3 Release Notes › New Features for further information on these changes. The PDB backend type is deprecated in DS 5 and removed in DS 6.

If you want to import directory data into OpenDJ 3, you should either use the export-ldif / import-ldif commands or the upgrade mechanism, which can migrate the OpenDJ backend configuration and directory data. See OpenDJ Installation Guide › Upgrading to OpenDJ 3 for further information on upgrading.

Q. What backup ID is used if incrementalBaseID is blank?

A. If you leave the incrementalBaseID option blank when performing a backup, the ID of the last backup is automatically used. If the backup directory is blank, a full backup will be taken; otherwise an incremental backup will be taken based on the last backup performed (regardless of whether it was an incremental or full backup).

Q. My incremental backup is bigger than expected, why is this?

A. The resolution of incremental backups is per JDB file. If your backend is made up of 10MB JDB files, then even with just a single operation between backups, the resulting incremental backup could be up to 10MB. If you have 100MB sized JDB files this could be up to 100MB. This is because the entire 'latest' JDB file would need to be copied for the incremental backup.

Q. Does the backup command back up everything I need to fully restore DS/OpenDJ?

A. No, there are a number of files that you should also manually back up, as detailed in the Other Required Files section in How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

Q. How do I check the data integrity of a backup?

A. You can validate the data integrity of a backup as follows:

  1. Restore the backup to a temporary isolated server that is not replicating with the existing topology.
  2. Run a command such as rebuild-index -all, export-ldif or verify-index to force the list of backup *.jdb files to be read. The command will fail if any of the files contain corrupt data.

Q. How should I restore the data if I have done a backup --backupAll?

A. You should restore the userRoot backup using the restore command, and then restore the schema and task backups by copying the relevant files across:

  • For schema, you need the schema directory (located in /path/to/ds/config; as of DS 6 and later, located in /path/to/ds/db for new installs).
  • For tasks, you just need the tasks.ldif file (located in /path/to/ds/db/tasks in DS 6 and later, or /path/to/ds/config in pre-DS 6).

You can either use the restore command to copy these files across or you can copy them yourself. If you copy them using the restore command, the backend is automatically taken offline during the copy; if you copy them yourself, you will need to ensure the server is stopped or the schema backend is offline. You can use a restore command such as the following to copy a directory or file across:

$ ./restore --backupDirectory /path/to/bak/schema/ --backupID 20160614031032Z

See How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)? for further information on the files that are backed up.

Q. If I add new attributes / objectClass and then create a backup, will my backup contain the definition of these entities?

A. When you add new attributes or an objectClass, the changes are written to the cn=schema backend; this is stored in the 99-user.ldif file (located in /path/to/ds/config/schema; as of DS 6 and later, located in /path/to/ds/db/schema for new installs). If you back up this directory and restore it to another server, your schema changes will be preserved.

Schema is also copied across from one server to another when you run the dsreplication configure (DS 5 and later) or dsreplication enable (pre-DS 5) command.

Q. What impact does a backup have on a running server?

A. There is little impact to LDAP clients. Users can still search and write data while the backup is taking place, and replication continues to work normally. The administrator should see an increased number of reads from the source disk (that is, holding the backend being backed up) as the backup process essentially reads the entire database. The administrator should also see an increased number of writes on the target disk as the backup is written. It is best practice for the source disk and the target disk to be different. There may be some additional CPU utilization if the backup is being compressed.

Q. Can I remove old backup files to free up space?

A. Yes, you can delete or move old backup files to a different location to free up space as required. You must then edit the backup.info file (located in the /path/to/ds/bak directory) to remove the reference to the file(s) you deleted or moved to ensure this file stays in sync with your actual backups.

There is a RFE for managing backups and keeping the backup.info file in sync: OPENDJ-1310 (Manage backup retention).

Q. What key is used if I encrypt the backup?

A. DS/OpenDJ uses a 128-bit secret key stored in the admin-backend, for example, <ds-cfg-key-id=d2d8c278-9fcb-434b-a8cc-0ec237e54644,cn=secret keys,cn=admin data> and encrypts using AES/CBC/PKCS5Padding by default. The key length and cipher are advanced properties of the Crypto Manager; you can use dsconfig --advanced to change them.

Additionally, the keys are shared between replicas, which allows you to restore an encrypted backup to any server.

Q. What happens if the backend being restored is also encrypted?

A. If the backend being restored is also encrypted and you still have operational servers in your replication topology, then you need to first enable replication before restoring the encrypted database. Enabling replication (without initializing) allows the new instance to get all the necessary key material to decrypt the backend. This is described in Administration Guide › To Restore a Replica.

In a disaster recovery type scenario where you do not have access to the servers with the key material, you should follow this process instead (which is noted in the above procedure):

  1. Set up a new server using ./setup. This should just be a basic setup that includes a backend with the correct suffix, but no data.
  2. Stop the new server.
  3. Overwrite the following files from your file-system backup of the original server: 
    • admin-backend.ldif (located in /db/adminRoot in DS 6.5 and later, /db/admin in DS 6 or /config in pre-DS 6)
    • ads-truststore (located in /path/to/ds/config; as of DS 6 and later, located in /path/to/ds/db/ads-truststore for new installs)
    • ads-truststore.pin (located in /path/to/ds/config; as of DS 6 and later, located in /path/to/ds/db/ads-truststore for new installs)
  4. Restore your encrypted backend data.
  5. Start the new server.

Q. Can I just back up one node if I have replication enabled?

A. This is subjective and can depend on the size of your database. However, you can just back up one node and use that to restore another node, but if you have a large database, transfer times may make this unviable.

It is also worth considering storing your backups offsite incase all nodes need recovering, for example, in a disaster recovery situation.

Q. What changes are applied when I restore a node?

A. When you restore a node, changes are applied according to the replication change log.

For example, if you stopped server 2 and restored it from a previous backup, added a new user on server 1 and restarted server 2, the new user you added on server 1 would be replicated to server 2.

Q. How can I recover accidentally deleted or changed data when replication is enabled?

A. Accidental deletions of data in DS/OpenDJ can be reverted in two ways:

Q. Can I rebuild DS/OpenDJ using data from another server if I don't have a current backup?

A. Yes you can, although it is always preferable to have a valid backup to restore. See How do I restore DS/OpenDJ from another DS/OpenDJ instance (All versions)? for further information.

Q. How do I change the replication purge delay setting?

A.  You can change the replication purge delay setting (replication-purge-delay) to control how long replication changes are retained in the changelogDb as described in How do I control how long replication changes are retained in DS/OpenDJ (All versions)?

Q. What should I change the replication purge delay setting to?

A. This depends on your system and backup schedule. For example, if you back up daily, you could reduce your purge delay to 1.5 or 2 days as you would be able to recover any changes from your backup.

Q. When does the replication purge take place?

A. The replication purge is continually happening in the background, removing individual entries / modifications as they become stale (based on the purge delay setting). For example, if you set your purge delay setting to 1 day, individual changes would be purged from the changelogDb as follows:

  • Change 1: added May 1, 1am - purged May 2, 1am.
  • Change 2: added May 1, 10am - purged May 2, 10am.
  • Change 3: added May 1, 1pm - purged May 2, 1pm.
  • Change 4: added May 1, 2pm - purged May 2, 2pm.
  • Change 5: added May 1, 3pm - purged May 2, 3pm.

Q. Can I recover a corrupted config.ldif file?

A. Yes, DS/OpenDJ makes copies of the config files as follows, which should enable you to recover this file if needed:

  • Each time you start the server and it starts without errors, DS/OpenDJ copies the config.ldif file to /path/to/ds/var/archived-configs in DS 6 and later, or /path/to/ds/config/archived-configs in pre-DS 6 and adds a timestamp to the filename. If you have changed something that prevents the server starting, you can copy the last known good file from this directory or use the files as a reference as to what has changed.
  • Each time you start the server and it starts without errors, DS/OpenDJ copies the config.ldif file to config.ldif.startok. This means that the last time the server started it was using the config.ldif.startok file. If the current config.ldif fails to start the server for some reason, you can manually copy back the config.ldif.startok to config.ldif and restart the server.
  • Each time you use the backup command, the server also saves the config files in the /archived-configs directory to a location within the backup archive itself. You can use these to manually copy (revert) back to a known good configuration.

Restoring an old config file

You can restore an old config file from /archived-configs as follows:

  1. Stop the DS/OpenDJ server:
    $ ./stop-ds
  2. Copy across the required archived config file:
    $ cp archived-configs/XXXXXXconfigfile ./config.ldif
  3. Restart the DS/OpenDJ server:
    $ ./start-ds

See Also

How do I relocate the backend database files in DS/OpenDJ (All versions) on to a separate file system?

Installing and Administering DS/OpenDJ

Administration Guide › Backing Up and Restoring Data

Administration Guide › Backing Up Directory Data

Administration Guide › Restoring Directory Data From Backup

Related Training

ForgeRock Directory Services Core Concepts


FAQ: Monitoring DS/OpenDJ

The purpose of this FAQ is to provide answers to commonly asked questions regarding monitoring in DS/OpenDJ.

Frequently asked questions

Q. How can I check that DS/OpenDJ is up and running?

A. You can perform a heartbeat check against DS/OpenDJ to verify that the server is up and running.

See How do I perform a heartbeat check against DS/OpenDJ (All versions)?for further information.

Q. How can I check if a backend is online?

A. You can check if a backend is online by performing a ldapsearch against the specific backend.

See How do I check if a backend is online in DS/OpenDJ (All versions)? for further information.

Q. How can I monitor replication?

A. You can use the dsreplication status command to give you an overall view of the replication topology, including whether the servers are synchronized. For example:

$ ./dsreplication status --adminUID admin --adminPassword password --hostname ds1.example.com --port 4444 --trustAll

See How do I troubleshoot replication issues in DS/OpenDJ (All versions)? and Reference › dsreplication for further information.

Note

The M.C.and A.O.M.C. metrics returned from the dsreplication status command are deprecated in DS 6 and replaced with replication delay in DS 6.5. You should monitor replication delay instead; you can also monitor this over LDAP and HTTP as detailed in Administration Guide › Monitoring Replication Delay Over LDAP and Administration Guide › Monitoring Replication Delay Over HTTP.

Q. How can I monitor performance?

A. You can monitor performance by performing a ldapsearch against cn=monitor to return operation statistics.

See How do I use cn=monitor entry in DS/OpenDJ (All versions) for monitoring? for further information on monitoring operation statistics.

Q. What can I monitor with the cn=monitor entry?

A. DS/OpenDJ exposes a lot of different monitoring information over LDAP under this entry.

See How do I use cn=monitor entry in DS/OpenDJ (All versions) for monitoring? for further information, including specific examples you may want to monitor.

Q. Do the statistics under the cn=monitor entry persist when the server is restarted?

A. No, the statistics under cn=monitor are reset each time the server is restarted.

Q. Do I have to use the Directory Manager user for JMX monitoring?

A. No, you can use any user for JMX monitoring; you just need to add the JMX privileges (jmx-notify, jmx-read, and jmx-write) to the user you want to access JMX monitoring. No users have access to JMX monitoring by default.

See Administration Guide › JMX-Based Monitoring for an example of how to add these privileges to a user (Directory Manager, but you can substitute any user you want).

Q. How can I connect to JMX to ensure mbeans are returned?

A. You must be authenticated to the DS/OpenDJ server via a JMX client to see the mbeans with associated attributes and operations. Authenticating to the server is the only way to expose the sensitive elements within the mbeans; connecting to the process directly will not show them. Additionally, you must ensure the user who authenticates has JMX privileges.

See JMX Client Access and JMX-Based Monitoring for more information.

Q. Can I change the default listen-address for the JMX Connection Handler?

A. Yes, as of OpenDJ 2.6.2 you can change the default listen-address. The default listen-address for the JMX Connection Handler is 0.0.0.0.

Q. What URL should I use in JConsole to log in?

A. You must always use a remote URL, even if you are using a local connection to the JVM. For example:

service:jmx:rmi:///jndi/rmi://localhost:1689/org.opends.server.protocols.jmx.client-unknown

You can substitute a hostname alias (such as ds1.example.com) or an IP address for the localhost part of this URL. 

Note

If SSL is enabled, you must set up the truststore on the system where JConsole is running and configure SSL properly to monitor a remote application. See Using JConsole for further information.

See Also

FAQ: DS/OpenDJ performance and tuning

How do I generate sample user data for performance testing in DS/OpenDJ (All versions)?

Unindexed searches causing slow searches and poor performance on DS/OpenDJ (All versions) server

Administration Guide › Monitoring, Logging, and Alerts

Related Training

ForgeRock Directory Services Core Concepts


FAQ: DS/OpenDJ compatibility with third-party products

The purpose of this FAQ is to provide answers to commonly asked questions regarding DS/OpenDJ compatibility with third-party products.

Frequently asked questions

Q. Can the DS/OpenDJ database be stored on a virtual VMware® server?

A. Yes, you can use virtual VMware servers for the database and DS/OpenDJ will work with this in the same way as if the database was stored on a local disk.

Q. Can I use the Amazon Web Services™ (AWS™) snapshot feature to quickly provision new DS/OpenDJ instances?

A. Yes, you can. See How do I use the AWS snapshot feature to quickly create DS/OpenDJ (All versions) instances? for further information.

Q. Which files should be excluded from antivirus scans in DS/OpenDJ?

A. Files in the following directories should be whitelisted:

  • /path/to/ds/db/
  • /path/to/ds/changelogDb/ if replication is enabled.

Additionally, you should whitelist the following files depending on which backend you are using:

  • Java Edition backends (the most important ones are the *.jdb files):
    opendj; userRoot/$ p
    /opt/instances/m1/db/userRoot
    opendj; userRoot/$ lls
    total 8520
    -rw-r--r--  1 opendj  opendj  4150161 Sep 28 09:48 00000000.jdb
    -rw-r--r--  1 opendj  opendj     7117 Sep 28 09:42 je.config.csv
    -rw-r--r--  1 opendj  opendj   120646 Oct  3 09:58 je.info.0
    -rw-r--r--  1 opendj  opendj        0 Sep 28 09:42 je.lck
    -rw-r--r--  1 opendj  opendj    75104 Oct  3 12:12 je.stat.csv
    
  • PDB backends (the most important ones are the dj_journal.* files):
    opendj; userRoot/$ pwd
    /opt/instances/pdb/db/userRoot
    opendj; userRoot/$ lls
    total 133984
    -rw-r--r--  1 opendj  opendj  67108864 Oct  5 11:58 dj
    -rw-r--r--  1 opendj  opendj         0 Oct  5 11:58 dj.lck
    -rw-r--r--  1 opendj  opendj       399 Oct  5 11:58 dj.log
    -rw-r--r--  1 opendj  opendj   1254686 Oct  5 11:57 dj_journal.000000000001
    -rw-r--r--  1 opendj  opendj    228735 Oct  5 11:58 dj_journal.000000000002

Q. Does DS/OpenDJ support PostgreSQL®?

A. No, DS/OpenDJ uses its own BerkeleyDB™ JE backend, which cannot currently be substituted for anything else.

Q. Is DS/OpenDJ FIPS 140-2 compliant?

A. ForgeRock conforms to FIPS 140-2 for encryption and hashing ciphers for both inflight messaging, protocol tunnels and credential storage. In addition, the ForgeRock solution has been certified in deployments against rigorous DoD STIG requirements and is in process at FedRAMP certification. All of these certifications require proper FIPS 140-2 compliance.  The solution allows deployments to choose ciphers; the most common ones used in deployments tend to be SHA-256 and AES-256, however we support stronger key lengths on both of these. A common trend for mobility is support for elliptical curve cryptography due to it's high security yet low process power nature and of course is also recommended by NIST as part of FIPS 140-2.

Regarding FIPS 140-2 certification, ForgeRock does not write its own crypto modules, but our software can utilize underlying crypto modules (usually those associated with Java®) which have options to be executed in FIPS-compliant mode.FIPS-compliant mode.

Q. Why can't I use the ControlPanel.app in the Mac® OS X® operating system?

A. There are two known issues with the ControlPanel.app:

In both cases, you can start the DS/OpenDJ Control Panel from the command line.

See Also

FAQ: Installing and configuring DS/OpenDJ

FAQ: Upgrading DS/OpenDJ

FAQ: Passwords in DS/OpenDJ

Installing and Administering DS/OpenDJ

Related Training

ForgeRock Directory Services Core Concepts


FAQ: General DS/OpenDJ

The purpose of this FAQ is to provide answers to commonly asked general questions regarding DS/OpenDJ.

Frequently asked questions

Q. What are the requirements to decommission or retire a DS/OpenDJ instance?

A. If you need to retire the system on which the DS/OpenDJ instance is running and this DS/OpenDJ instance is part of a replication topology, you must first permanently disable replication on this instance only, before decommissioning the system. 

See Administration Guide › To Stop Replication Permanently For a Replica for further information on stopping replication.

Caution

Failing to permanently disable replication before retiring the system will result in replication connection failures when using the dsreplication status command.

Example dsreplication status command:

$ ./dsreplication status --adminUID admin --adminPassword password --hostname ds1.example.com --port 4444 --trustAll

Resulting error if replication was not disabled before retiring the system:

The displayed information might not be complete because the following errors were encountered reading the configuration of the existing servers:
Error on ds1.example.com:5444: An error occurred connecting to the server.  Details: javax.naming.CommunicationException: ds1.example.com:5444 [Root exception is java.net.ConnectException: Connection refused]

If you have already retired the system and are seeing error messages such as the above, please contact ForgeRock Support.

Q. Is there a limit to the number of users and groups that can be created in DS/OpenDJ?

A. The theoretical limits for maximum number of entries are so high that you will almost certainly hit hardware, performance or infrastructure problems before you can reach them. The possible number of users and groups you can support depends on what hardware resources you have available, how well your data is structured and how well crafted the operations your clients make on the database.

We regularly do testing with hundreds of millions of entries.

Q. Can a group be both dynamic and static?

A. No, a group cannot be both for implementation reasons. See Developer's Guide › Creating Static Groups and Developer's Guide › Creating Dynamic Groups for further information on these group types.

Q. Can I use static groups for large numbers of users?

A. It is recommended that you use dynamic groups for large numbers of users as large static groups can cause performance problems. An alternative to using dynamic groups is to invert the logic and have a small static group that only contains the exceptions rather than all users.

See Best practice for managing groups in DS/OpenDJ (All versions) and Developer's Guide › Working With Groups of Entries for further information.

Q. How can I query DS/OpenDJ to find the number of user entries?

A. You can query your LDAP user store to find the number of users. The following examples demonstrate two approaches, where the countentries option is used to return the total number of entries that match the search filter and displays the total number on the last line.

Using a user based objectclass

You can use a query such as the following if you are using the default schema objectClass of inetOrgPerson in DS/OpenDJ:

$ ./ldapsearch --port 1389 --bindDN "cn=Directory Manager" --bindPassword password --baseDN dc=example,dc=com --countentries objectClass=inetOrgPerson

This approach will not work if you use the same objectClass for real people as well as non real person objects.

Using uids

You can use a query such as the following to return all users and all attributes:

$ ./ldapsearch --hostname ds1.example.com --port 1389 --bindDN “cn=Directory Manager” --bindPassword password --baseDN dc=example,dc=com --searchScope sub --countEntries “(uid=*)” \*

This approach will not work if you use a non-uid based RDN.

Q. Are there any restrictions on the length and characters that can be included in the userid attribute?

A. The userid attribute must be at least one character, although there is no maximum length.

Note

Our connection handlers do impose an upper limit on the size of every request received to avoid DoS style attacks. The default is 5MB, so that is indirectly an upper limit on the length of userid.

All unicode characters are permitted for inclusion in the userid attribute.

Q. Does DS/OpenDJ support attribute encryption?

A. DS/OpenDJ supports one-way encryption of password values (for example, Salted SHA-1 hashing), but it does not support reversibly encrypting attributes.

Q. Can I have Pass Through Authentication (PTA) use a mapped attribute for use with Active Directory®?

A. Yes, as of OpenDJ 3.5, you can map LDAP attributes in DS/OpenDJ for PTA using the mapped-search-filter-template property. For example, you can map different attributes, such as uid in DS/OpenDJ to sAMAccountName in Active Directory. Prior to this, it was not possible to map different attributes in OpenDJ for PTA.

See How do I configure mapping for Pass Through Authentication (PTA) in DS/OpenDJ (All versions) to Active Directory? for further information.

Resolved RFE: OPENDJ-1626 (Mapping attributes for passthrough authentication).

Q. Is Elasticsearch 6 supported for audit logging?

A. No, Elasticsearch 6.x is not currently supported for audit logging because 6.0 introduced a breaking change with regards to how indices are handled (you can now only have one _type per index). An RFE exists to provide this support in a future release: CAUD-424 (ElasticSearch Audit Handler: provide support for ElasticSearch 6.0).

Per the Elasticsearch documentation (Removal of mapping types):

Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type. Indices created in 5.x with multiple mapping types will continue to function as before in Elasticsearch 6.x. Mapping types will be completely removed in Elasticsearch 7.0.0. 

This is considered a "phased roll-in" intended to give vendors and implementers some phase-in time. As a result, a possible workaround is to create the index in Elasticsearch 5 and then upgrade to Elasticsearch 6.

Q. Is Syslog logging supported in DS/OpenDJ?

A. Yes, as of OpenDJ 3, you can log audit events to Syslog as described in Administration Guide › Configuring Access Logging to Syslog. Additionally, you can configure the Syslog handler as an external log publisher that logs access messages if required as detailed in Administration Guide › Common ForgeRock Access Logs.

See How do I configure DS (All versions) and OpenDJ 3.x to use the Syslog audit event handler? for further information.

Q. What information do the native-windows.out and server.out log files contain?

A. These files can help with troubleshooting issues with the server's startup: 

  • The native-windows.out file is generated by the executable that allows DS/OpenDJ to run as a Microsoft® Windows® service. 
  • The server.out file is essentially the STDERR (Standard Errors) for the DS/OpenDJ server.

DS/OpenDJ currently overwrites these files when they reaches 0.5 MB, although they are not expected to grow much. This behavior is discussed in OPENDJ-2706 (OpenDJ overwrites native-windows.out after the file size is reached to 0.5 MB).

Q. Is the dsframework command still supported in DS/OpenDJ?

A. No, the dsframework command has been removed in OpenDJ 3. The dsframework command was included in previous versions, but as a deprecated interface; it was not documented and should not be used. 

See Also

FAQ: Installing and configuring DS/OpenDJ

FAQ: Upgrading DS/OpenDJ

FAQ: Passwords in DS/OpenDJ

FAQ: Monitoring DS/OpenDJ

FAQ: Backup and restore in DS/OpenDJ

Related Training

ForgeRock Directory Services Core Concepts (DS-400)


FAQ: REST API in DS/OpenDJ

The purpose of this FAQ is to provide answers to commonly asked questions regarding the REST API in DS/OpenDJ.

Frequently asked questions

Q. Can I use cn=Directory Manager for creating new entries?

A. You cannot easily do this if you use the HTTP connection handler nor is it recommended. If you use the REST2LDAP gateway servlet, you can achieve this as follows depending on which version of DS/OpenDJ you use:

  • DS / OpenDJ 3.5: set the bind to simple under authorization > basic to allow you to pick specific bind DNs.
  • Pre-OpenDJ 3.5: set the method to simple in the authenticationFilter to allow you to pick specific bind DNs.

However, best practice is to create a separate administrative user under the base DN who has the necessary access controls (ACIs) for creating the required entries. See Security Guide › Managing Administrator Accounts for further information.

Q. Can I do a negation query with the REST API?

A. Yes, you can using the boolean operator ! (not). Examples of queries that can be performed via REST (with the equivalent LDAP filter for comparison) are shown in Developer's Guide › Querying Resource Collections.

Q. Can I trace what is happening when a REST call is made for troubleshooting purposes?

A. Yes, if you are using the HTTP connection handler, you can enable logging for the REST interface as described in How do I troubleshoot a DS/OpenDJ (All versions) REST configuration?

As of OpenDJ 3.5, you can also enable logging in the REST to LDAP (REST2LDAP) gateway servlet if it runs in the Apache Tomcat™ web container. This is also described in How do I troubleshoot a DS/OpenDJ (All versions) REST configuration?

Q. Can I create password policies at the system level using the REST API?

A. Yes you can add and edit password policy definitions at the system level using REST (over the /admin/config endpoint) in OpenDJ 3.5 and later provided you have created the necessary mapping details in the Mapping Configuration file. See Developer's Guide › Adding Subentry Password Policies for an example.

Note

It is not possible to assign a password policy to specific users or groups using REST, nor to see and manage a user's password state (that is, locked, reset, etc.).

In pre-OpenDJ 3.5 versions, the REST API is suited to endpoints with immediate subordinate entries, which works well for users and groups, but is less well suited for reading entries from arbitrary points in the DIT. If all your password policies are at the same level in the DIT, then you can create another endpoint in your REST configuration (http-config.json file). See  OpenDJ Reference › REST to LDAP Configuration (3.0) for further information.

Q. Are there any REST endpoints in DS/OpenDJ for creating organizational units and organizations?

A. Endpoints in the DS/OpenDJ REST API can be configured to allow the creation of any type of entry in the DS/OpenDJ DIT.

As of OpenDJ 3.5, it is possible to define relationships between resources using SubResources, meaning entries can be created at different levels beneath the base DN. You can configure mapping details for a resource in the mapping configuration file to allow organizational units and organizations to be created; these resource details should be similar to the existing users resource, which allows you to create person objects below ou=People,dc=example,dc=com. See Reference › REST to LDAP Configuration for further information and specifically Reference › REST to LDAP Configuration › Mapping Configuration File for details about the mapping configuration file.

In pre-OpenDJ 3.5, each endpoint was restricted to only allowing new entries directly beneath the base DN in the endpoint configuration. You can configure endpoints and the corresponding base DN in the http-config.json file to allow organizational units and organizations to be created; these endpoints should be similar to the existing /users endpoint, which allows you to create person objects below ou=People,dc=example,dc=com. See OpenDJ Reference › REST to LDAP Configuration (3.0) for further information.

Resolved RFE: OPENDJ-2871 (Add support for sub-resources).

Q. Can I define mappings/resources using search scopes and filters in the same way as for LDAP searches?

A. No, the way mappings/resources are defined in the REST API is not directly equivalent to LDAP searches. The concept of a subtree scope for searches does not relate to mappings as the server needs to be able to compute base DNs based on the REST resources. The concept of a subtree search does not fit this model. For example, if a CREATE operation was issued, the server would not know where to write the new entry in the tree. Each mapping needs to be definable in a way that you can map the REST resource to base DN and vice versa in a strictly defined way; this is done using namingStrategy in the REST API. 

See the following for further information on namingStrategy, depending on which version of DS/OpenDJ you are using:

You cannot use filters for defining the collections included in the mappings. However, you can run filters when using the REST interface by including them in a QUERY.

Q. Does the REST API support user names and identifiers that contain *?

A. Yes, the REST API in DS/OpenDJ does work with user names and identifiers containing *. However, there is a serious risk that LDAP client applications may not escape the * character when using it in String representation of LDAP search filters, which will result in heavy, resource consuming requests in the directory.

Note

It is recommended that you avoid using * (and other special characters) in your user names and identifiers to avoid any potential issues.

Q. How can I retrieve data for a user who is under a different base DN (pre-OpenDJ 3.5)?

A. You must add another mapping to the http-config.json file to achieve this. The following mapping exists by default:

{
        "mappings" : {
            "/users" : {
                "baseDN"              : "ou=people,dc=example,dc=com",
}

If you want to query users under another base DN, for example: ou=employees,ou=people,dc=example,dc=com, you would need to add a second mapping and restart the server:

{
            "/otherusers" : {
                "baseDN"              : "ou=employees,ou=people,dc=example,dc=com",
}

A RFE exists for making this dynamic: OPENDJ-2888 (Implement a dynamic routing and mapping mode for Rest2LDAP).

See Also

How do I troubleshoot a DS/OpenDJ (All versions) REST configuration?

Administration Guide › RESTful Client Access

Developer's Guide › Performing RESTful Operations

Administration Guide › Configuring Privileges and Access Control › About ACIs

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-2888 (Implement a dynamic routing and mapping mode for Rest2LDAP)

OPENDJ-2871 (Add support for sub-resources)

OPENDJ-2196 (OpenDJ does not return the isMemberOf attribute via REST)


FAQ: Moving from Oracle DSEE to OpenDJ

The purpose of this FAQ is to provide answers to commonly asked questions regarding moving from Oracle® Directory Server Enterprise Edition (DSEE) or Sun® DSEE to OpenDJ.

Frequently asked questions

Q. What are the main changes I should be aware of when planning my migration from DSEE to OpenDJ?

A. There are a number of key areas to take into consideration when planning your migration. Unfortunately, there is not a 'one size fits all' approach as it depends on your deployment.

The main changes that you should be aware of and investigate in terms of your deployment are:

  • LDAPv3 - DSEE is tolerant of certain schema and syntax errors; OpenDJ enforces strict compliance. This can impact data migration as well as data runtime modifications.
  • Password policy - DSEE uses a legacy form of password policy; OpenDJ uses an internet draft specification of password policy with additional capabilities.
  • Replication - DSEE is peer-peer and the DS does everything; OpenDJ is hub (RS) and spoke (DS), and all RSs are fully meshed.
  • Database - DSEE uses Berkeley DB (C edition); OpenDJ uses Berkeley DB (Java edition). On disk formats are very different.
  • Roles - DSEE implements iPlanet roles; OpenDJ uses standard groups in addition to virtual attributes such as isMemberOf.
  • ACI - DSEE has macro ACIs; OpenDJ does not. The syntax of ACIs is the same but is strictly checked in OpenDJ.
  • Class of service - this is specific to DSEE; OpenDJ uses collective attributes instead.
  • Certificates - DSEE uses NSS-based utilities; OpenDJ uses Java-based utilities, and may have different ciphers.
  • API - DSEE has a C-based API for plugins; OpenDJ uses Java and has a different API.

See Migrating from Oracle DSEE to ForgeRock Directory Services for further information on planning your migration.

Q. Can I export schema and data from DSEE and then import it into OpenDJ?

A. Yes you can, but you must ensure the exported data does not include any replication metadata as this will cause conflicts; DSEE replication is not compatible with OpenDJ replication.

Additionally, you will need to make changes to the schema prior to importing it such as removing DSEE specific values and entries:

Values not used

OpenDJ uses entryUUID in place of nsUniqueId; likewise, it will create its own createTimestamp etc.

creatorsName: cn=Directory Manager
modifiersName: cn=Directory Manager
createTimestamp: 20040315171642Z
modifyTimestamp: 20040315171642Z
nsUniqueId: d88c698d-1dd111b2-80c8c04d-ec0219a2

Entries not used

The following type of entry is used only for DSEE Replication and cannot be used with OpenDJ; these must be removed as well.

# entry-id: 1
dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff, dc=sundsee,dc=com
objectClass: nsTombstone
objectClass: extensibleobject
nsds50ruv: {replicageneration} 5433f974000000000000
nsds50ruv: {replica 1 ldap://localhost.localdomain:10389} 5433f9ff000000010000
  5433f9ff000000010000
nsds50ruv: {replica 2 ldap://opendj.sundsee.com:10390}
ds6ruv: {PRIO 1 ldap://localhost.localdomain:10389}
ds6ruv: {PRIO 2 ldap://opendj.forgerock.com:10390}
nsUniqueId: ffffffff-ffffffff-ffffffff-ffffffff

You can use the attached Perl script (dseeStripper.pl) to remove all these values and entries from a DSEE export.​ The script is executed using the following command:

$ cat [export ldif file] | ./dseeStripper.pl > [new ldif filename]

You may need to make other changes depending on your deployment to allow the schema to be successfully imported as OpenDJ has strict LDAPv3 compliance and checking, and will reject errors that were tolerated in DSEE. This is likely to be an iterative process where you attempt an import, correct the errors, attempt another import and so on until your import is successful. You can relax the checking if required, although it is preferable to fix the errors to ensure you have standard LDAP data, which is more portable going forward.

Note

You can use any good text processing script language (such as Perl or Python) to make these additional changes.

See OpenDJ Administration Guide › Managing Schema for further information.

Q. Does OpenDJ support DSEE roles?

A. No, OpenDJ does not use roles; you should use dynamic and static groups in OpenDJ to perform the same function as roles.

See OpenDJ Directory Server Developer's Guide › Working With Groups of Entries and Migrating Oracle DSEE roles to OpenDJ for further information.

Q. Is there an equivalent attribute in OpenDJ for the DSEE attribute nsaccountlock?

A. Yes. OpenDJ has a ds-pwp-account-disabled attribute, which is the equivalent of the DSEE nsaccountlock attribute. You can set this attribute using the manage account command and the set-account-is-disabled sub-command.

See OpenDJ Administration Guide › Implementing Account Lockout and Notification › Managing Accounts Manually for further information.

Caution

If you disable an account in OpenDJ by setting the ds-pwp-account-disabled attribute via Sun® IDM, the user's password is scrambled.

Q. Can I use the same index entry limit in OpenDJ that I used in DSEE?

A. The default index entry limit for OpenDJ is 4000 and this is considered sufficient for most indexes. It is not recommended that you increase this as it will have an impact on performance. DSEE recommended that the index entry limit (all-ids-threshold property) was set to 5% of the total number of users, which could result in a much higher limit than 4000.

See OpenDJ Administration Guide › Indexing Attribute Values › Understanding Index Entry Limits for further information.

Q. Can our users keep their passwords when we migrate?

A. Yes. The password hashing support in OpenDJ is backwards compatible with DSEE, so you should be able to migrate and keep the same passwords.

See OpenDJ Administration Guide › Configuring Password Policy for further information.

Q. Is the tuning I implemented for DSEE sufficient for OpenDJ?

A. No. Tuning in OpenDJ is completely different to DSEE and you should look to do some tuning once you have migrated to ensure the best performance.

See OpenDJ Administration Guide › Tuning Servers For Performance and FAQ: DS/OpenDJ performance and tuning for further information.

See Also

Migrating from Oracle DSEE to OpenDJ

Migrating Oracle DSEE roles to OpenDJ

Migrating Oracle DSEE CoS to OpenDJ

OpenDJ Administration Guide


Known Issues


LDAP connection fails with No subject alternative DNS name matching error in AM 5.1.x, 6.x and DS 5.5.1, 5.5.2, 6.x

The purpose of this article is to provide assistance if your LDAP/LDAPS connection fails with a "java.security.cert.CertificateException: No subject alternative DNS name matching" error. This issue can affect AM and DS.

Symptoms

The "No subject alternative DNS name matching" error can be seen in various AM debug logs with different contexts, for example:

  • Configuration debug log:
    ERROR: SMSEntry: Unable to initalize(exception):
    SMSException Exception Code:5
    Message:Unexpected LDAP exception occurred.
    --------------------------------------------------
    The lower level exception message
    Connect Error: No operational connection factories available
    The lower level exception:
    Connect Error: No operational connection factories available
       at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:206)
       at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:144)
       at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:113)
    ...
    Caused by: Connect Error: The LDAP connection has failed because an error occurred during the SSL handshake: java.security.cert.CertificateException: No subject alternative DNS name matching host1.example.com found.
    ...
    Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
       at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1478)
       at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
       at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
       at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
       at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
    ...
    Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching host1.example.com found.
    
  • IdRepo debug log:
    ERROR: An error occurred while trying to initiate persistent search connection
    Connect Error: No operational connection factories available
       at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:206)
       at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:144)
       at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:113)
    ...
    Caused by: Connect Error: The LDAP connection has failed because an error occurred during the SSL handshake: java.security.cert.CertificateException: No subject alternative DNS name matching host1.example.com found
    ...
    Caused by: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
       at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1529)
       at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535)
       at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214)
       at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186)
       at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469)
    ...
    Caused by: java.security.cert.CertificateException: No subject alternative DNS name matching host1.example.com found.
    
    
  • OpenDJ-SDK debug log:
    WARNING: Connection factory 'CachedConnectionPool(size=0[in:0 + out:0 + pending:0], maxSize=10, blocked=0, ldapClient=org.forgerock.opendj.ldap.LdapClientImpl@57c4d233)' is no longer operational: Connect Error: The LDAP connection has failed because an error occurred during the SSL handshake: java.security.cert.CertificateException: No subject alternative DNS name matching host1.example.com found.
    

Amster

You may also see a similar error when trying to install AM using Amster:

"stderr": "SLF4J: Failed to load class \"org.slf4j.impl.StaticLoggerBinder\".\nSLF4J: Defaulting to no-operation (NOP) logger implementation\nSLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.\nFailed to execute the 'install-openam' command: java.security.cert.CertificateException: No subject alternative DNS name matching host1.example.com found.",

Upgrade

You may encounter this error during an AM upgrade as well; in which case, you will also see an error similar to the following in the web application container log (for example, catalina.out for Apache Tomcat™):

org.forgerock.openam.upgrade.UpgradeException: Unable to parse product versions for comparison. Current: null war: ForgeRock Access Management 6.0.0 Build 3676519ec1 (2018-May-08 10:07)
Note

If you see the "Unable to parse product versions for comparison" error without the "No subject alternative DNS name matching" error, you should refer to AM/OpenAM (All versions) upgrade fails when com.iplanet.am.version is empty or corrupted for further information on resolving this known upgrade issue.

Recent Changes

There are various changes that could cause this issue, including (but not limited to):

  • Upgraded to, or installed AM 5.1.x; or AM 6 or later.
  • Upgraded to, or installed DS 5.5.1 or later.
  • Upgraded Java® to 1.7.0_191 or later; or 1.8.0_181 or later (including Oracle® JDK and OpenJDK).
  • Created a new data store (configuration, identity or CTS) or changed the DNS name of an existing one.
  • Changed server certificates.
  • Enabled SSL (changed the connection from LDAP to LDAPS).

Causes

Changes introduced in recent AM and DS versions (AM 5.1.x; AM 6 or later; DS 5.5.1 or later) have made LDAP SSL hostname validation stricter. AM checks the hostname in the LDAP server certificate correctly matches the hostname used to connect to the secured directory server (for example, DS or Active Directory®) and DS checks that the server it is trying to connect to has a certificate that matches the hostname.

These changes affect any secured LDAP/LDAPS connections:

  • AM connecting to a configuration store, identity store, CTS store etc.
  • Connections from LDAP command line tools such as ldapsearch.
  • Connections from admin tools such as dsconfig and dsreplication.
  • Connections to DS using the external REST2LDAP interface or DSML gateway; this includes IDM connecting to an external DS repository as that uses the REST2LDAP interface.
  • DS connecting to other LDAP servers when configured for pass-through authentication.

Java

Java 1.7.0_191 and 1.8.0_181 introduced changes to improve LDAP support by enabling endpoint identification algorithms by default for LDAPS connections; this also results in stricter hostname validation. For further information see:

Solution

This issue can be resolved by ensuring the hostname you are connecting to the LDAP server with matches the hostnames specified in the server certificate via the SAN (Subject Alternative Name). The SAN allows you to specify multiple hostnames and takes precedence over the Subject/CN entry in the certificate; you should list all the DNS hostnames that you expect to connect to the LDAP server in the SAN. See Administration Guide › Setting Up Server Certificates for further information.

Put simply, this means the hostname in the error (for example, host1.example.com) should be added to the SAN in the LDAP server certificate. The server certificate is located in the JVM truststore used by AM, which is $JAVA_HOME/jre/lib/security/cacerts by default. In the AM configuration, you should also check that the hostname specified for the LDAP connection is correct (and matches what is in the certificate's SAN).

Alternatives

The following options can be used to switch off stricter hostname validation; however this would make your deployment less secure and as such is not recommended: 

  • You can switch off hostname checks in DS by setting the org.forgerock.opendj.hostNameVerificationDisabled environment variable to true. See Javadoc › SSL_HOST_NAME_VALIDATION_DISABLED_PROPERTY for further information. 
  • You can switch off hostname validation in Java by setting the com.sun.jndi.ldap.object.disableEndpointIdentification JVM option to true. 

See Also

How do I configure LDAPS clients in DS/OpenDJ (All versions)?

How do I make AM/OpenAM (All versions) communicate with a secured LDAP server?

How do I import a certificate into the truststore used by AM/OpenAM (All versions) for SSL?

FAQ: SSL certificate management in AM/OpenAM and Policy Agents

FAQ: SSL certificate management in DS/OpenDJ

Best practice for upgrading to AM 6.x

SSL in DS/OpenDJ

SSL in AM/OpenAM and Policy Agents

Upgrade Guide

Related Training

N/A

Related Issue Tracker IDs

OPENAM-13984 (Upgrade/Install Document need for stricter Hostname Matching for LDAP certificates)


SSL handshake failed with no cipher suites in common in DS 5 after restricting cipher suites or upgrading Java

The purpose of this article is to provide assistance if you encounter "SSL handshake failed" errors in DS 5 after restricting cipher suites to more secure ones (for example SHA384), installing DS in production mode and/or updating Java® to JDK 1.8.0_192 or later. The reason for the failure is given as "no cipher suites in common".

Symptoms

Running commands that connect over LDAPS fail with an SSL handshake failed error, for example:

  • Using the ldapsearch command:
    Unable to connect to the server: 82 (Local Error)
    Additional Information: SSL handshake failed
    
  • Using the dsreplication command:
    Could not connect to localhost:4444. Check that the server is running and that the provided credentials are valid.
    Error details: Local Error: SSL handshake failed
    

If you enable SSL debug logging, you will see errors similar to the following in the server.out log depending on how you are connecting:

  • LDAPS Connection Handler:
    %% Initialized:  [Session-9, SSL_NULL_WITH_NULL_NULL]
    LDAPS Connection Handler 0.0.0.0 port 1636(2) SelectorRunner, fatal error: 40: no cipher suites in common
    javax.net.ssl.SSLHandshakeException: no cipher suites in common
    %% Invalidated:  [Session-9, SSL_NULL_WITH_NULL_NULL]
    LDAPS Connection Handler 0.0.0.0 port 1636(2) SelectorRunner, SEND TLSv1.2 ALERT:  fatal, description = handshake_failure
    LDAPS Connection Handler 0.0.0.0 port 1636(2) SelectorRunner, WRITE: TLSv1.2 Alert, length = 2
    LDAPS Connection Handler 0.0.0.0 port 1636(2) SelectorRunner, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLHandshakeException: no cipher suites in common
    LDAPS Connection Handler 0.0.0.0 port 1636(2) SelectorRunner, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLException: SSLEngine is closing/closed
    
  • Administration Connector:
    %% Initialized:  [Session-50, SSL_NULL_WITH_NULL_NULL]
    Administration Connector 0.0.0.0 port 4444(1) SelectorRunner, fatal error: 40: no cipher suites in common
    javax.net.ssl.SSLHandshakeException: no cipher suites in common
    %% Invalidated:  [Session-50, SSL_NULL_WITH_NULL_NULL]
    Administration Connector 0.0.0.0 port 4444(1) SelectorRunner, SEND TLSv1.2 ALERT:  fatal, description = handshake_failure
    Administration Connector 0.0.0.0 port 4444(1) SelectorRunner, WRITE: TLSv1.2 Alert, length = 2
    Administration Connector 0.0.0.0 port 4444(1) SelectorRunner, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLHandshakeException: no cipher suites in common
    Administration Connector 0.0.0.0 port 4444(1) SelectorRunner, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLException: SSLEngine is closing/closed
    

You can enable SSL debug logging as described in FAQ: SSL certificate management in DS/OpenDJ (Q. How do I debug a SSL handshake error?).

Additionally, the server.out log does not show the more secure cipher suites that have been enabled.

Recent Changes

Upgraded to JDK 1.8.0_192 or later.

Restricted cipher suites to more secure ones such as TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, which use protocol version TLSv1.2.

Installed DS using the --productionMode setup option (which restricts the available ciphers and forces the protocol version to TLSv1.2).

Causes

Due to a change introduced in JDK 1.8.0_192 (JDK-8162362 : Introduce system property to control enabled ciphersuites), DS does not set the enabled cipher suites, which means the LDAPS connection falls back to using the default cipher suites list. Another consequence of this Java change is that only protocol TLSv1 is used with the cipher suites despite TLSv1.2 being set. These changes mean the client is using different cipher suites and protocol to the server, which causes this SSL handshake error since both the client and server must support the same cipher suites and protocol agreed upon when attempting to establish a secure connection. 

Changes have been made in DS 5.5 (via OPENDJ-4341 (setup with production mode with java 9 )) to provide support for these Java changes in JDK 1.8.0_192 and later.

Solution

This issue can be resolved by upgrading to DS 5.5 and later; you can download this from BackStage.

Workaround

You can workaround this issue by downgrading Java to JDK 1.8.0_191 or earlier.

See Also

How do I prevent the use of weak SSL cipher suites in DS/OpenDJ?

How do I troubleshoot connection via LDAPS issues in DS/OpenDJ (All versions)?

FAQ: SSL certificate management in DS/OpenDJ

LDAPS client connections fail with SSLHandshakeException: no cipher suites in common in DS 5 and OpenDJ 3.x

Invalid Padding length error when attempting to connect to DS 5 or OpenDJ 3.x via LDAPS

SSL in DS/OpenDJ

Administration Guide › TLS Protocols and Cipher Suites

Configuration Reference › LDAP Connection Handler

Security Guide › Set Up Servers in Production Mode

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-4341 (setup with production mode with java 9 )


LDAPS client connections fail with SSLHandshakeException: no cipher suites in common in DS 5 and OpenDJ 3.x

The purpose of this article is to provide assistance if ldapsearch fails with a "javax.net.ssl.SSLHandshakeException: no cipher suites in common" error in DS/OpenDJ when connecting via LDAPS if you are using Java® 7.

Symptoms

The following error is shown when searching using the ldapsearch command if you are connecting to DS/OpenDJ via LDAPS:

java.net.SocketException: Socket Closed 
at java.net.PlainSocketImpl.setOption(PlainSocketImpl.java:201) 
at java.net.Socket.setSoTimeout(Socket.java:1017) 
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.setSoTimeout(SSLSocketImpl.java:2107) 
at org.opends.server.tools.LDAPConnection.connectToHost(LDAPConnection.java:507) 
at org.opends.server.tools.LDAPSearch.mainSearch(LDAPSearch.java:1777) 
at org.opends.server.tools.LDAPSearch.main(LDAPSearch.java:580) 
Cannot send the simple bind request: SSLHandshakeException(Remote host closed connection during handshake) 
Result Code: 81 (Server Connection Closed) 

An error similar to the following is shown in the DS/OpenDJ access log when the above error is received:

[21/Sep/2014:10:07:52 +0200] DISCONNECT conn=9359 reason="I/O Error" msg="An IO error occurred while reading a request from the client: javax.net.ssl.SSLHandshakeException: no cipher suites in common"

Recent Changes

Upgraded to Java 7 (pre-update 51).

Configured LDAPS (LDAP/SSL) between the client and the server.

Causes

The SSL error is caused by the client failing to encode the SSL handshake protocol properly; the no cipher suites in common relates to the client offering cipher suites that are not supported by the server.

This issue occurs when using DH (Diffie-Hellman) based cipher suites (such as: TLS_DHE_RSA_WITH_AES_128_CBC_SHA) and is a known Java 7 issue: JDK-8014618 : Need to strip leading zeros in TlsPremasterSecret of DHKeyAgreement

You can check which cipher suites are being used by performing a simple search for "supportedTLSCiphers supportedTLSProtocols" as described in Administration Guide › To List Protocols and Cipher Suites.

Solution

This issue can be resolved by upgrading to at least Java 7u51 or by using non-DH cipher suites.

To use non-DH cipher suites:

You can either configure your client to use non-DH cipher suites or configure the LDAPS connection handler in DS/OpenDJ to offer a limited set of cipher suites using the dsconfig set-connection-handler-prop command.

For example, to permit TLS_RSA_WITH_AES_128_CBC_SHA and SSL_RSA_WITH_RC4_128_SHA cipher suites, you would use a command similar to the following:

$ ./dsconfig set-connection-handler-prop --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --handler-name "LDAPS Connection Handler" --add ssl-cipher-suite:TLS_EMPTY_RENEGOTIATION_INFO_SCSV --add ssl-cipher-suite:TLS_RSA_WITH_AES_128_CBC_SHA --add ssl-cipher-suite:SSL_RSA_WITH_RC4_128_SHA -n -X
Note

The TLS_EMPTY_RENEGOTIATION_INFO_SCSV cipher suite is a special cipher suite used by Java and should always be included.

See Also

FAQ: SSL certificate management in DS/OpenDJ

Invalid Padding length error when attempting to connect to DS 5 or OpenDJ 3.x via LDAPS

How do I troubleshoot connection via LDAPS issues in DS/OpenDJ (All versions)?

How do I prevent the use of weak SSL cipher suites on DS/OpenDJ (All versions) administration port?

How do I prevent the use of weak SSL cipher suites on DS/OpenDJ (All versions) replication port?

SSL in DS/OpenDJ

Configuration Reference › LDAP Connection Handler

OpenDJ: Troubleshooting LDAP SSL connections

Related Training

N/A

Related Issue Tracker IDs

N/A


DS (All versions) fails to start when using a JKS keystore from an earlier version

The purpose of this article is to provide assistance if DS fails to start when using a JKS keystore, which was likely copied from an earlier version of OpenDJ. You will see "An error occurred while trying to load the keystore contents from file /path/to/ds/keystore: IOException(DerInputStream.getLength(): lengthTag=109, too big.)" message when this happens.

Symptoms

The following error is shown when DS fails to start:

category=CORE severity=NOTICE msgID=139 msg=The Directory Server has sent an alert notification generated by class org.opends.server.core.DirectoryServer (alert type org.opends.server.DirectoryServerShutdown, alert ID org.opends.messages.core-141): The Directory Server has started the shutdown process. The shutdown was initiated by an instance of class org.opends.server.core.DirectoryServer and the reason provided for the shutdown was An error occurred while trying to start the Directory Server: InitializationException: An error occurred while attempting to initialize the SSL context for use in the LDAP Connection Handler: An error occurred while trying to load the keystore contents from file /path/to/ds/keystore: IOException(DerInputStream.getLength(): lengthTag=109, too big.) (id=org.opends.messages.extension-62) (LDAPConnectionHandler.java:416 LDAPConnectionHandler.java:114 ConnectionHandler.java:134 AdministrationConnector.java:95 ConnectionHandlerConfigManager.java:240 ConnectionHandlerConfigManager.java:203 DirectoryServer.java:1546 DirectoryServer.java:1317 DirectoryServer.java:4210)

Recent Changes

Installed DS 5 or later, and used a keystore from an earlier version of OpenDJ.

Causes

The default keystore format in DS 5 and later is PKCS#12. If you use a different keystore type (such as a JKS keystore from an earlier version of OpenDJ) DS will fail to start due to the keystore incompatibility.

The default keystore type in OpenDJ 3.5.x and earlier was JKS. Changes were made in DS 5 to simplify the default keystore and truststore configuration; this included changing the default keystore format to PKCS#12. See Release Notes › Important Changes to Existing Functionality (setup tool) for further information.

Solution

This issue can be resolved using one of the following approaches:

  • Convert your existing JKS keystore to PKCS#12 using keytool, for example:
    $ keytool -importkeystore -srckeystore [existing_JKS_keystore] -srcstoretype JKS -srcstorepass password -destkeystore new-keystore -deststoretype PKCS12 -deststorepass password -destkeypass password
  • Change the Default Key Manager to JKS using dsconfig, for example:
    $ ./dsconfig set-key-manager-provider-prop --provider-name "Default Key Manager" --set key-store-type:JKS --hostname ds1.example.com --port 4444 --bindDn "cn=Directory Manager" --bindPassword password --trustAll --no-prompt
    
    You can now continue to use the JKS keystore with DS.

See Also

SSL in DS/OpenDJ

Security Guide › Managing Certificates and Private Keys

Related Training

N/A

Related Issue Tracker IDs

N/A


An error occurred while trying to decode the response from the server when running commands in DS/OpenDJ (All versions)

The purpose of this article is to provide assistance if you encounter an "ERROR: An error occurred while trying to decode the response from the server: null" response when running commands such as backup, restore and status in DS/OpenDJ.

Symptoms

The following response is shown when running commands such as backup, restore, status etc from the command line:

ERROR: An error occurred while trying to decode the response from the server: 
null

Details: [LDAP: error code 80 - An unexpected error was encountered while 
processing a search in one of the Directory Server backends: 
NullPointerException(TaskScheduler.java:1541)]

An uncaught exception while processing operation AddOperation is shown in the access logs when this occurs:

[16/Aug/2016:10:04:52 +0100] DISCONNECT conn=0 reason="Server Error" msg="Worker Thread 60 encountered an uncaught exception while processing operation AddOperation(connID=0, opID=1, dn=ds-task-id=20150805080006222,cn=Scheduled Tasks,cn=Tasks):  
NullPointerException (TaskScheduler.java:1555 TaskBackend.java:491 DirectoryServer.java:6688 LocalBackendAddOperation.java:354 LocalBackendWorkflowElement.java:539 WorkflowImpl.java:197 WorkflowTopologyNode.java:100 AddOperationBasis.java:764 TraditionalWorkerThread.java:167)"

Recent Changes

N/A

Causes

The exceptions seen in the error messages indicate there is an issue adding a task to the task scheduler backend; this can be caused by an empty or corrupt tasks.ldif file (located in /path/to/ds/db/tasks in DS 6 and later, or /path/to/ds/config in pre-DS 6). The tasks.ldif is a file-based database that stores information on past, current and future tasks for the task scheduler backend, for example, backup tasks that are currently running or scheduled.

An empty or corrupt tasks.ldif file can occur for a variety of reasons, including the server running out of disk space or taking a VM snapshot while the server is running; this is not recommended anyway, since taking a VM snapshot while the server is running can corrupt the database if a change is being made to one of the main backends at the same time.

Solution

This issue can be resolved by populating the tasks.ldif file and restarting the DS/OpenDJ server.

You can recover the file from a valid copy of your previous tasks.ldif in the file tasks.ldif.save, else you can create a generic tasks.ldif file containing the following:

# This file contains the data used by the Directory Server task scheduler
# backend.  Do not edit this file directly, as there is a risk that those
# changes will be lost.  Scheduled and recurring task definitions should only
# be edited using the administration utilities provided with the Directory
# Server
dn: cn=tasks
objectClass: top
objectClass: untypedObject
cn: tasks

dn: cn=Scheduled Tasks,cn=tasks
objectClass: top
objectClass: untypedObject
cn: Scheduled Tasks

dn: cn=Recurring Tasks,cn=tasks
objectClass: top
objectClass: untypedObject
cn: Recurring Tasks

See Also

FAQ: Backup and restore in DS/OpenDJ

How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-3234 (Unhelpful error messages when server cannot read/write tasks backend)


The provided value cannot be parsed as a valid IA5 string because it contains an illegal character error in DS/OpenDJ (All versions)

The purpose of this article is to provide assistance if you encounter the following error when working with directory data: "The provided value "[value with special characters]" cannot be parsed as a valid IA5 string because it contains an illegal character "-n" that is not allowed in the IA5 (ASCII) character set". This error can also manifest in IDM/OpenIDM when used in conjunction with DS/OpenDJ.

Symptoms

This error occurs when working with directory data and typically affects the mail attribute, but can affect custom attributes. Some common scenarios are:

  • Modifying the mail attribute, for example using the ldapmodify command will give a response similar to the following:
    # The LDAP modify request failed: 21 (Invalid Attribute Syntax)
    # Additional Information:  When attempting to modify entry uid=jdoe,ou=People,dc=example,dc=com to replace the set of values for attribute mail, value "ää@example.com" was found to be invalid according to the associated syntax: The provided value "ää@example.com" cannot be parsed as a valid IA5 string because it contains an illegal character "-61" that is not allowed in the IA5 (ASCII) character set
    
  • Importing data using the import-ldif command will give a response similar to the following:
    # Entry cn=jdoe,ou=people,dc=example,dc=com read from LDIF starting at line 100 includes value "Buenos Días" for attribute customGreeting that is invalid according to the associated syntax:  The provided value "Buenos Días" cannot be parsed as a valid IA5 string because it contains an illegal character "-61" that is not allowed in the IA5 (ASCII) character set
    
  • Adding a new user (could be via another system such as IDM/OpenIDM) will result in an error similar to the following in the Access log:
    "response":{"status":"FAILED","statusCode":"21","elapsedTime":1,"elapsedTimeUnits":"MILLISECONDS","detail":"Entry "uid=jdoe,ou=People,dc=example,dc=com" contains a value "êê@example.com" for attribute mail that is invalid according to the syntax for that attribute: The provided value "êê@example.com" cannot be parsed as a valid IA5 string because it contains an illegal character "-61" that is not allowed in the IA5 (ASCII) character set"},"timestamp":"2018-11-10T12:04:53.850Z","_id":"2fff31b5-4682-4760-b32a-fbbfed6bd9ac-433"}
    

Recent Changes

The following are some example use cases of administrative changes that commonly introduce these errors:

  • Changed the mail attribute to use internationalized email addresses.
  • Created users with special characters (symbols) or internationalized characters in their username.
  • Defined a custom attribute where the provided value contains special characters (symbols) or internationalized characters.

Causes

The mail attribute is a standard RFC definition that predates the SMTP enhancements that allow UTF-8 addresses and is defined with the following syntax: IA5String 1.3.6.1.4.1.1466.115.121.1.26 as shown in the 00-core-ldif file:

attributeTypes: ( 0.9.2342.19200300.100.1.3 NAME ( 'mail' 'rfc822Mailbox' )
  EQUALITY caseIgnoreIA5Match SUBSTR caseIgnoreIA5SubstringsMatch
  SYNTAX 1.3.6.1.4.1.1466.115.121.1.26{256} X-ORIGIN 'RFC 4524' )

See LDAP Schema Reference › mail for further information.

The IA5 character set does not include special characters (such as accented characters, Japanese characters etc):

This means the default mail attribute (and any other attributes defined in a similar way) do not support these special characters.

Solution

This issue can be resolved by ensuring all entries for the affected attribute only contain permitted characters.

Alternatively, you could change the default schema that comes with DS/OpenDJ (in the 00-core-ldif file above), but in doing so you may run into issues with other systems that expect the mail attribute to conform to the standard schema. Similarly, email servers may also take exception to the characters used. As such, we strongly advise against changing the schema.

The Developer Guide states that you should: Reuse Schemas Where Possible, with the pertinent information being:

You therefore should reuse schema definitions that already exist whenever you reasonably can. Reuse them as is. Do not try to redefine existing schema definitions.

If you must add schema definitions for your application, extend existing object classes with AUXILIARY classes of your own. Take care to name your definitions such that they do not clash with other names.

See Also

Installing and Administering DS/OpenDJ

Administration Guide › Managing Schema

Reference › Standards, RFCs, and Internet-Drafts

Related Training

N/A

Related Issue Tracker IDs

N/A


Generation IDs do not match error after restoring a DS/OpenDJ (All versions) replica

The purpose of this article is to provide assistance if a "Generation IDs do not match" error occurs during replication after restoring an old backup or LDIF data to a DS/OpenDJ instance.

Symptoms

This article is specific to restoring old data, only. Similar errors are expected on a new directory server node “after” you run dsreplication configure and "before" you run dsreplication initialize. This is normal and the final phase of setting up replication, as the new backend needs the Generation ID task run to reset the Generation ID. See Initializing Replicas for detailed instructions. Ensure to refer to the admin guide for your specific version of DS/OpenDJ.

When restoring a replica, the following error is shown in the DS/OpenDJ error logs:

[10/Oct/2016:15:19:33 +0000] category=SYNC severity=SEVERE_WARNING msgID=14811232 msg=Directory server DS(42134) has connected to replication server RS(42706) for domain "cn=admin data" at ds1.example.com/10.127.0.61:8080, but the generation IDs do not match, indicating that a full re-initialization is required. The local (DS) generation ID is 36215293 and the remote (RS) generation ID is 172193
[10/Oct/2016:15:19:33 +0000] category=SYNC severity=SEVERE_WARNING msgID=14811272 msg=Replication server RS(42706) not sending update 00000148c7a7e07c102600000001 for domain "cn=admin data" to directory server DS(42134) at ds1.example.com/10.127.0.61:8989 because its generation ID 36215293 is different to the local generation ID 172193

The restored replica is not synchronized with the other replication servers.

Recent Changes

Restored old backup or imported LDIF data to a DS/OpenDJ instance.

Causes

The generation ID contained in the restored data is not the same as the one in the current replication topology. The generation ID is a checksum of attributes from some of the entries and is used during replication to check that the suffix being updated is the same as the one offering the updates.

Solution

This issue can be resolved using one of the following approaches:

  • Re-initializing the suffix from another replica, since the generation IDs will naturally be in sync.
  • Using pre-external-initialization and post-external-initialization as part of your restore process if you are certain the data is correct.

Re-initializing the suffix

You can re-initialize the suffix as follows:

  1. Initialize the suffix using the dsreplication command, for example:
    $ ./dsreplication initialize --adminUID admin --adminPassword password --baseDN dc=example,dc=com --hostSource ds1.example.com --portSource 4444 --hostDestination ds2.example.com --portDestination 5444 --trustAll --no-prompt

Using pre-external-initialization and post-external-initialization

If you are certain the data is correct, you can use pre-external-initialization and post-external-initialization as part of your restore process. These commands ensure the generation ID of the replicated domain is updated. "Old" changes will not get replayed because they were targeting the data using the previous generation ID.  

You restore a replica as follows:

  1. Prepare the domain on all servers for being externally initialized. You must specify the baseDN of the data you are going to be changing, for example:
    $ ./dsreplication pre-external-initialization --hostname ds1.forgerock.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt
  2. Restore or import from ldif to restore the data:
    • Using the restore command, for example:
      $ ./restore --hostname ds1.forgerock.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --backupID 20160614031032Z --backupDirectory /path/to/ds/backupfile
      
    • Using the import-ldif command, for example:
      $ ./import-ldif --hostname ds1.example.com --port 4444--includeBranch dc=example,dc=com --backendID userRoot --ldifFile /path/to/ldif_file
  3. Enter the following command to set the new generation ID for the domain and broadcast it to all the servers, which allows them to replicate again. Ensure you use the same baseDN as in step 1:
    $ ./dsreplication post-external-initialization --hostname ds1.forgerock.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --trustAll --no-prompt

See Also

How do I restore old backup data to a DS/OpenDJ (All versions) replication topology?

How do I roll back an entire network of DS/OpenDJ (All versions) replicas to a previous backup?

How do I configure DS/OpenDJ (All versions) to ensure accidentally deleted or changed data can be restored when replication is enabled?

How do I design and implement my backup and restore strategies for DS/OpenDJ (All versions)?

FAQ: Backup and restore in DS/OpenDJ

Administration Guide › Backing Up and Restoring Data

Administration Guide › Managing Data Replication › Initializing Replicas

Related Training

ForgeRock Directory Services Core Concepts

Related Issue Tracker IDs

N/A


JE database backend growing rapidly in OpenDJ 2.6.x, 3.0, 3.5 and 3.5.1

The purpose of this article is to provide assistance if a database backend (for example, userRoot) starts growing rapidly in OpenDJ 2.6.x, 3.0, 3.5 and 3.5.1 for no apparent reason. This issue can cause you to run out of disk space if left unchecked. The embedded OpenDJ can also be affected in OpenAM 11.x, 12.x and 13.x.

Symptoms

A database backend grows rapidly for no reason. Restarting the server may restore the database backend to its usual size.

You may see signs in the je.info.0 file (located in the /path/to/opendj/db/backend_name directory) that the cleaning process is not working effectively for the affected backend. Things to look for are as follows, where the backend in these examples is userRoot:

  • lnSizeCorrectionFactor drops below 0.5, for example:
    2017-01-10 10:37:22.477 GMT INFO [/path/to/opendj/db/userRoot] Chose lowest utilized file for cleaning. fileChosen: 0xf044 lnSizeCorrectionFactor: 0.341327 totalUtilization: 49 bestFileUtilization: 10 isProbe: false
    
  • Files are not being deleted (fileDeleted=false), for example:
    2017-01-10 10:37:21.429 GMT INFO [/path/to/opendj/db/userRoot] CleanerRun 788 ends on file 0xf044 probe=false invokedFromDaemon=true finished=true fileDeleted=false nEntriesRead=20455 nINsObsolete=4304 nINsCleaned=1314 nINsDead=0 nINsMigrated=1314 nBINDeltasObsolete=7925 nBINDeltasCleaned=1794 nBINDeltasDead=0 nBINDeltasMigrated=1794 nLNsObsolete=1406 nLNsCleaned=3709 nLNsDead=0 nLNsMigrated=3709 nLNsMarked=0 nLNQueueHits=1412 nLNsLocked=0 logSummary=<CleanerLogSummary endFileNumAtLastAdjustment="0xf052" initialAdjustments="5" recentLNSizesAndCounts="Cor:172/1-Est:9571857/3694 Cor:140/1-Est:10254376/3164 Cor:124/1-Est:9360777/4179 Cor:5536/4-Est:9867834/6213 Cor:59139/26-Est:7818668/3255 Cor:214/1-Est:10019874/4069 Cor:1734/3-Est:10336372/4604 Cor:9155/5-Est:8702308/4023 Cor:156/1-Est:9266763/4814 Cor:69442/30-Est:7293631/3739 "> inSummary=<INSummary totalINCount="5618" totalINSize="11579684" totalBINDeltaCount="9719" totalBINDeltaSize="1645150" obsoleteINCount="4304" obsoleteINSize="9408393" obsoleteBINDeltaCount="7925" obsoleteBINDeltaSize="1409529"/> estFileSummary=<summary totalCount="20836" totalSize="99958921" totalINCount="15337" totalINSize="13224834" totalLNCount="5115" totalLNSize="86713267" maxLNSize="798458" obsoleteINCount="12229" obsoleteLNCount="1406" obsoleteLNSize="79419636" obsoleteLNSizeCounted="1376" getObsoleteSize="90043834" getObsoleteINSize="10544858" getObsoleteLNSize="79478156" getMaxObsoleteSize="97219601" getMaxObsoleteLNSize="86653923" getAvgObsoleteLNSizeNotCounted="1950.6903"/> recalcFileSummary=<summary totalCount="20836" totalSize="99958921" totalINCount="15337" totalINSize="13224834" totalLNCount="5115" totalLNSize="86713267" maxLNSize="0" obsoleteINCount="12229" obsoleteLNCount="1406" obsoleteLNSize="79489078" obsoleteLNSizeCounted="1406" getObsoleteSize="90054756" getObsoleteINSize="10544858" getObsoleteLNSize="79489078" getMaxObsoleteSize="90054756" getMaxObsoleteLNSize="79489078" getAvgObsoleteLNSizeNotCounted="NaN"/> lnSizeCorrection=0.33163622 newLnSizeCorrection=0.33163622 estimatedUtilization=10 correctedUtilization=10 recalcUtilization=10 correctionRejected=false
    

Checking space utilization

You can also check space utilization as follows:

  1. Change to the OpenDJ directory:
    • External OpenDJ:
      $ cd /path/to/opendj
    • Embedded OpenDJ:
      $ cd $HOME/[openam_instance]/opends
  2. Run the following command (replacing backend with the actual name of your backend):
    $ java -cp lib/je.jar:lib/OpenDJ.jar com.sleepycat.je.util.DbSpace -h db/backend
    The presence of 0 occupancy files indicates the cleaning process is not working as expected, for example, if you see an output such as:
      File    Size (KB)  % Used
    --------  ---------  ------
    0xf044      97654       2
    0xf045      97653       0
    0xf046      97653       0
    0xf047      97653       0
    0xf048      97654       0
    0xf049      97655       0
    

Recent Changes

N/A

Causes

An underlying bug in JE 5 is responsible for this issue. It causes the cleaning process to be adversely affected, which leaves the database full of log files that have a high percentage of deleted records. This bug is fixed in JE 6 and later.

A JE database backend is effectively an ordered sequence of append-only transaction log files with the jdb extension. Each one can grow up to a configurable maximum, and in OpenDJ 2.6.2 and later, the default maximum size is 100MB (ds-cfg-db-log-file-max). Each log file contains a percentage of live records and (when data is modified) deleted records. These percentages are monitored, and when the amount of live data falls below 50% (this is also configurable), cleaner threads actively migrate the remaining live records to the end of the transaction log (the latest jdb file) and remove the log that's now full of deleted records.

Warning

You must not delete these jdb files yourself, else you will corrupt the database.

Solution

This issue can be resolved by upgrading to DS 5 and later, or OpenDJ 3.5.2; you can download these from BackStage.

These newer versions use JE 7, which is under an Apache™ License.

Workaround

A suggested workaround is to try setting the je.cleaner.adjustUtilization property to false. This setting causes the cleaner to assume that all records in a log are average size; this may occasionally result in under or over cleaning, but overall cleaning should be consistent and can prevent this issue from re-occurring.

For example, to set this for the userRoot database backend, you would use a dsconfig command such as:

$ ./dsconfig set-backend-prop --backend-name userRoot --set je-property:je.cleaner.adjustUtilization=false --hostname localhost--port 4444 --bindDN "cn=Directory Manager" --bindPassword password --no-prompt --trustAll
Note

OpenDJ 2.6.2 and later has a default database size of 100mb: (ds-cfg-db-log-file-max: 100 megabytes). If you use an earlier version, or have upgraded to OpenDJ 2.6.2 or later, you should check this setting and increase to 100 megabytes if needed.

See Also

What versions of DS/OpenDJ are compatible with AM/OpenAM?

Administration Guide › About Database Backends

Administration Guide › Setting Disk Space Thresholds For Database Backends

Cleaner making no progress / lnSizeCorrectionFactor became very small

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-3428 (JE cleaner threads stop deleting files)

OPENDJ-2362 (Setting CLEANER_MIN_FILE_UTILIZATION instead of CLEANER_MIN_UTILIZATION)

OPENDJ-1554 (Backend in default config.ldif does not use the correct defaults)


Insufficient free memory error when installing OpenDJ 3.5

The purpose of this article is to provide assistance if you encounter the following error when installing or upgrading to OpenDJ 3.5 using the setup command: "An error occurred while attempting to process the LDIF import: Insufficient free memory".

Symptoms

The following error is shown when running the ./setup command to install or upgrade to OpenDJ 3.5:

[22/09/2016:12:11:57 +0000] category=BACKEND seq=21 severity=INFO msg=Import LDIF environment close took 0 seconds 
[22/09/2016:12:11:57 +0000] category=QUICKSETUP seq=72 severity=INFO msg=import-ldif out log: [22/09/2016:12:11:57 +0000] category=BACKEND seq=22 severity=INFO msg=Flushing data to disk 
[22/09/2016:12:11:57 +0000] category=QUICKSETUP seq=73 severity=WARNING msg=import-ldif error log: 
[22/09/2016:12:11:57 +0000] category=TOOLS seq=23 severity=SEVERE msg=An error occurred while attempting to process the LDIF import: Insufficient free memory (86113843 bytes) to perform import. At least 87293952 bytes of free memory is required 
[22/09/2016:12:11:57 +0000] category=QUICKSETUP seq=74 severity=SEVERE msg=Error: Error Creating Base Entry. 
[22/09/2016:12:11:57 +0000] category=QUICKSETUP seq=75 severity=SEVERE msg=Caught exception: Error Creating Base Entry. 
[22/09/2016:12:11:57 +0000] category=QUICKSETUP seq=76 severity=SEVERE msg=Error installing.

Recent Changes

N/A

Causes

The setup process launches a separate import-ldif process to create the baseDN and uses the default JVM parameters for this.

The JVM heap value being used is too small for the import-ldif process to complete. This can also happen if the server you are using has low memory (for example, a VM with only 1GB RAM) as the JVM automatically assigns a heap value based on the total memory available on the server.

Solution

This issue can be resolved as follows:

  1. Run ./setup without creating the baseDN entry (that is, exclude the --addBaseEntry or -a option) and include the --doNotStart option to prevent the server from starting once setup is complete, for example:
    $ ./setup --cli --acceptLicense --doNotStart --propertiesFilePath /tmp/properties --no-prompt
    An example properties file that works successfully on a low memory VM is:
    ldapPort=1389
    generateSelfSignedCertificate=true
    enableStartTLS=true
    ldapsPort=3636
    jmxPort=1689
    adminConnectorPort=4444
    rootUserDN=cn=Directory Manager
    rootUserPassword=password
    baseDN=dc=example,dc=com
  2. Run the import-ldif command offline to create the baseDN entry. You should use a command such as the following, which excludes the connection details and increases the JVM heap available to the command:
    $ export OPENDJ_JAVA_OPTS="-Xms512m -Xmx512m" && ./import-ldif --backendID userRoot --ldifFile /tmp/basedn.ldif

You should also look to increase the JVM heap size on your server. See How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache? for further information. 

See Also

How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache?

Best practice for JVM Tuning

Related Training

N/A

Related Issue Tracker IDs

N/A


OpenDJ 3.x Java upgrade causes certificate exceptions with contol-panel/dsreplication/status commands

The purpose of this article is to provide assistance if you encounter certificate and "Unable to connect to the server" errors when running dsreplication commands after upgrading to Java® 1.7.0_191, 1.8.0_181 or later. A similar "An error occurred connecting to the server" is shown when running control-panel.

Symptoms

You will encounter connecting to server errors when using dsreplication or control-panel commands:

  • dsreplication commands: running any dsreplication commands where the hostname is not a FQDN, for example:
    $ ./dsreplication status --hostname host1 --port 4444 --adminUID admin --adminPassword password --trustAll --no-prompt
    
    Gives a response similar to one of the following:
    Unable to connect to the server at host1 on port 4444. Check this port is an administration port
    
    Error reading data from server host1:4444. There is an error with the certificate presented by the server.
    Details: simple bind failed: host1:4444
    
    Whereas running the same command using a FQDN as the hostname succeeds.
  • control-panel command: login will fail with the following error when running the control-panel command:
    An error occurred connecting to the server. Details:
    javax.naming.CommunicationException: 0.0.0.0:4444 [Root exception is javax.net.ssl.SSLHandshakeException:
    java.security.cert.CertificateException: No subject alternative names present]
    

SSL debug log

An error similar to the following is shown in the SSL debug log when this happens:

LDAP Request Handler 0 for connection handler Administration Connector 192.0.2.0 port 4444, WRITE: TLSv1.2 Handshake, length = 947

LDAP Request Handler 0 for connection handler Administration Connector 192.0.2.0 port 4444, READ: TLSv1.2 Alert, length = 2

LDAP Request Handler 0 for connection handler Administration Connector 192.0.2.0 port 4444, RECV TLSv1.2 ALERT:  fatal, certificate_unknown

LDAP Request Handler 0 for connection handler Administration Connector 192.0.2.0 port 4444, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLException: Received fatal alert: certificate_unknown

LDAP Request Handler 0 for connection handler Administration Connector 192.0.2.0 port 4444, fatal: engine already closed.  Rethrowing javax.net.ssl.SSLException: Received fatal alert: certificate_unknown
Note

You can generate SSL debug logs as described in FAQ: SSL certificate management in DS/OpenDJ (Q. How do I debug a SSL handshake error?)

Recent Changes

Upgraded Java to version 1.7.0_191, 1.8.0_181 or later (including Oracle® JDK and OpenJDK).

Causes

Java 1.7.0_191 and 1.8.0_181 introduced changes to improve LDAP support by enabling endpoint identification algorithms by default for LDAPS connections.

For further information see:

 Java SE Development Kit 8, Update 181 (JDK 8u181) 

 Java SE Development Kit 7, Update 191 (JDK 7u191) 

Solution

This issue can be resolved using one of the following options:

  • Always use a FQDN for the hostname. This is a requirement for replication as stated in the Release Notes: Release Notes › FQDNs For Replication.
  • Set the new system property (com.sun.jndi.ldap.object.disableEndpointIdentification) to disable endpoint identification if appropriate for your environment. 
  • Downgrade your version of Java.

Setting the new system property

You can set this system property in OpenDJ as follows:

  1. Add the new system property to dsreplication.java-args in the java.properties file, for example:
    dsreplication.java-args=... -Dcom.sun.jndi.ldap.object.disableEndpointIdentification=true
  2. Apply this change by running the bin/dsjavaproperties command:
    $ ./dsjavaproperties

See Also

How do I change DS/OpenDJ (All versions) to use a different JDK version?

How do I ensure DS/OpenDJ (All versions) uses the Java settings from java.properties file when DS/OpenDJ is started?

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-5336 (Dsreplication and control-panel connection fails with JVM 1.8.0_181)


isMemberOf values not returned with an anonymous ldapsearch in OpenDJ 2.6.3, 2.6.4 and 3.0

The purpose of this article is to provide assistance when isMemberOf values are not returned when querying members of a group using an anonymous ldapsearch in OpenDJ 2.6.3, 2.6.4 and 3.0. This also happens if you bind using a non-Root DN.

Symptoms

Using an ldapsearch in the following situations does not return any results:

  • No bind details (anonymous):
    $ ./ldapsearch --port 4444 --baseDN dc=example,dc=com "(&(objectClass=person)(uid=jdoe)(isMemberOf=cn=internal,ou=employees,dc=example,dc=com))"
    
  • Bind details for a non-Root DN:
    $ ./ldapsearch --port 4444 --bindDN "cn=admin,ou=services,dc=example,dc=com" --bindPassword password --baseDN dc=example,dc=com "(&(objectClass=person)(uid=jdoe)(isMemberOf=cn=internal,ou=employees,dc=example,dc=com))"

Whereas binding as the Root DN will return results:

$ ./ldapsearch --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --baseDN dc=example,dc=com "(&(objectClass=person)(uid=jdoe)(isMemberOf=cn=internal,ou=employees,dc=example,dc=com))"

Recent Changes

Upgraded to, or installed OpenDJ 2.6.3, 2.6.4 or 3.0, unless you have upgraded directly from OpenDJ 2.6.0.

Causes

The isMemberOf attribute was removed from the global ACI in OpenDJ 2.6.3 and became an Operation attribute that was computed at search time. This meant it was not available in the user-visible attributes; however it is always visible to the Root DN, which is why searches using the Root DN work.

If you have upgraded directly from OpenDJ 2.6.0, anonymous searches will still return isMemberOf.

Solution

You can reinstate the isMemberOf attribute in the global ACI by upgrading to OpenDJ 3.5 or later; you can download this from BackStage.

Alternatively, you can add the isMemberOf attribute to the User-Visible Operational Attributes global ACI:

  • Default:
    ds-cfg-global-aci: (targetattr="createTimestamp||creatorsName||modifiersName||modifyTimestamp||entryDN||entryUUID||subschemaSubentry||etag||governingStructureRule||structuralObjectClass||hasSubordinates||numSubordinates")(version 3.0; acl "User-Visible Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)
    
  • After modification:
    ds-cfg-global-aci: (targetattr="createTimestamp||creatorsName||modifiersName||modifyTimestamp||entryDN||entryUUID||subschemaSubentry||etag||governingStructureRule||structuralObjectClass||hasSubordinates||numSubordinates||isMemberOf")(version 3.0; acl "User-Visible Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)

You can modify this using interactive mode or the following dsconfig command: 

$ ./dsconfig set-access-control-handler-prop --remove 'global-aci: (targetattr="createTimestamp||creatorsName||modifiersName||modifyTimestamp||entryDN||entryUUID||subschemaSubentry||etag||governingStructureRule||structuralObjectClass||hasSubordinates||numSubordinates")(version 3.0; acl "User-Visible Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)' --add 'global-aci: (targetattr="createTimestamp||creatorsName||modifiersName||modifyTimestamp||entryDN||entryUUID||subschemaSubentry||etag||governingStructureRule||structuralObjectClass||hasSubordinates||numSubordinates||isMemberOf")(version 3.0; acl "User-Visible Operational Attributes"; allow (read,search,compare) userdn="ldap:///anyone";)' --hostname localhost --port 4444 --trustAll --bindDN "cn=Directory Manager" --bindPassword password --no-prompt

See Also

How do I know what the default Global ACIs are used for in DS/OpenDJ (All versions)?

How do I only allow selected users to search, update and delete LDAP entries in DS/OpenDJ (All versions)?

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-2965 (isMemberOf searches are inefficient)


Out Of Memory Error when installing OpenDJ 3, or using import-ldif, rebuild-index or dsreplication commands

The purpose of this article is to provide assistance if you encounter an "Exception in thread "main" java.lang.OutOfMemoryError at sun.misc.Unsafe.allocateMemory(Native Method)" error. This error can occur when you use the setup command to install or upgrade, or you use the import-ldif or rebuild-index commands. Similarly, you might see an "OutOfMemoryError (Unsafe.java" error when using the dsreplication command.

Symptoms

You will see one of the following errors in your errors log depending on which command you are using:

  • setup (this includes an import-ldif command that's used during the setup process):
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=73 severity=INFO msg=import-ldif out log: [12/09/2016:08:39:51 +0100] category=PLUGGABLE seq=11 severity=INFO msg=Import LDIF environment close took 0 seconds
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=74 severity=INFO msg=import-ldif out log: [12/09/2016:08:39:51 +0100] category=PLUGGABLE seq=12 severity=INFO msg=Flushing data to disk
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=75 severity=WARNING msg=import-ldif error log: Exception in thread "main" java.lang.OutOfMemoryError
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=76 severity=WARNING msg=import-ldif error log:  at sun.misc.Unsafe.allocateMemory(Native Method)
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=77 severity=WARNING msg=import-ldif error log:  at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool$OffHeapBuffer.<init>(OnDiskMergeImporter.java:2858)
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=78 severity=WARNING msg=import-ldif error log:  at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool.<init>(OnDiskMergeImporter.java:2780)
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=79 severity=WARNING msg=import-ldif error log:  at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.importLDIF(OnDiskMergeImporter.java:203)
    [12/09/2016:08:39:51 +0100] category=QUICKSETUP seq=80 severity=WARNING msg=import-ldif error log:  at org.opends.server.backends.pluggable.BackendImpl.importLDIF(BackendImpl.java:689)
    ...
    
  • import-ldif:
    Exception in thread "main" java.lang.OutOfMemoryError 
       at sun.misc.Unsafe.allocateMemory(Native Method) 
       at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool$OffHeapBuffer.<init>(OnDiskMergeImporter.java:2858) 
       at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool.<init>(OnDiskMergeImporter.java:2780) 
       at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.importLDIF(OnDiskMergeImporter.java:203) 
       at org.opends.server.backends.pluggable.BackendImpl.importLDIF(BackendImpl.java:689) 
       at org.opends.server.tools.ImportLDIF.processLocal(ImportLDIF.java:1092) 
       at org.opends.server.tools.tasks.TaskTool.process(TaskTool.java:362) 
       at org.opends.server.tools.ImportLDIF.process(ImportLDIF.java:292) 
       at org.opends.server.tools.ImportLDIF.mainImportLDIF(ImportLDIF.java:147) 
       at org.opends.server.tools.ImportLDIF.main(ImportLDIF.java:110)
    ...
    
  • rebuild-index:
    Exception in thread "main" java.lang.OutOfMemoryError 
       at sun.misc.Unsafe.allocateMemory(Native Method) 
       at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool$OffHeapBuffer.<init>(OnDiskMergeImporter.java:2858) 
       at org.opends.server.backends.pluggable.OnDiskMergeImporter$BufferPool.<init>(OnDiskMergeImporter.java:2780) 
       at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.rebuildIndex(OnDiskMergeImporter.java:312) 
       at org.opends.server.backends.pluggable.OnDiskMergeImporter$StrategyImpl.rebuildIndex(OnDiskMergeImporter.java:275) 
       at org.opends.server.backends.pluggable.BackendImpl.rebuildBackend(BackendImpl.java:806) 
       at org.opends.server.tools.RebuildIndex.rebuildIndex(RebuildIndex.java:559) 
       at org.opends.server.tools.RebuildIndex.processLocal(RebuildIndex.java:321) 
       at org.opends.server.tools.tasks.TaskTool.process(TaskTool.java:362) 
       at org.opends.server.tools.RebuildIndex.process(RebuildIndex.java:228) 
       at org.opends.server.tools.RebuildIndex.mainRebuildIndex(RebuildIndex.java:138) 
       at org.opends.server.tools.RebuildIndex.main(RebuildIndex.java:110) 
    ...
    
  • dsreplication:
    [17/Sep/2016:08:39:51 +0100] category=org.opends.server.api.DirectoryThread severity=ERROR msgID=org.opends.messages.core.140 msg=An uncaught exception during processing for thread Replica DS(18687) listener for domain "dc=forgerock,dc=com" has caused it to terminate abnormally. The stack trace for that exception is: OutOfMemoryError (Unsafe.java:-2 OnDiskMergeImporter.java:2858 OnDiskMergeImporter.java:2780 OnDiskMergeImporter.java:203 BackendImpl.java:689 LDAPReplicationDomain.java:3565 ReplicationDomain.java:2275 ReplicationDomain.java:778 ReplicationDomain.java:106 ReplicationDomain.java:2965 Thread.java:744)
    [17/Sep/2016:08:39:51 +0100] category=CORE severity=NOTICE msgID=org.opends.messages.core.139 msg=The Directory Server has sent an alert notification generated by class org.opends.server.api.DirectoryThread (alert type org.opends.server.UncaughtException, alert ID org.opends.messages.core-140): An uncaught exception during processing for thread Replica DS(18687) listener for domain "dc=forgerock,dc=com" has caused it to terminate abnormally. The stack trace for that exception is: OutOfMemoryError (Unsafe.java:-2 OnDiskMergeImporter.java:2858 OnDiskMergeImporter.java:2780 OnDiskMergeImporter.java:203 BackendImpl.java:689 LDAPReplicationDomain.java:3565 ReplicationDomain.java:2275 ReplicationDomain.java:778 ReplicationDomain.java:106 ReplicationDomain.java:2965 Thread.java:744)
    ...
    

Recent Changes

Upgraded to, or installed OpenDJ 3.0.

Causes

Changes were made in OpenDJ 3.0.0 tools that introduced a new off-heap memory management mechanism, which mean that certain commands now request memory outside of the Java® heap using the sun.misc.Unsafe.allocateMemory() method. 

The number of allocations depend on the number of threads and the number of attribute indexes. OpenDJ calculates how many threads it uses based on the number of CPUs available to the system. It is likely this error occurs because the number of allocations being done when you are using the default number of threads exceeds the amount of memory a 32-bit JVM can allocate.

Solution

This issue can be resolved by upgrading to OpenDJ 3.5 or later which has improved/fixed memory management for imports; you can download this from BackStage.

Workaround

You can workaround this issue by switching to a 64-bit JVM as follows to allow more memory to be allocated:

  1. Add the -d64 option to the start-ds.java-args property in the java.properties file (located in the /path/to/opendj/config directory), for example:
    start-ds.java-args=-server -Xms2g -Xmx2g -d64
    
  2. Run the bin/dsjavaproperties command to apply the changes you have made to the java.properties file:
    $ ./dsjavaproperties
  3. Restart the OpenDJ server.
Note

If your heap size is less than 32GB, you should also specify CompressedOops as detailed in How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache?

If your JVM does not support 64-bit, you can try one of the following approaches instead:

  • Limit the number of CPUs being used by these commands as this will reduce the amount of memory required. This is a server level change, but for import-ldif you can limit the number of threads being used by adding the --threadCount option to your command. For example, add the following to use 8 threads:
    --threadCount 8
  • Increase the JVM heap size. See How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache? for further information.
  • Run ./setup without creating the baseDN entry (that is, exclude the --addBaseEntry or -a option) and then run an import-ldif command afterwards to create it.

See Also

OpenDJ Release Notes › OpenDJ Fixes, Limitations, and Known Issues › Key Fixes in 3.5.0

How do I tune DS/OpenDJ (All versions) process sizes: JVM heap and database cache?

How do I ensure DS/OpenDJ (All versions) uses the Java settings from java.properties file when DS/OpenDJ is started?

Best practice for JVM Tuning

OpenDJ Administration Guide › Managing Directory Data › Importing and Exporting Data

How do I collect JVM data for troubleshooting DS/OpenDJ (All versions)?

Related Training

N/A

Related Issue Tracker IDs

OPENDJ-3212 (When we are upgrading Opendj from 2.4.4 to 3.0.0 version, java.lang.OutOfMemoryError occured)

OPENDJ-2721 (JE is using all the available heap memory during import.)


Copyright and TrademarksCopyright © 2018 - 2019 ForgeRock, all rights reserved.

This content has been optimized for printing.

Loading...