Data stores in AM/OpenAM

This book provides information on the different types of data stores (repositories) in AM/OpenAM, including the configuration store, the user data store (identity repository) and the CTS store.


How do I check if AM/OpenAM (All versions) is up and running?

The purpose of this article is to provide information on checking if AM/OpenAM is up and running; this information applies to standalone servers and servers running in site behind a load balancer. This can include checking connections from AM/OpenAM to the configuration store and user data store (also known as the identity repository, which is the data store that contains user profile information).

Checking if AM/OpenAM is up and running

You can perform a couple of checks to see if AM/OpenAM is up and running, depending on your requirements.

AM/OpenAM server and configuration store

You can send a request to the isAlive.jsp endpoint; this request can either be sent to the server URL or the load balancer URL, for example: 

http://host1.example.com:8080/openam/isAlive.jsp
http://lb.example.com:8080/openam/isAlive.jsp

If you receive a HTTP 200 OK response, the status page will return Server is ALIVE: at this point you have verified AM/OpenAM is up and running, and can communicate with its configuration store; otherwise, this currently throws an exception, which causes the web server to return an error code of 500. If AM/OpenAM is not running you will receive a message saying the browser could not connect.

If AM/OpenAM is running and the directory server used for the configuration store is not reachable, the status page will return Server is DOWN.

User data store

You can perform a login operation using a REST call followed by a logout operation to remove the created session. If you can authenticate successfully, you know that your user store is reachable; this also means that AM/OpenAM is up and running, so you can use this authentication check instead of sending a request to the isAlive.jsp endpoint for a more comprehensive check.

Note

The core authentication settings for the realm you are authenticating against must have been set to Profile Required, and you must have an authentication module in your authentication chain that utilizes username and password authentication to be able to use this approach. 

Alternatively, you could perform a login operation using invalid credentials. The failed authentication due to invalid credentials verifies AM/OpenAM and the user data store are communicating and functioning correctly, but means that a logout operation to destroy the created session is not needed.

See Development Guide › Developing with the REST API › Authentication and Logout for further information on using REST calls to authenticate.

See Also

How do I understand what the user data store is used for in AM/OpenAM (All versions)?

How do I check that a Policy Agent (All versions) can connect to AM/OpenAM?

How do I perform a heartbeat check against DS/OpenDJ (All versions)?

How do I check if a backend is online in DS/OpenDJ (All versions)?

How do I diagnose a hung AM/OpenAM (All versions) server?

How do I check if IG/OpenIG (All versions) is up and running?

Setup and Maintenance Guide › Maintaining an Instance › Monitoring Services

Development Guide › Developing with the REST API

Related Training

ForgeRock Access Management Core Concepts (AM-400)

Related Issue Tracker IDs

N/A


Configuration Store


How do I migrate from an embedded to external DS/OpenDJ in AM/OpenAM (All versions)?

The purpose of this article is to provide information on migrating from an embedded to external DS/OpenDJ for the configuration data store in AM/OpenAM. If you plan on using an external configuration store, it is recommended that you start with the external configuration store rather than migrating to it after you have gone live. This information can also be used to migrate from one external instance to another to change the location of the configuration store. Both the old and new configuration stores must have been set up with the same base DN and have the same password.

Migrating configuration data stores

Note

You should ensure you have up to date backups (including a backup of the bootstrap file) prior to migrating configuration data stores in case you encounter any issues. See How do I make a backup of configuration data in AM/OpenAM (All versions)? for further information.

You can migrate configuration data stores as follows:

  1. Navigate as follows: 
    • AM / OpenAM 13.5 console: navigate to Deployment > Servers > [Server Name] > Advanced
    • Pre-OpenAM 13.5 console: navigate to Configuration > Servers and Sites > [Server Name] > Advanced
    And update the value of the com.sun.identity.sm.sms_object_class_name advanced property to:
    com.sun.identity.sm.ldap.SMSLdapObject
    This change allows multiple AM/OpenAM servers to communicate with the external DS/OpenDJ configuration store for all types of LDAP operations.
  2. Add a new external configuration server by navigating as follows:
    • AM / OpenAM 13.5 console: navigate to Deployment > Servers > [Server Name] > Directory Configuration > Server. Enter details for the new external configuration server, click + to add followed by Save Changes.
    • Pre-OpenAM 13.5 console: navigate to Configuration > Servers and Sites > [Server Name] > Directory Configuration > Server and click Add. Enter details for the new external configuration server and click OK to save.
    The bootstrap file is automatically updated with the new directory configuration details. 
Note

Port number 1389 is typically used for Simple connections and 1636 for SSL connections. Attempting to use 1636 for non-SSL connections can cause errors. 

  1. Delete the old configuration server from the console; this step only removes the configuration store from AM/OpenAM; it does not actually delete it. Again, the bootstrap file is automatically updated to remove the deleted directory configuration details.
  2. Stop the web application container in which AM/OpenAM runs.
  3. Stop the DS/OpenDJ server that hosts the old configuration store if it is external; if it is embedded, it will have stopped when you stopped AM/OpenAM. 
  4. Export data from the old configuration store server to an LDIF file using the following command depending on your versions:
    • AM 5 and later / DS 5 and later (needs the --offline option when the server is stopped):
      $ ./export-ldif --offline --includeBranch dc=example,dc=com --backendID userRoot --ldifFile /path/to/exportfile.ldif
    • OpenAM 13.5.x and earlier / OpenDJ 3.5.x and earlier:
      $ ./export-ldif --includeBranch dc=example,dc=com --backendID userRoot --ldifFile /path/to/exportfile.ldif
  5. Stop the DS/OpenDJ server that hosts the new configuration store.
  6. Take a backup of the config.ldif file for the new configuration store server (located in the /path/to/ds/config directory).
  7. Edit two properties within this file so they are set as follows to prepare for importing LDIF data:
    ds-cfg-single-structural-objectclass-behavior: warn
    ds-cfg-allow-pre-encoded-passwords: true
    
  8. Copy the 99-user.ldif file from the old configuration store server to the new one. This file contains custom schema definitions and is located in /config/schema; as of DS 6 and later, located in /db/schema for new installs.
  9. Import data into the new configuration store server from the LDIF file you created in step 6 using the following command depending on your versions:
    • AM 5 and later / DS 5 and later (needs the --offline option when the server is stopped):
      $ ./import-ldif --offline --includeBranch dc=example,dc=com --backendID userRoot --ldifFile /path/to/exportfile.ldif
    • OpenAM 13.5.x and earlier / OpenDJ 3.5.x and earlier:
      $ ./import-ldif --includeBranch dc=example,dc=com --backendID userRoot --ldifFile /path/to/exportfile.ldif
Note

Both the old and new configuration stores must have the same base DN, otherwise the import will fail. 

  1. Start the DS/OpenDJ server that hosts the new configuration store.
  2. Start the web application container in which AM/OpenAM runs.
  3. Take a backup of the config.ldif file for the new configuration store server (located in the /path/to/ds/config directory).
  4. Edit one of the properties within this file that you changed previously so it is set as follows for improved security:
    ds-cfg-allow-pre-encoded-passwords: false

See Also

How do I remove the embedded DS/OpenDJ (All versions) after migrating to an external instance?

How do I add a second configuration store or edit an existing configuration store in AM/OpenAM (All versions)?

Data stores in AM/OpenAM

Reference › Configuration Reference › Directory Configuration Properties

Administration Guide › Managing Directory Data › Importing and Exporting Data

Reference › Tools Reference › export-ldif

Reference › Tools Reference › import-ldif​

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I add a second configuration store or edit an existing configuration store in AM/OpenAM (All versions)?

The purpose of this article is to provide information on adding a second configuration store to AM/OpenAM. You may want to do this in a high availability (HA) environment where you have multiple servers in a site. This information can also be used to update an existing directory configuration and includes a way of doing it via ssoadm.

Maintaining configuration store details

Note

In OpenAM 11.x and 12.x, you can only have two configuration stores as they use the legacy Netscape® LDAP SDK. If you have more than two configuration stores listed, OpenAM will only use the first two in the list.

You can add a second configuration store or update details for an existing configuration store using either the console or ssoadm:

Console

  1. Navigate as follows:
    • AM / OpenAM 13.5 console: navigate to: Deployment > Servers > [Server Name] > Directory Configuration > Server, and add a new server or edit the existing server details as needed.
    • Pre-OpenAM 13.5 console: navigate to: Configuration > Servers and Sites > [Server Name] > Directory Configuration > Server, and add a new server or edit the existing server details as needed.
  2. Restart the web application container in which AM/OpenAM runs.

ssoadm 

  1. Output the current server configuration details to an XML file using the following command:
    $ ./ssoadm get-svrcfg-xml -u [adminID] -f [passwordfile] -s [serverName] -o [XMLfile]
    
    replacing [adminID], [passwordfile], [serverName] and [XMLfile] with appropriate values.
  2. Locate the following section in the XML file you output in step 1 (ServerGroup name="sms"):
    <ServerGroup name="sms" minConnPool="1" maxConnPool="10">
        <Server name="Server1" host="localhost" port="18080" type="SIMPLE" />
    
  3. Update these server details if you want to make amendments; otherwise add a new line if you want to add a second configuration store. For example:
    <ServerGroup name="sms" minConnPool="1" maxConnPool="10">
        <Server name="Server1" host="localhost" port="18080" type="SIMPLE" />
        <Server name="Server2" host="localhost" port="28080" type="SIMPLE" />
    
  4. Upload the updated server configuration XML file to AM/OpenAM using the following command:
    $ ./ssoadm set-svrcfg-xml -u [adminID] -f [passwordfile] -s [serverName] -X [XMLfile]
    replacing [adminID], [passwordfile], [serverName] and [XMLfile] with appropriate values.
  5. Restart the web application container in which AM/OpenAM runs.

See Also

How do I change the password for the configuration store in AM/OpenAM (All versions)?

How do I migrate from an embedded to external DS/OpenDJ in AM/OpenAM (All versions)?

Reference › Command Line Tools › ssoadm

Related Training

N/A

Related Issue Tracker IDs

OPENAM-8738 (Need more scriptable method to change Config Stores "Directory Configuration" than xml files)


How do I change the password for the configuration store in AM/OpenAM (All versions)?

The purpose of this article is to provide information on changing the password for the configuration store in AM/OpenAM. This article also covers changing the configuration store password in a high availability (HA) environment where you have multiple servers in a site.

Changing the configuration store password

As of AM 5, the password for the configuration store is held in the configstorepwd alias in the keystore. Each time you update the configuration store password, AM also modifies the content of the configstorepwd alias in the keystore.jceks keystore file. See Setup and Maintenance Guide › Setting Up Keys and Keystores › Configuring Password String Aliases for further information.

You can change the configuration store password using either the console or ssoadm:

Note

If you use embedded DS/OpenDJ your configuration store password and amadmin password should match. See How do I change the amadmin password in AM/OpenAM (All versions)? for further information on changing the amadmin password.

Console

  1. Update the configuration store (directory bind) password on all AM/OpenAM servers using the console:
    • AM / OpenAM 13.5 console: navigate to: Deployment > Servers > [Server Name] > Directory Configuration > Bind Password and enter your new password.​
    • Pre-OpenAM 13.5 console: navigate to: Configuration > Servers and Sites > [Server Name] > Directory Configuration > Bind Password and enter your new password.​
    This configuration is server specific and must be changed on all servers in your site.
  2. Update the password for cn=Directory Manager on all directory servers to match your new password. This configuration is server specific and must be changed on all directory servers. If you use DS/OpenDJ, refer to Administration Guide › Troubleshooting Server Problems › Resetting Administrator Passwords for further information. If you use the embedded DS/OpenDJ, you must restart the web application container in which AM/OpenAM runs to restart the DS/OpenDJ server.

ssoadm 

  1. Export the current server configuration details to a XML file using the following command:
    $ ./ssoadm get-svrcfg-xml -u [adminID] -f [passwordfile] -s [serverName] -o [XMLfile]
    
    replacing [adminID], [passwordfile], [serverName] and [XMLfile] with appropriate values.
  2. Encrypt your new configuration store password using ampassword:
    $ ./ampassword -e [passwordfile]
    
    replacing [passwordfile] with an appropriate value.
  3. Edit the XML file you exported in step 1 to replace the <DirPassword> value in the sms section with the encrypted password you generated in step 2:
        <ServerGroup name="sms" minConnPool="1" maxConnPool="10">
            <Server name="Server1" host="host1.example.com" port="8080"
                 type="SIMPLE" />
                 <User name="User2" type="admin">
                    <DirDN>cn=Directory Manager</DirDN>
                    <DirPassword>AQICUsKIqPqiF0SPBdLZ99beokClGjSB0vuR</DirPassword>
                 </User>
                 <BaseDN>dc=example,dc=com</BaseDN>
        </ServerGroup>
    
  4. Import the updated server configuration XML file to AM/OpenAM using the following command:
    $ ./ssoadm set-svrcfg-xml -u [adminID] -f [passwordfile] -s [serverName] -X [XMLfile]
    replacing [adminID], [passwordfile], [serverName] and [XMLfile] with appropriate values.
  5. Repeat steps 1 to 4 on on all AM/OpenAM servers if you have multiple servers in a site; this configuration is server specific and must be changed on all servers in your site.
  6. Update the password for cn=Directory Manager on all directory servers to match your new password. This configuration is server specific and must be changed on all directory servers. If you use DS/OpenDJ, refer to Administration Guide › Troubleshooting Server Problems › Resetting Administrator Passwords for further information. To restart the embedded DS/OpenDJ server, you must restart the web application container in which AM/OpenAM runs.
  1. Restart the web application container in which AM/OpenAM runs to update the configuration.

Example using ssoadm

The following example changes the configuration store password from cangetinam to newPassw0rd on a single server using ssoadm:

  1. Export the current server configuration details to a XML file:
    $ ./ssoadm get-svrcfg-xml -u amadmin -f pwd.txt -s http://host1.example.com:8080/openam -o serverConfig.xml
  2. Create a text file containing your new password:
    $ echo newPassw0rd > password.txt
  3. Encrypt your new configuration store password using ampassword:
    $ ./ampassword -e password.txt
    Output:
    AQIC61K+PS/+xIv3c4Y4Wxwb66cCcCYeKa/9
    
  4. Edit the serverConfig.xml file you exported in step 1 to replace the <DirPassword> value in the sms section with the encrypted password you generated in step 3:
        <ServerGroup name="sms" minConnPool="1" maxConnPool="10">
            <Server name="Server1" host="host1.example.com" port="8080"
                 type="SIMPLE" />
                 <User name="User2" type="admin">
                    <DirDN>cn=Directory Manager</DirDN>
                    <DirPassword>AQIC61K+PS/+xIv3c4Y4Wxwb66cCcCYeKa/9</DirPassword>
                 </User>
                 <BaseDN>dc=example,dc=com</BaseDN>
        </ServerGroup>
    
  5. Import the updated serverConfig.xml file to AM/OpenAM:
    $ ./ssoadm set-svrcfg-xml -u amadmin -f pwd.txt -s http://host1.example.com:8080/openam -X serverConfig.xml
  6. Update the password for cn=Directory Manager on all directory servers to newPassw0rd.
  7. Restart the web application container in which AM/OpenAM runs.

See Also

How do I change the amadmin password in AM/OpenAM (All versions)?

How do I change the dsameuser password in AM/OpenAM (All versions)?

How do I add a second configuration store or edit an existing configuration store in AM/OpenAM (All versions)?

Setup and Maintenance Guide › Setting Up Keys and Keystores › Configuring Password String Aliases 

Reference › Command Line Tools › ssoadm

Reference › Configuration Reference › Directory Configuration Properties

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I find network errors in the logs for the configuration store in AM/OpenAM (All versions)?

The purpose of this article is to provide information on finding network errors in the logs for the configuration store in AM/OpenAM. This is useful if you suspect network errors are causing connection issues with the configuration store.

Finding network errors

The Configuration debug log shows network errors when the debug level is set to Warning or Message (default is Error). You can set this debug level for all logs as described in How do I enable Message level debugging in AM/OpenAM (All versions) debug files? or you can just set it for the amEventService instance as follows: 

  1. Log into AM/OpenAM as the admin user (called amadmin by default).
  2. Navigate to: <protocol>://host.fqdn:port/openam/Debug.jsp, for example: http://host1.example.com:8080/openam/Debug.jsp.
  3. Select amEventService from the Debug Instances field.
  4. Select Warning or Message from the Level field and click Submit to change the debug level.
Note

The Debug.jsp page always shows the debug level as Error, regardless of its actual setting.

If you have network based errors, you will see errors such as the following in your Configuration log (located in the $HOME/[am_instance]/openam/debug directory by default):

amEventService
WARNING: EventService.run() LDAPException received: 
com.sun.identity.ldap.LDAPException: Server or Network error (81)
   at com.sun.identity.shared.ldap.LDAPConnThread.networkError(LDAPConnThread.java:910)
   at com sun.identity.shared.ldap.LDAPConnThread.networkError(LDAPConnThread.java:895)
   at com.sun.identity.shared.ldap.LDAPConnThread.run(LDAPConnThread.java:676)
   at java.lang.Thread.run(Thread.java:745)
...

Once you have reproduced the problem and captured the debug logs, you should revert the debug level to Error to avoid filling up the disks where the debug logs are stored.

See Also

Attempting to access AM/OpenAM (All versions) fails with ConfigurationException: Configuration store is not available

How do I check if AM/OpenAM (All versions) is up and running?

How do I enable Message level debugging in AM/OpenAM (All versions) debug files?

How do I clear debug logs in AM/OpenAM (All versions)?

How do I record troubleshooting information in AM/OpenAM (All versions)?

Best practice for recording troubleshooting information in AM (All versions) and OpenAM 13.x

Related Training

N/A

Related Issue Tracker IDs

N/A


Known Issues


Default Configuration page shown instead of Login page in AM/OpenAM (All versions)

The purpose of this article is to provide assistance if the Default Configuration page is shown instead of the Login page when attempting to log into AM/OpenAM. The Default Configuration page is normally displayed when installing AM/OpenAM.

Symptoms

When logging into AM/OpenAM using the usual login URL, the Default Configuration page is displayed instead of the Login page, even though AM/OpenAM has already been configured and previously functioned correctly.

The configuration appears to be lost.

Recent Changes

Restarted AM/OpenAM.

Causes

AM/OpenAM attempts to find its configuration when bootstrapping by looking at the path specified in the bootstrap locator file (located in the $HOME/.openamcfg/ directory of the user running the web application container).

This issue occurs when AM/OpenAM fails to find the configuration when bootstrapping, which can be caused by one of the following:

  • Configuration store not running.
  • Missing $HOME/.openamcfg/ directory; the file used to bootstrap (the bootstrap locator file) is located in this directory.
  • Redeploying openam.war to a different location; the bootstrap locator file is named according to where AM/OpenAM is deployed.
  • Starting AM/OpenAM as a different user to the one who initially configured it; the $HOME/.openamcfg/ directory is user specific.

Solution

The solution depends on the cause; you should check the following to establish the cause and then rectify accordingly:

Configuration store

  • Check that the configuration store is up and running, and that AM/OpenAM can successfully communicate with it: 
    • You can check this by running the OpenDJ status command (located in /path/to/ds/bin) or if using the embedded Configuration store (located in the $HOME/[am_instance]/opends/bin directory). The server status returned to you should be: Server Run Status: Started.
    • You can also send a request to the isAlive.jsp endpoint. If AM/OpenAM is running and the directory server used for the configuration store is not reachable, the page will return Server is DOWN as described in How do I check if AM/OpenAM (All versions) is up and running?

$HOME/.openamcfg/ directory

  • Check the $HOME/.openamcfg/ directory exists for the process owner and contains the bootstrap locator file (AMConfig). This is a file called AMConfig_[path_to_openam], for example, AMConfig_opt_tomcat_webapps_openam if /opt/tomcat/webapps/openam is the deployment path of AM/OpenAM. 

Deployment path

Process owner

  • Check the process owner; if AM/OpenAM is installed on a Linux® or Unix® system and is deployed on an Apache Tomcat™ web container, you can check the process owner using the following command:
    ps -ef |grep tomcat
    
    The process owner must be same as the user who restarted AM/OpenAM.

Configuration directory

  • Check that the AMConfig file contains the path to a valid configuration directory, which is $HOME/openam by default.
  • Check the configuration directory referenced in the AMConfig file contains a file called bootstrap or boot.json (depending on your version of AM/OpenAM); check the contents of the bootstrap or boot.json file to ensure the configuration store details it contains are correct. Example configuration store details look like this:
    • boot.json (AM 5 and later):
        "configStoreList" : [ {
          "baseDN" : "dc=openam,dc=forgerock,dc=org",
          "dirManagerDN" : "cn=Directory Manager",
          "ldapHost" : "localhost",
          "ldapPort" : 50389,
          "ldapProtocol" : "ldap"
        } ]
      
      
    • bootstrap (Pre-AM 5):
      ldap://localhost:50389/http%3A%2F%2Fhost1.example.com%3A8080%2Fopenam?user=cn%3Ddsameuser%2Cou%3DDSAME+Users%2Cdc%3Dopenam%2Cdc%3Dforgerock%2Cdc%3Dorg&pwd=AQIC5wM2LY4z9R5f8R&dsbasedn=dc%3Dopenam%2Cdc%3Dforgerock%2Cdc%3Dorg&dsmgr=cn%3DDirectory+Manager&dspwd=AQIC5wM2LY4z9R5f8R&ver=1.0
      

See Also

How do I enable message level debugging for install and upgrade issues with AM/OpenAM (All versions) ?

AM 5 Installation Guide › Preparing for Installation › Preparing for JBoss and WildFly

OpenAM 13 Installation Guide › Preparing For Installation › Preparing OpenAM for JBoss and WildFly

OpenAM 12 Installation Guide › Preparing For Installation › Preparing OpenAM & JBoss AS 7 / EAP 6

Related Training

ForgeRock Access Management Core Concepts (AM-400)

Related Issue Tracker IDs

N/A


Attempting to access AM/OpenAM (All versions) fails with ConfigurationException: Configuration store is not available

The purpose of this article is to provide assistance if accessing AM/OpenAM results in an HTTP Status 500 - "ConfigurationException: Configuration store is not available".

Symptoms

The following message is shown in the browser when you try to access the console:

HTTP Status 500 - AMSetupFilter.doFilter

type Exception report

message AMSetupFilter.doFilter

description The server encountered an internal error that prevented it from fulfilling this request.

exception 

javax.servlet.ServletException: AMSetupFilter.doFilter
	com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:136)

root cause

com.sun.identity.common.configuration.ConfigurationException: Configuration store is not available.
	com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:109)

note The full stack trace of the root cause is available in the Apache Tomcat/7.0.35 logs.

The following error is shown in the web application container log (for example, catalina.out for Apache Tomcat™):

com.sun.identity.common.configuration.ConfigurationException: Configuration store is not available.
	at com.sun.identity.setup.AMSetupFilter.doFilter(AMSetupFilter.java:109)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
	at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:931)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
	at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
	at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
	at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)

Recent Changes

Restarted AM/OpenAM.

Made configuration changes.

Installed or upgraded to a different version of AM/OpenAM.

Changed the cn=Directory Manager password on the directory server but not in AM/OpenAM. 

Made environmental changes, such as networking, firewall or operating system changes.

Deleted a previously configured server and then re-added it to a site.

Changed debug level to message in OpenAM 13.0.

Causes

This error indicates that AM/OpenAM is unable to communicate with its configuration store. This can occur for a number of reasons, including:

  • The configuration store is not running or AM/OpenAM cannot access the configuration store due to a network failure, or a network issue such as a firewall preventing access. This can also happen in OpenAM 13.0 where the EmbeddedDJ debug log becomes too full and prevents OpenAM from writing to the configuration store. This is a known issue, which is fixed in OpenAM 13.5: OPENAM-8696 (OpenAM 13 EmbeddedDJ log spam )  
  • The bootstrap file is empty (0 bytes) or corrupt. The bootstrap file can be empty if the disk was out of space when the AM/OpenAM server was started.
  • The $HOME/.openamcfg/ directory is missing for the current user; the file used to bootstrap (the bootstrap locator file) is located in this directory for the process owner. If you're not the process owner, this directory will not exist.
  • The cn=Directory Manager password has been changed on the directory server but not in AM/OpenAM. AM/OpenAM uses this password to connect to the configuration store.
  • The server that was deleted and re-added to a site has lost all its configuration and has been given the default server settings again.
  • The admin-backend.ldif file (located in the $HOME/[am_instance]/ opends/config directory) is empty (only applicable if you are running a pre-2.6.3 version of OpenDJ). The most common cause for this is the server running out of disk space.

Solution

The solution depends on the cause; you should check the following to establish the cause and then rectify accordingly:

Configuration store

  • Check that the configuration store is up and running, and that AM/OpenAM can successfully communicate with it: 
    • You can check this by running the OpenDJ status command (located in /path/to/ds/bin) or if using the embedded Configuration store (located in the $HOME/[am_instance]/opends/bin directory). The server status returned to you should be: Server Run Status: Started.
    • You can also send a request to the isAlive.jsp endpoint. If AM/OpenAM is running and the directory server used for the configuration store is not reachable, the page will return Server is DOWN as described in How do I check if AM/OpenAM (All versions) is up and running?

If you are using OpenAM 13.0 and have a large EmbeddedDJ debug log, you should clear the log file to allow OpenAM to write to the configuration store again.

Bootstrap file

$HOME/.openamcfg/ directory

  • Check the $HOME/.openamcfg/ directory exists for the process owner and contains the bootstrap locator file (AMConfig). This is a file called AMConfig_[path_to_openam], for example, AMConfig_opt_tomcat_webapps_openam if /opt/tomcat/webapps/openam is the deployment path of AM/OpenAM. 

Process owner

  • Check the process owner; if AM/OpenAM is installed on a Linux® or Unix® system and is deployed on an Apache Tomcat™ web container, you can check the process owner using the following command:
    ps -ef |grep tomcat
    
    The process owner must be same as the user who restarted AM/OpenAM.

Configuration directory

  • Check that the AMConfig file contains the path to a valid configuration directory, which is $HOME/openam by default.
  • Check the configuration directory referenced in the AMConfig file contains a file called bootstrap; check the contents of the bootstrap file to ensure the configuration store details it contains are correct. Example configuration store details look like this:
    ldap://localhost:50389/http%3A%2F%2Fhost1.example.com%3A8080%2Fopenam?user=cn%3Ddsameuser%2Cou%3DDSAME+Users%2Cdc%3Dopenam%2Cdc%3Dforgerock%2Cdc%3Dorg&pwd=AQIC5wM2LY4z9R5f8R&dsbasedn=dc%3Dopenam%2Cdc%3Dforgerock%2Cdc%3Dorg&dsmgr=cn%3DDirectory+Manager&dspwd=AQIC5wM2LY4z9R5f8R&ver=1.0
    

cn=Directory Manager password

Revert the cn=Directory Manager password to its old value on the directory server to allow access to AM/OpenAM again.

You can then update it as follows:

  1. Update the directory bind (configuration store) password on all AM/OpenAM servers using the console (see How do I change the password for the configuration store in AM/OpenAM (All versions)? for details on doing this via ssoadm):
    • AM / OpenAM 13.5 console: navigate to: Deployment > Servers > [Server Name] > Directory Configuration > Bind Password and enter your new amadmin password.​
    • Pre-OpenAM 13.5 console: navigate to: Configuration > Servers and Sites > [Server Name] > Directory Configuration > Bind Password and enter your new password.​
    This configuration is server specific and must be changed on all servers in your site.
  2. Update the password for cn=Directory Manager on all directory servers to match your new password. This configuration is server specific and must be changed on all directory servers. If you use OpenDJ, refer to  Administration Guide › Troubleshooting Server Problems › Resetting Administrator Passwords for further information. If you use the embedded DS/OpenDJ, you must restart the web application container in which AM/OpenAM runs to restart the DS/OpenDJ server.

Server configuration

Note

When you reconfigure the server, you might need to change the inheritance settings to prevent default server settings being applied again. In AM / OpenAM 13.5, you can change these settings within the server configuration; a closed lock means the property is inherited from the defaults. To change an inherited value, click the lock to unlock it; you can now enter a server specific value. In OpenAM 13.0 and earlier,  you must navigate to Configuration > Servers and Sites > [Server Name] > Inheritance Settings and deselect any properties that you want to override; you can now enter server specific values for these properties when updating the server configuration.

The server that was deleted and re-added to a site needs to be reconfigured as follows:

  1. Stop replication on this server if it is enabled:
    • DS 5 and later (AM 5 and later):
      $ ./dsreplication unconfigure --unconfigureAll --port 4444 --hostname ds1.example.com --bindDN "cn=Directory Manager" --adminPassword password --trustAll --no-prompt
    • Pre-DS 5 (pre-AM 5):
      $ ./dsreplication disable --disableAll --port 4444 --hostname ds1.example.com --bindDN "cn=Directory Manager" --adminPassword password --trustAll --no-prompt
  2. Reconfigure the server from a working server:
    • AM / OpenAM 13.5 console: navigate to: Deployment > Servers > [Server Name]
    • Pre-OpenAM 13.5 console: navigate to: Configuration > Servers and Sites > [Server Name]
  3. In particular, you should check/change the following properties:
    • General > System: Base installation directory.
    • Security > Encryption: Password Encryption Key and the Authentication Service Share Secret.
    • Directory Configuration: all options.
    • Advanced: all options.
  4. Re-enable replication on this server if needed:
    • DS 5 and later (AM 5 and later):
      $ ./dsreplication configure --adminUid admin --adminPassword password --baseDn dc=example,dc=com --host1 ds2.example.com --port1 4444 --bindDn1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 8989 --host2 ds1.example.com --port2 4444 --bindDn2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 8989 --trustAll --no-prompt
    • Pre-DS 5 (pre-AM 5):
      $ ./dsreplication enable --adminUID admin --adminPassword password --baseDN dc=example,dc=com --host1 ds2.example.com --port1 4444 --bindDN1 "cn=Directory Manager" --bindPassword1 password --replicationPort1 8989 --host2 ds1.example.com --port2 4444 --bindDN2 "cn=Directory Manager" --bindPassword2 password --replicationPort2 8989 --trustAll --no-prompt
  5. Restart the web application container in which AM/OpenAM runs.

Admin-backend.ldif file

See Also

How do I re-create a bootstrap file for OpenAM 12.x and 13.x if the bootstrap file has become corrupt?

Default Configuration page shown instead of Login page in AM/OpenAM (All versions)

How do I change the amadmin password in AM/OpenAM (All versions)?

How do I change the password for the configuration store in AM/OpenAM (All versions)?

Reference › Configuration Reference › Configuring Servers

Related Training

N/A

Related Issue Tracker IDs

OPENAM-8696 (OpenAM 13 EmbeddedDJ log spam )


Login to AM/OpenAM console (All versions) fails for amadmin user

The purpose of this article is to provide assistance if login to the AM/OpenAM console fails for the amadmin user. The amadmin user is stored in the configuration data store rather than the user data store.

Symptoms

The amadmin user cannot log into the console. You may see one of the following errors when you try to log in depending on the root cause:

Browser

One of the following error(s) is shown in the browser, even though your credentials are correct:

User name/password combination is invalid.
Login/password combination is invalid.
Authentication service is not initialized. Contact your system administrator.
Unable to login to OpenAM
Service Unavailable

The service is currently unavailable. It may be temporarily overloaded or under going maintenance. Please try again later.

Authentication debug log

The following error is shown in your Authentication debug log; no error is shown in the browser:

amAuth:12/15/2016 09:27:03:209 AM UTC: Thread[http-nio-8080-exec-8,5,main] 
LoginState: getting identity Got IdRepException in IdUtils.getIdentity 
Message:Illegal universal identifier amAdmin.

IdRepo debug log

The following error is shown in your IdRepo debug log; the "Authentication service is not initialized" error may also be seen in the browser:

amSDK:12/15/2016 09:35:51:166 AM UTC: Thread[localhost-startStop-1,5,main]: TransactionId[b3fea1eb-aaad-4d20-9b66-941b90e78ad8-2]
ERROR: JCEEncryption:: failed to decrypt data
javax.crypto.BadPaddingException: Given final block not properly padded

Session debug log

The following error is shown in your Session debug log and an authentication failed error may also be seen in the browser:

CTS: Operation failed:
Result Code: Object Class Violation
Diagnostic Message: Entry coreTokenId=-3581527715699050299,ou=famrecords,ou=openam-session,ou=tokens,dc=example,dc=com violates the Directory Server schema configuration because it includes attribute coreTokenMultiString01 which is not allowed by any of the objectclasses defined in that entry
Matched DN: 
   at org.forgerock.openam.cts.impl.LdapAdapter.create(LdapAdapter.java:110)
   at org.forgerock.openam.sm.datalayer.impl.tasks.CreateTask.performTask(CreateTask.java:48)
   at org.forgerock.openam.sm.datalayer.api.AbstractTask.execute(AbstractTask.java:41)
   at org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutor$AuditRequestContextPropagatingTask.execute(SeriesTaskExecutor.java:209)
   at org.forgerock.openam.sm.datalayer.impl.SimpleTaskExecutor.execute(SimpleTaskExecutor.java:59)
   at org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutorThread.run(SeriesTaskExecutorThread.java:85)

Recent Changes

Upgraded to, or installed AM 5 or later.

Created a new authentication chain and set it as the default for the top level realm.

Enabled secure cookies.

Changed the federation signing key.

Changed the default cookie domain in AM / OpenAM 13.5 or upgraded to AM / OpenAM 13.5

Enabled host-based cookies in OpenAM 12.0.0, 12.0.1, 12.0.2 or 13.0.

Upgraded to OpenAM 12.0.4 or 13.0. 

Causes

The cause varies depending on what recent changes you have made:

AM 5 and later

When installing or upgrading to AM 5 or later from a previous version, missing schemas for the external CTS will prevent you from logging in (and you will see the CTS: Operation failed error in the Session debug log).

Additionally, the Core Token Service (CTS) token store is no longer optional as of AM 5 as it is now the authoritative source for sessions. See Release Notes › What's New › Cloud for further information on this change. As a consequence of this change, the amadmin user will not be able to log into the console if the CTS store is down or misconfigured. This is a known issue: OPENAM-10382 (If we set up External CTS wrongly then OpenAM fails catastrophically on start up.); the misconfiguration aspect will be addressed in a future release per OPENAM-10383 (Validation of External CTS store using OpenAM GUI)

Authentication chain

If you have created a new authentication chain, set it as the default for the top level realm and use a login URL such as:

http://host1.example.com:8080/openam/XUI/#login

The amadmin user cannot log in as this URL format (openam/XUI/#login) directs them to the authentication chain specified in the Organization Authentication Configuration setting. The amadmin user needs to log in via the DataStore module. See Authentication and Single Sign-On Guide › Configuring the Default Authentication Tree or Chain for further information.

Secure cookies

If you have enabled AM/OpenAM to set cookies in secure mode, the browser will only return the session cookie if a secure protocol such as HTTPS is used; the cookie will not be returned over non-SSL connections. Although you have successfully authenticated, the cookie containing the session token is not passed to AM/OpenAM, which prevents login from succeeding. 

Federation signing key

The default 'test' certificate alias used for SAML2 and OAuth signing keys is also used by the XUI and for REST authentication. If you change the signing key for federation but do not change the certificate alias used for authentication, you will not be able to log into the console with XUI enabled or make REST calls.

Cookie domain

As of OpenAM 13.5, the cookie domain defaults to the full FQDN. Login will not succeed unless the cookie domain is set correctly.

See FAQ: Cookies in AM/OpenAM (Q. What does the cookie domain default to?) for further information about this change.

Host-based cookies

There is a known issue in OpenAM 12.0.0, 12.0.1, 12.0.2 and 13.0 where the cookie domains property is incorrectly set when host-based cookies are used in the XUI. The empty cookie domains server property prevents the session cookie being set, which prevents the login succeeding: OPENAM-5264 (Can't login to OpenAM with no cookies set in the platform service).

Encryption key

There is a known issue in OpenAM 12.0.4 and 13.0 that causes the "ERROR: JCEEncryption:: failed to decrypt data" message in the IdRepo debug log. See JCEEncryption:: failed to decrypt data error when accessing the admin console in OpenAM 12.x or 13.x for further information on this cause.

Additionally, this error can happen in other versions of AM/OpenAM when the encryption key is corrupt, which means the directory manager's password (which is stored in the bootstrap file) cannot be decrypted. The encryption key is specified in the AM_ENC_KEY property when you configure AM/OpenAM.

Solution

The solution depends on the cause:

AM 5 and later

Authentication chain

You should force the amadmin user to log in to the console (/openam/console) using one of the following URLs:

  • Use the /openam/console URL, for example:
    http://host1.example.com:8080/openam/console
  • Append the login URL with the DataStore module, for example:
    http://host1.example.com:8080/openam/XUI/#login/&module=DataStore
  • Append the login URL with the adminconsoleservice service, for example:
    http://host1.example.com:8080/openam/XUI/#login/&service=adminconsoleservice 
    The adminconsoleservice service uses the authentication chain defined for the administrator (Administrator Authentication Configuration).

If you changed the default authentication chain in the top level realm by mistake, you can use one of the above URLs to log in and revert the authentication chain.

Secure cookies

You can either use a secured HTTPS connection to access AM/OpenAM or disable Secure cookies as detailed in: Login to AM/OpenAM (All versions) fails with valid username/password after enabling Secure cookies.

Federation signing key

You can resolve this issue by updating the authentication certificate alias to match the alias of the new signing key as described in Unable to login to OpenAM console 12.x and 13.x or access REST API after changing the Federation Signing Key.

See How do I change the Signing Key for Federation in OpenAM 12.x and 13.x? for further information. 

Cookie domain

You should ensure the cookie domain is set to the full FQDN of the server in AM / OpenAM 13.5. If you have a site configuration, you should use the FQDN of the first server in your deployment.

See Reference › Configuration Reference › Platform for further information on setting the cookie domain.

Host-based cookies

You can upgrade to a later version of AM/OpenAM which fixes this issue or set the cookie domains to "" using ssoadm as detailed in: OpenAM 12.0.0, 12.0.1, 12.0.2 and 13.0 login fails with User name/password combination is invalid error if you use host-based cookies.

Encryption key

You can resolve this issue in one of two way depending on the cause as detailed in: JCEEncryption:: failed to decrypt data error when accessing the admin console in OpenAM 12.x or 13.x.

See Also

How do I change the amadmin password in AM/OpenAM (All versions)?

How do I change the amadmin and dsameuser passwords at the same time in AM/OpenAM (All versions)?

FAQ: Users in AM/OpenAM

Administrator and user accounts in AM/OpenAM

Authentication and Single Sign-On Guide › Reference › Core

Authentication and Single Sign-On Guide › Implementing Authentication › Configuring Authentication Chains

AM 5 Release Notes

Related Training

N/A

Related Issue Tracker IDs

OPENAM-11398 (OpenAM ACI installation instruction does not work for OpenDJ productionMode)

OPENAM-11116 (Admin console should still be accessible even if CTS is not available.)

OPENAM-10966 (Missing step for upgrade from AM 13 to 14 - CTS)

 OPENAM-10451 (Could not get OpenDJ 2.6.4 to work with OpenAM 14.0.0M10 for External CTS setup)

OPENAM-10383 (Validation of External CTS store using OpenAM GUI)

OPENAM-10382 (If we set up External CTS wrongly then OpenAM fails catastrophically on start up.)

OPENAM-8214 (JCEEncryption errors are recorded during/after upgrading to 13)

OPENAM-5264 (Can't login to OpenAM with no cookies set in the platform service)


Upgrade to AM/OpenAM (All versions) fails when anonymous access is disabled in DS/OpenDJ

The purpose of this article is to provide assistance if your upgrade to AM/OpenAM fails with a "Connect Error: No operational connection factories available". This issue will only affect you if you are using DS/OpenDJ for your configuration store and anonymous access is disabled for that DS/OpenDJ instance.

Symptoms

The upgrade process fails and a Upgrade Failed message is shown if you are using the upgrade.jar tool (openam-upgrade-tool-13.0.0.jar for OpenAM 13.0) to perform the upgrade. If you are using the Upgrade Wizard, the upgrade hangs on the Upgrade in progress window.

An error similar to the following is shown in the amUpgrade log when this happens:

amUpgrade:10/08/2015 09:32:11:664 AM GMT: Thread[http-bio-8080-exec-11,5,main]
ERROR: An error occurred while trying to get a connection
org.forgerock.opendj.ldap.ConnectionException: Connect Error: No operational connection factories available
    at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:210)
    at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:172)
    at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:142)
... 
Caused by: org.forgerock.opendj.ldap.ConnectionException: Server Connection Closed: Heartbeat failed
    at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:210)
... 
Caused by: org.forgerock.opendj.ldap.ErrorResultException: Unwilling to Perform: Rejecting the requested operation because the connection has not been authenticated

Correspondingly, the DS/OpenDJ access log shows a CONNECT and DISCONNECT without a SEARCH occurring:

[12/Jun/2015:15:46:31 +0100] CONNECT conn=6 from=127.0.0.1:35590 to=127.0.0.1:20389 protocol=LDAP 
[12/Jun/2015:15:46:31 +0100] UNBIND REQ conn=6 op=1 msgID=2 
[12/Jun/2015:15:46:31 +0100] DISCONNECT conn=6 reason="Client Unbind"  

Recent Changes

Disabled anonymous access in DS/OpenDJ using the following command:

$ ./dsconfig --hostname server.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password set-global-configuration-prop --set reject-unauthenticated-requests:true --trustAll --no-prompt

Causes

Completely disabling anonymous access in DS/OpenDJ prevents the SEARCH request from succeeding if AM/OpenAM uses the heartbeat mechanism (which it does by default) and causes connections from AM/OpenAM to fail. AM/OpenAM needs access to the configuration store during the upgrade process, so the upgrade process fails if AM/OpenAM cannot connect.

See How does AM/OpenAM (All versions) use anonymous access calls to DS/OpenDJ? for further information.

Solution

This issue can be resolved by re-enabling anonymous access in DS/OpenDJ using the following command and then performing the upgrade again:

$ ./dsconfig --hostname server.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password set-global-configuration-prop --set reject-unauthenticated-requests:false --trustAll --no-prompt

You can use Access Control Instruction (ACI) in DS/OpenDJ to prevent anonymous access without affecting the heartbeat mechanism. See How do I prevent anonymous access in DS/OpenDJ (All versions)? for further information.

See Also

How do I prevent anonymous access in DS/OpenDJ (All versions)?

How does AM/OpenAM (All versions) use anonymous access calls to DS/OpenDJ?

AM/OpenAM (All versions) fails to connect to the user data store when anonymous access is disabled in DS/OpenDJ

Best practice for upgrading to OpenAM 13.x

Best practice for upgrading to OpenAM 12.x

Setup and Maintenance Guide › Setting Up Identity Data Stores

Installation Guide › Implementing the Core Token Service

Related Training

N/A

Related Issue Tracker IDs

OPENAM-3498 (IdRepo connection pool fails if anonymous access is disabled)


User Data Store


How do I understand what the user data store is used for in AM/OpenAM (All versions)?

The purpose of this article is to provide information to help you understand what the user store is used for in AM/OpenAM. The term user store can be used to refer to different data stores in AM/OpenAM. This article also discusses how the identity data store interacts with identities and authentication modules.

Understanding the user store

The term user store can be used to refer to one of the following data stores in AM/OpenAM:

  • Authentication - this data store contains the credentials used for authenticating a user. The location of this data store is determined by the type of Authentication module you are using and how it has been configured. For example, if you are using the LDAP authentication module, the user's credentials will be stored on your LDAP server. This data store is also referred to as the Credential Store. 
  • User profiles - this data store contains user profiles or identity data (attributes) for users who have authenticated to AM/OpenAM. The location of this data store is determined by the data store you have configured for the applicable realm. This data store is also referred to as the Identity Data Store or Identity Repository. You can check what data store has been configured in a realm as follows:
    • AM / OpenAM 13.x console: navigate to: Realms > [Realm Name] > Data Stores.
    • Pre-OpenAM 13 console: navigate to: Access Control > [Realm Name] > Data Stores.

When a user wants to access an application protected by AM/OpenAM, they supply credentials (such as user name and password) to log in. These credentials are then validated against the ones held in the Credential store (authentication); if they match, the user successfully authenticates to AM/OpenAM and the user profile is retrieved if an Identity data store is configured.

Note

Although these two user stores are conceptually different, they may in fact refer to the same data store. For example, if you are using the LDAP authentication module and have also configured your LDAP server to be used as the data store, then these user stores will be the same.

How does the identity data store, identities and authentication modules interact?

Component 1 - the identity data store 

The purpose of the identity data store is to provide information about the authenticated user, which allows you to identify users. This identity data store can be any directory server or an LDAP, and can be new or existing. The identity data store is configured on the Data Store page for each realm; this means you can have completely distinct identity data stores for each realm or inherit the top level realm configuration. It is not always necessary to have an identity data store though as the authentication module can authenticate users without one; this is discussed in Component 4 where other credentials are used for authenticating users and the identity data store is used purely to retrieve the user profile. 

In summary, the identity data store allows you to identify your users.

Component 2 - the Identities page (previously the Subjects tab)

The Identities page provides a view of some of your users (first 100 by default). It is a useful way of checking that your connection to the identity data store is working (if you can see your users, you know that you are connected). The Identities page provides basic information about users only. You should use a dedicated tool for managing users such as IDM/OpenIDM or DS/OpenDJ, depending on your use case.

In summary, the Identities page allows you check your connection to the identity data store.

Component 3 - authentication modules and how they relate to the identity data store

When configuring user authentication, you create authentication modules and add these modules to chains. A simple way to identify and authenticate a realm user is to use the default DataStore module in a chain. The DataStore module is whatever identity data store you have configured for that particular realm. The resulting flow when using the DataStore module is:

  1. When the user authenticates, AM/OpenAM validates the credentials entered by the user using the connection details you configured when you created your identity data store.
  2. Once validated, AM/OpenAM obtains the user details from the directory you use as your identity data store to begin building the user's profile. 

This flow explains why the DataStore module configuration is very simple; you configured everything the module needs when you configured the identity data store and added your identities initially. 

In summary, the DataStore authentication module uses the identity data store configured for a particular realm.

Component 4 - using something other than the realm identity data store for authentication

To attain flexibility and increase security, we need to provide the capability of authenticating users with a variety of credentials. These credentials may or may not reside in your directory of choice. Further, how users are authenticated depends on a number of things, including the resource you are protecting, its criticality and security requirements, the device they authenticate with, etc. The authentication method / credentials could be Active Directory® (AD) / DS/OpenDJ or other LDAP, SAML, OIDC, RSA token, etc. Therefore, we have decoupled the identity data store (where we identify users, provide the user profile) from the authentication method(s) themselves. This allows us to create and use a variety of other authentication modules, which offers greater flexibility when creating authentication chains, offering different authentication methods, 2FA, etc. 

However (and very importantly), any additional authentication methods need to be associated with the identity created in the identity data store via a common attribute so we know who we are dealing with, that is, if a user authenticates against DS/OpenDJ, but your identity data store is AD, we need to be able to identify the authenticated user within the identity data store. To achieve this, we map the identifier of the module to the cn/uid/sAMAccountName/[whatever you use to identify your user in AD] of the identity data store. Without this mapping, we can authenticate users but cannot identity them in the identity data store and therefore cannot retrieve their user profile.

Confusion arises when creating authentication modules. It is possible (and often correct) to re-create a connection to the directory used to populate the identity data store. A common scenario, for example, is you are using a lower level realm to limit the users who can access a resource to a particular geographical location. Provided you use exactly the same unique identifier in your newly created authentication module as you used in your identity data store, it will still work. Therefore, when the user authenticates, AM/OpenAM takes the unique identifier used to authenticate, searches the identity data store with it, finds the appropriate user and is happy. This is a supported approach and entirely appropriate, particularly if you are using numerous modules. However, keep in mind for troubleshooting, if AM/OpenAM cannot find the user using the unique identifier, the authentication will fail. 

In summary, you map your authentication module back to a user with a common attribute to allow AM/OpenAM to identify the identity and retrieve the user profile. 

See Also

FAQ: Users in AM/OpenAM

How do I configure AM/OpenAM (All versions) to use the sAMAccountName for authentication?

User has no profile in this organization message received when user authenticates in AM/OpenAM (All versions)

AM/OpenAM (All versions) fails to connect to the user data store when anonymous access is disabled in DS/OpenDJ

How do I make a whole user data store read-only to users in AM/OpenAM (All versions)?

Data stores in AM/OpenAM

Release Notes › Before You Install › Data Store Requirements

Authentication and Single Sign-On Guide › Implementing Authentication › Configuring Authentication Modules

Setup and Maintenance Guide › Setting Up Identity Data Stores

Setup and Maintenance Guide › Setting Up Identity Data Stores › About the Identity Repository Plugin

Related Training

ForgeRock Access Management Core Concepts (AM-400)

Related Issue Tracker IDs

N/A


How does AM/OpenAM (All versions) use anonymous access calls to DS/OpenDJ?

The purpose of this article is to provide information on how AM/OpenAM uses anonymous access calls to DS/OpenDJ. This is distinct from the anonymous user in AM/OpenAM.

Understanding anonymous access calls

AM/OpenAM uses heartbeats to monitor LDAP connections and the availability of the LDAP server; they are implemented as anonymous search requests targeted to the Root DSE entry. Heartbeats are the only way AM/OpenAM knows if an idle connection has been dropped by a firewall or load balancer.

AM/OpenAM uses anonymous search requests such as the following to achieve this:

SEARCH REQ conn=6 op=2468 msgID=2469 base="" scope=baseObject filter="(objectClass=*)" attrs="1.1"
Note

These searches do not pose a security risk as they just check the Root DSE, which does not expose any sensitive information. See Security Guide › Securing the Server Installation › Reconsider Default Global Access Control for further information on the Root DSE.

Completely disabling anonymous access in DS/OpenDJ (using set-global-configuration-prop with set reject-unauthenticated-requests:true) prevents this SEARCH request from succeeding if AM/OpenAM uses heartbeats (which it does by default) and causes connections from AM/OpenAM to fail. See the following articles for examples of the issues caused when connections fail:

See How do I prevent anonymous access in DS/OpenDJ (All versions)? for information on the preferred way to prevent anonymous access in DS/OpenDJ.

See Also

How do I prevent anonymous access in DS/OpenDJ (All versions)?

AM/OpenAM (All versions) fails to connect to the user data store when anonymous access is disabled in DS/OpenDJ

Upgrade to AM/OpenAM (All versions) fails when anonymous access is disabled in DS/OpenDJ

How do I deactivate the default anonymous user in AM/OpenAM (All versions)?

Security Guide › Securing the Server Installation › Reconsider Default Global Access Control

Authentication and Single Sign-On Guide › Implementing Authentication › Configuring Authentication Modules

Setup and Maintenance Guide › Setting Up Identity Data Stores

Installation Guide › Implementing the Core Token Service

Related Training

N/A

Related Issue Tracker IDs

OPENAM-3033 (CTS does not use heartbeat)

OPENAM-3498 (IdRepo connection pool fails if anonymous access is disabled)


How do I change the data store minimum password length in AM/OpenAM (All versions)?

The purpose of this article is to provide information on changing the data store minimum password length in AM/OpenAM. The minimum password length defaults to 8; this is a data store setting that applies to password changes (when existing users reset their password or change their password) and is independent of any password length restrictions in DS/OpenDJ. This also includes forgotten password resets made via the REST API user self-service functionality (XUI).

Overview

You can change the data store minimum password length using ssoadm (in all versions) or Amster (in AM 5 and later):

This setting cannot be changed in the console.

Using Amster

You can change the data store minimum password length using Amster; you can do this globally or in a specific realm, where realm level takes precedence over the global level. 

Follow the steps in How do I update property values in AM (All versions) using Amster?with these values:

  • Entity: IdRepository
  • Property: minimumPasswordLength
Note

You must restart the web application container in which AM/OpenAM runs to apply these configuration changes. 

Using ssoadm

You can change the data store minimum password length using ssoadm; you can do this globally or in a specific realm, where realm level takes precedence over the global level:

  • ssoadm - Global: 
    1. Run the following command to create a data file (called DATA_FILE to match the next command), which is populated with the current sunIdRepoAttributeValidator property values to ensure you don't lose any existing changes:
      $ ./ssoadm get-attr-defs -s sunIdentityRepositoryService -t Organization -u [adminID] -f [passwordfile] | grep sunIdRepoAttributeValidator > DATA_FILE
      
      replacing [adminID] and [passwordfile] with appropriate values.
    2. Update the data file you just created by amending the sunIdRepoAttributeValidator=minimumPasswordLength property value. For example, if you want to increase the minimum password length to 10, you would change it to:
      sunIdRepoAttributeValidator=minimumPasswordLength=10
    3. Run the following command to update the sunIdRepoAttributeValidator property values:
      $ ./ssoadm set-attr-defs -s sunIdentityRepositoryService -t Organization -u [adminID] -f [passwordfile] -D DATA_FILE
      replacing [adminID] and [passwordfile] with appropriate values.
    4. Restart the web application container in which AM/OpenAM runs to apply these configuration changes.
  • ssoadm - Realm:
    1. Run the following command to create a data file (called DATA_FILE to match the next command), which is populated with the current sunIdRepoAttributeValidator property values to ensure you don't lose any existing changes:
      $ ./ssoadm get-realm-svc-attrs -s sunIdentityRepositoryService -e [realmname] -u [adminID] -f [passwordfile] | grep sunIdRepoAttributeValidator > DATA_FILE
      
      replacing [realmname], [adminID] and [passwordfile] with appropriate values.
    2. Update the data file you just created by amending the sunIdRepoAttributeValidator=minimumPasswordLength property value. For example, if you want to increase the minimum password length to 10, you would change it to:
      sunIdRepoAttributeValidator=minimumPasswordLength=10
    3. Run the following command to update the sunIdRepoAttributeValidator property values:
      $ ./ssoadm set-realm-svc-attrs -s sunIdentityRepositoryService -e [realmname] -u [adminID] -f [passwordfile] -D DATA_FILE
      replacing [realmname], [adminID] and [passwordfile] with appropriate values.
Note

When changing the data store minimum password length in OpenAM 13.5.x, AM 5 or AM 5.1, you may encounter an "Exception in thread "SystemTimer" java.lang.Error: java.lang.ExceptionInInitializerError" response. This error can be safely ignored since the operation performed by ssoadm still completes successfully. See java.lang.ExceptionInInitializerError when using ssoadm commands in AM 5, 5.1 and OpenAM 13, 13.5, 13.5.1 for further information.

See Also

Forgotten password reset or password change fails with Minimum password length is 8 error in AM/OpenAM (All versions)

How do I change what characters are permitted in user names in AM/OpenAM (All versions) for authentication purposes?

Setup and Maintenance Guide › Setting Up Realms › Changing Passwords using the REST API

Related Training

N/A

Related Issue Tracker IDs

OPENAM-8999 (Incorrect messages are displayed when a user tries to change their password and it does not meet the minimum length)

OPENAM-6867 (changePassword REST endpoint is not returning LDAP issues that are related to a user mistake.)

OPENAM-6166 (Allow Forgotten Password functionality in OpenAM to use other user attributes besides uid and mail)


How do I create a user data store in AM/OpenAM (All versions) using ssoadm?

The purpose of this article is to provide information on creating a user data store in AM/OpenAM using ssoadm. This is an external identity repository that contains user profiles or identity data (attributes) for users who have authenticated to AM/OpenAM.

Creating a data store

The properties you can set when creating a data store depend on the data store type you are creating. You can find the equivalent ssoadm attribute names by looking in the documentation for the appropriate data store type: Setup and Maintenance Guide › Setting Up Identity Data Stores.

Note

Please be aware of the following:

  • As of AM 6.5.1, the LDAP Server property is now sun-idrepo-ldapv3-config-ldap-server=[0] to allow the servers to be specified in a specific order. In earlier versions, it was sun-idrepo-ldapv3-config-ldap-server.
  • You cannot change the Load Schema or Load schema when saved option via ssoadm when creating a data store, as AM/OpenAM should not amend the directory schema. This is discussed in OPENAM-1853 (Add ssoadm flag to load schema while creating datastore).

You can create a data store via ssoadm as follows:

  1. Create a data file (called DATA_FILE to match the next command) and populate it with the required properties. The attached DATASTORE_DATA_FILE provides all the properties for the DS data store in AM 6.5.1 (with default values); you can use this as a template to create a DS data store.
  2. Enter the following command to create the data store with these settings: 
    $ ./ssoadm create-datastore -e [realmname] -u [adminID] -f [passwordfile] -m [datastoreName] -t [datastoreType] -D DATA_FILE
    
    replacing [realmname], [adminID], [passwordfile], [datastoreName] and [datastoreType] with appropriate values, where [datastoreType] is one of the following values (which correspond to supported data store types):
    datastoreType Description
    LDAPv3ForAD Active Directory
    LDAPv3ForTivoli Tivoli Directory Server
    LDAPv3ForAMDS Sun DS with AM/OpenAM schema
    LDAPv3ForOpenDS DS/OpenDJ

Refer to the release notes for a list of supported data stores. For example: Release Notes › Data Store Requirements

See Also

How do I understand what the user data store is used for in AM/OpenAM (All versions)?

How do I make a whole user data store read-only to users in AM/OpenAM (All versions)?

How do I change the data store minimum password length in AM/OpenAM (All versions)?

Data stores in AM/OpenAM

Setup and Maintenance Guide › Setting Up Identity Data Stores

Reference › ssoadm

Related Training

N/A

Related Issue Tracker IDs

OPENAM-11912 (LDAPv3 data store type does not handle property 'sun-idrepo-ldapv3-config-auth-kba-attr')

OPENAM-5867 (Data Store LDAP server (admin-ordered) list is reordered by OpenAM)

OPENAM-3762 (sun-idrepo-ldapv3-config-memberurl doesn't exist for LDAPv3ForOpenDS )

OPENAM-1853 (Add ssoadm flag to load schema while creating datastore)


How do I configure AM/OpenAM (All versions) to use the sAMAccountName for authentication?

The purpose of this article is to provide information on configuring AM/OpenAM to use the sAMAccountName attribute for authentication when you have an Active Directory® user data store. This change allows your users to log in using sAMAccountName as their user ID.

Using sAMAccountName as the user ID

Active Directory does not have a uid attribute, but instead uses sAMAccountName as the equivalent of the uid attribute. This means you need to configure AM/OpenAM to use sAMAccountName as the user ID. This article assumes you are already using the Active Directory authentication module; if not, you should configure it and set the attributes as described below. See Authentication and Single Sign-On Guide › Active Directory Module Properties for further information.

Note

The Search and Naming Attributes specified in your Active Directory authentication module configuration must match your user store configuration, otherwise you will encounter an error. See User has no profile in this organization message received when user authenticates in AM/OpenAM (All versions) for further information. Additionally the Search and Naming Attributes set in the data store must match, otherwise you will see "User Requires Profile to Login" errors.

You can configure AM/OpenAM as follows:

  1. Change the attributes used in the Active Directory data store using either the console, Amster (AM 5 and later) or ssoadm:
    • AM 6 and later console: navigate to: Realms > [Realm Name] > Data Stores > [AD Data Store] and set the following attributes to sAMAccountName:
      User Configuration tab > LDAP Users Search Attribute
      Authentication Configuration tab > Authentication Naming Attribute
    • AM 5.x / OpenAM 13.x console: navigate to: Realms > [Realm Name] > Data Stores > [AD Data Store] and set the following attributes to sAMAccountName:
      LDAP Users Search Attribute
      Authentication Naming Attribute
    • Pre-OpenAM 13 console: navigate to: Access Control > [Realm Name] > Data Stores > [AD Data Store] and set the above attributes to sAMAccountName.
    • Amster: follow the steps in How do I update property values in AM (All versions) using Amster? with these values:
      • Entity: ActiveDirectory
      • Properties: sun-idrepo-ldapv3-config-users-search-attribute, sun-idrepo-ldapv3-config-auth-naming-attr
    • ssoadm: enter the following command:
      $ ./ssoadm update-datastore -e [realmname] -m [datastorename] -u [adminID] -f [passwordfile] -a sun-idrepo-ldapv3-config-users-search-attribute=sAMAccountName sun-idrepo-ldapv3-config-auth-naming-attr=sAMAccountName
      replacing [realmname], [datastorename], [adminID] and [passwordfile] with appropriate values.
  2. Change the attributes used in the Active Directory authentication module using either the console, Amster (AM 5 and later) or ssoadm:
    • AM / OpenAM 13.x console: navigate to: Configure > Authentication > Active Directory and set the following attributes to sAMAccountName:
      Attribute Used to Retrieve User Profile
      Attributes Used to Search for a User to be Authenticated
      
    • Pre-OpenAM 13 console: navigate to: Configuration > Authentication > Active Directory and set the above attributes to sAMAccountName:
    • Amster: follow the steps in How do I update property values in AM (All versions) using Amster? with these values:
      • Entity: ActiveDirectoryModule
      • Properties: userProfileRetrievalAttribute, userSearchAttributes
    • ssoadm: enter the following command:
      $ ./ssoadm update-auth-instance -e [realmname] -u [adminID] -f [passwordfile] -a iplanet-am-auth-ldap-user-naming-attribute=sAMAccountName iplanet-am-auth-ldap-user-search-attributes=sAMAccountName
      replacing [realmname], [adminID] and [passwordfile] with appropriate values.

See Also

Setup and Maintenance Guide › Active Directory Configuration Properties

How do I understand what the user data store is used for in AM/OpenAM (All versions)?

How do I create a user data store in AM/OpenAM (All versions) using ssoadm?

Data stores in AM/OpenAM

Authentication modules in AM/OpenAM

Configuring and troubleshooting WDSSO in AM/OpenAM

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I make a whole user data store read-only to users in AM/OpenAM (All versions)?

The purpose of this article is to provide information on making a whole user data store read-only to users in AM/OpenAM when the user data store is being accessed via the LDAP identity repository plug-in. You would do this if you want to prevent users in AM/OpenAM from creating, changing or deleting user entries in the data store.

Making a user data store read-only

You can make a user data store read-only to all users in AM/OpenAM (by making these changes against the top level realm) or to a subset of users (by making these changes to a realm). You cannot make the user data store read-only for individual users.

You can make these changes using either the console or ssoadm:

  • AM / OpenAM 13.x console: navigate to: Realms > [Realm Name] > Data Stores > [Data Store Name] > Plug-in Configuration > LDAPv3 Plug-in Supported Types and Operations and remove the user=read,create,edit,delete,service value. Add a new value of user=read,service.
  • Pre-OpenAM 13 console: navigate to: Access Control > [Realm Name] > Data Stores > [Data Store Name] > Plug-in Configuration > LDAPv3 Plug-in Supported Types and Operations and remove the user=read,create,edit,delete,service value. Add a new value of user=read,service.
  • ssoadm: 
    1. Create a data file (called DATA_FILE to match the next command) with the following contents: The realm and group settings shown are default settings; if you have already changed these, you should substitute your values instead to ensure your changes are not lost.
      sunIdRepoSupportedOperations=user=read,service
      sunIdRepoSupportedOperations=realm=read,create,edit,delete,service
      sunIdRepoSupportedOperations=group=read,create,edit,delete,service​
    2. Enter the following command:
      $ ./ssoadm update-datastore -e [realmname] -m [datastorename] -u [adminID] -f [passwordfile] -D DATA_FILE 
      
      replacing [realmname], [datastorename], [adminID] and [passwordfile] with appropriate values. If you are making changes against the top level realm, you must specify -e /

You can check the settings for the data store to ensure the changes are as expected using the following command:

$ ./ssoadm show-datastore -e [realmname] -m [datastorename] -u [adminID] -f [passwordfile]

replacing [realmname], [datastorename], [adminID] and [passwordfile] with appropriate values. If you are making changes against the top level realm, you must specify -e /

Note

You can also do this by specifying access control to the directory itself in DS/OpenDJ as described in: Administration Guide › Configuring Privileges and Access Control​; this is the recommended approach.

See Also

How do I make individual user profile attributes read-only in AM/OpenAM (All versions)?

How do I create a user data store in AM/OpenAM (All versions) using ssoadm?

How do I understand what the user data store is used for in AM/OpenAM (All versions)?

Setup and Maintenance Guide › Setting Up Identity Data Stores › About the Identity Repository Plugin

Reference › Command Line Tools › ssoadm

Administration Guide › Configuring Privileges and Access Control

Related Training

ForgeRock Access Management Core Concepts (AM-400)

Related Issue Tracker IDs

N/A


FAQ: Users in AM/OpenAM

The purpose of this FAQ is to provide answers to commonly asked questions regarding users in AM/OpenAM.

Frequently asked questions

Q. Can I use AM/OpenAM to manage users?

A. AM/OpenAM is not designed to be a fully featured user administration tool; the user functionality is intended to be used for validating connectivity to your identity repository. You should use a dedicated tool for managing users such as IDM/OpenIDM or DS/OpenDJ, depending on your use case.

Q. Why can’t I see a user on the Identities page (previously the Subjects tab)?

A. One possible reason is that you may have more than 100 users. Only the first 100 users are shown on the Identities page by default; therefore, you should search for a user if you have more than 100 users.

In AM / OpenAM 13.5, you can increase the number of users shown by editing the schema directly using an LDAP browser such as the DS/OpenDJ control panel (removed in DS 6) or Apache™ Directory Studio as follows:

  1. Navigate to the following DN:
    ou=1.0,ou=iPlanetAMAdminConsoleService,ou=services,dc=example,dc=com
  2. Search for "iplanet-am-admin-console-search-limit" in the editor box for the schema XML and change the default value from 100 to your desired value.

In OpenAM 13 and earlier, you can increase the number of users shown by setting the Maximum Results Returned from Search field to your desired value. See OpenAM Reference › Administration for further information. This field has been removed in OpenAM 13.5. 

Caution

Increasing the number of users shown on the Identities page may slow the loading time of users.

Q. How can I check how many users are being protected by AM/OpenAM?

A. You can query your LDAP user store to find the number of users. The following examples demonstrate two approaches, where the countentries option is used to return the total number of entries that match the search filter and displays the total number on the last line.

Using a user based objectclass

You can use a query such as the following if you are using the default schema objectClass of inetOrgPerson in DS/OpenDJ:

$ ./ldapsearch --port 1389 --bindDN "cn=Directory Manager" --bindPassword password --baseDN dc=example,dc=com --countentries objectClass=inetOrgPerson

This approach will not work if you use the same objectClass for real people as well as non real person objects.

Using uids

You can use a query such as the following to return all users and all attributes:

$ ./ldapsearch --hostname ds1.example.com --port 1389 --bindDN “cn=Directory Manager” --bindPassword password --baseDN dc=example,dc=com --searchScope sub --countEntries “(uid=*)” \*

This approach will not work if you use a non-uid based RDN.

Q. Can I specify the exact DN/container that the user is to be stored on when creating a new user?

A. No, AM/OpenAM does not allow end-users to specify the exact DN/container for newly created users; AM/OpenAM will just utilize the configured LDAP people container name/value setting. The DN value provided for the /json/users REST endpoint is essentially ignored by the underlying code and the entry will be created based on the data store settings (which again is only able to work with a single container).

Q. What is the default search attribute used when searching for users on the Identities page?

A. The default search attribute is cn. This can be changed as described in How do I change the search attribute used to search for users in AM/OpenAM console (All versions)?

Q. What is the Universal ID shown against a user or group on the Identities page?

A. The Identities page displays identities from the configured data stores. As AM/OpenAM abstracts the underlying data store technology (for example, directory server, database, NoSQL database, external web service etc) away from the "identity" concept, no assumptions are made about the underlying data.

When AM/OpenAM displays the user or group "identity", it displays generic information such as the user's name or the group members. It also displays the Universal ID, which is not to be confused with the actual DN of the entry in your configured LDAP based data store. The Universal ID is used by AM/OpenAM to differentiate identities, and is in the following format:

id=[identity name],ou=[identity type],[realm DN]

Where:

  • [identity name] is the name of the identity.
  • [identity type] is the type, which can be user, group, role or filteredrole.
  • [realm DN] is the identifier of the realm the given identity belongs to (which contains the configuration store's root suffix).

Q. Why can the admin user (called amadmin by default) log in to AM/OpenAM when no other users can?

A. The admin user is stored in the configuration repository in AM/OpenAM, whereas other users are stored in your identity repository. If there are connection issues to your identity repository, these will prevent other users logging in but the admin user will still be able to log in.

See Login to AM/OpenAM console (All versions) fails for amadmin user for further information if the amadmin user cannot log in.

Q. Why can't I log into a sub-realm with amadmin?

A. amadmin is only available in the top level realm; sub-realms use the same identity repository as the top level realm by default, but because this user isn't in the identity repository they are not available in sub-realms. You can use the demo user in sub-realms as this user is stored in the identity repository. 

Q. How can I be certain that a user is logging in to the top level realm?

A. You can specify the top level realm in your URL by setting realm = /, for example:

  • AM 5 and later:
    http://host1.example.com:8080/openam/XUI/?realm=/#login
  • Pre-AM 5:
    http://host1.example.com:8080/openam/XUI/#login/&realm=/

See AM Authentication and Single Sign-On Guide › Using Authentication › Authenticating From a Browser or OpenAM 13 Administration Guide › Defining Authentication Services › Authenticating To OpenAM for optional ways of specifying the Realm in XUI Login URLs.

Q. Can I prevent the default anonymous user logging in to the top level realm?

A. The default anonymous user is used in conjunction with the Anonymous Authentication module. It is safe to deactivate this user if you do not use this module and this will prevent them logging in to the top level realm.

See How do I deactivate the default anonymous user in AM/OpenAM (All versions)? for further information.

Q. What happens to users' sessions when I deactivate a realm?

A. Users who are currently logged in and have a valid session will be unaffected; their session will remain active. However, other users will not be able to authenticate to that realm once it is deactivated. This is also true if you are using SAML federation and AM/OpenAM is the IdP. If AM/OpenAM is acting as the SP, SAML based SSO will fail as AM/OpenAM will be unable to process assertions issued by the IdP.

You cannot end all user sessions by deactivating a realm.

Q. Can I warn a user that their session is about to expire?

A. No, there is no way of warning a user that their session will expire. The session will automatically expire once it has timed out.

Q. Is it possible to create read-only users in AM/OpenAM?

A. You cannot configure this on a per user basis but you can make a whole data store read-only in AM/OpenAM. See How do I make a whole user data store read-only to users in AM/OpenAM (All versions)? for further information.

There is an RFE, which may address this in the future: OPENAM-5714 (Add support for more granular delegation privileges).

Q. Can I create an admin user who has similar access rights to the amAdmin user?

A. Yes you can. You can create admin users with different access rights (privileges) according to your needs. See How do I create an admin user in AM/OpenAM (All versions) with amadmin privileges? for details on how you assign these privileges to a user and Setup and Maintenance Guide › Reference › Realm Privileges Configuration Reference for details on the different privileges that are available.

There is an RFE for additional privileges: OPENAM-5714 (Add support for more granular delegation privileges).

See Also

Login to AM/OpenAM console (All versions) fails for amadmin user

How do I unlock a user's account using the REST API in AM/OpenAM (All versions)?

How do I configure user level session timeouts in AM/OpenAM (All versions)?

How do I obtain the user's session ID in AM/OpenAM (All versions) when browser cookies are disabled?

How do I understand what the user data store is used for in AM/OpenAM (All versions)?

How do I create a user data store in AM/OpenAM (All versions) using ssoadm?

Administrator and user accounts in AM/OpenAM

Related Training

ForgeRock Access Management Core Concepts (AM-400)


Known Issues


isMemberOf attribute does not return current group membership details for a user in AM/OpenAM (All versions)

The purpose of this article is to provide information on getting up-to-date group membership details for a user in AM/OpenAM using the REST API and the DS/OpenDJ isMemberOf attribute.

Symptoms

The value of the isMemberOf attribute does not show the latest group membership data when queried via REST. For example, if you update a user's group membership and then perform a simple GET on that user (assuming your data store has been configured to use the isMemberOf attribute for group membership), you will see old membership data returned for the user:

Example

  1. Check user's group membership:
    $ curl -X GET http://host1.example.com:8080/openam/json/realms/root/users/demo
    
    ...
     "isMemberOf": [
            "cn=adminGroup,ou=groups,dc=example,dc=com"
    
  2. Add the user to a new group:
    $ curl -X PUT -H "Content-type: application/json" -H "Accept-API-Version: resource=3.0" -H "iPlanetDirectoryPro: AQIC5wM2LY4Sfcxs...EwNDU2NjE0*" -d'{
        "uniquemember":["uid=demo,ou=people,dc=example,dc=com"]
    }' http://host1.example.com:8080/openam/json/realms/root/groups/newGroup
  3. Check the user's membership again:
    $ curl -X GET http://host1.example.com:8080/openam/json/realms/root/users/demo
    
    ...
     "isMemberOf": [
            "cn=adminGroup,ou=groups,dc=example,dc=com"
    
    The expected result is:
     "isMemberOf": [
            "cn=newGroup,ou=groups,dc=example,dc=com"
    

You can see the correct group membership details on the Identities page (previously the Subjects tab) in the AM/OpenAM console.

Recent Changes

N/A

Causes

The 'isMemberOf' attribute is a virtual operational attribute in DS/OpenDJ. Operational attributes are not updated by the persistent search mechanism, which means AM/OpenAM does not receive notifications of changes to cached data that would normally occur with a regular attribute and therefore assumes that all the attributes still have the same values.

The Identities page within the console queries the membership of LDAP static groups and does not rely on caching at all, which is why this view is accurate.

Solution

A new option has been added to the users endpoint in AM 6 that allows you to query groups for an individual user. For example:

$ curl -X GET -H 'Accept: application/json' 'http://host1.example.com:8080/openam/json/realms/root/users/demo/groups?_queryFilter=true'

Example response:

{
  "result": [
    {
      "_id": "newGroup",
      "_rev": "635651178",
      "groupname": "newGroup"
    }
  ],
  "resultCount": 1,
  "pagedResultsCookie": null,
  "totalPagedResultsPolicy": "NONE",
  "totalPagedResults": -1,
  "remainingPagedResults": 0
}

isMemberOf attribute

You can use one of the following options to ensure current information is returned when using the isMemberOf attribute:

  • Perform a pseudo update on a user entry when group memberships are changed to trigger the persistent search notification. For example, when you change the group membership you could also do one of the following at the same time to trigger the persistent search notification:
    • Update an existing regular attribute (for example, mail) with the same value it already has.
    • Update a user attribute that's not used for other purposes.
  • Use time-based IdRepo cache aging (Time-to-Live) to give you confidence in data accuracy based on the cache age configured. See FAQ: Caching in AM/OpenAM(Q. How can I control caching for configuration and user data using ssoadm? > Time-to-Live) for further information on setting these properties.
  • Disable the IdRepo cache completely. See Setup and Maintenance Guide › To Turn Off Global User Data Caching for further information. You should test this in a pre-production environment first to assess what impact this has on your setup.
Caution

Disabling the IdRepo cache can have a severe negative impact on performance, since AM/OpenAM must query the data store each time it needs data when caching is disabled. Additionally, there is a known issue in OpenAM 13.0: OPENAM-8269 ("AuthId JWT Signature not valid" error in multi-instance deployments on 13). See Authentication fails in OpenAM 13.0 with an AuthId JWT Signature not valid error for further information.

See Also

FAQ: Caching in AM/OpenAM

isMemberOf values not returned with an anonymous ldapsearch in OpenDJ 2.6.3, 2.6.4 and 3.0

Authentication fails in OpenAM 13.0 with an AuthId JWT Signature not valid error 

Related Training

N/A

Related Issue Tracker IDs

OPENAM-9030 (Improve group management implementation)

OPENAM-8521 (CacheBlockBase will deadlock when com.sun.identity.idm.cache.entry.expire.enabled=true)

OPENAM-8269 ("AuthId JWT Signature not valid" error in multi-instance deployments on 13)


AM/OpenAM (All versions) fails to connect to the user data store when anonymous access is disabled in DS/OpenDJ

The purpose of this article is to provide assistance if you experience connectivity failures between AM/OpenAM and the user data store (Identity Repository). Common errors logged include "Connection factory became offline" error and data store module fails with "ldap errorcode=91". This issue will only affect you if you are using DS/OpenDJ for your user data store and anonymous access is disabled for that DS/OpenDJ instance.

Symptoms

Users cannot authenticate to AM/OpenAM protected applications and you cannot see your users on the Identities page (previously the Subjects tab) in the console.

An error similar to the following is shown in the IdRepo log when this happens:

LDAPUtils:12/06/2015:03:46:31:049 PM CST: Thread[SystemTimerPool,5,main]
**********************************************
LDAPUtils:12/06/2015:03:46:11:048 PM CST: Thread[SystemTimerPool,5,main]
ERROR: Connection factory became offline: AuthenticatedConnectionFactory(HeartBeatConnectionFactory(LDAPConnectionFactory(host1.example.com/192.168.1.8:1636)), SimpleBindRequest(name=uid=openam-admin,ou=sysaccounts,dc=example,dc=com, authentication=simple, controls=[]))
org.forgerock.opendj.ldap.ConnectionException: Connect Error: Connection refused
     at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:163)
     at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:125)
     at com.forgerock.opendj.ldap.LDAPConnectionFactoryImpl$CompletionHandlerAdapter.adaptConnectionException(LDAPConnectionFactoryImpl.java:183)
     at com.forgerock.opendj.ldap.LDAPConnectionFactoryImpl$CompletionHandlerAdapter.failed(LDAPConn
...
Caused by: java.net.ConnectException: Connection refused
     at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
     at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
     at org.glassfish.grizzly.nio.transport.TCPNIOConnectorHandler.onConnectedAsync(TCPNIOConnectorHandler.java:206)

One of the following errors is shown in the Authentication log, depending on which authentication module you are using:

  • LDAP authentication module:
    amAuthLDAP:12/06/2015 03:46:26:919 PM CST: Thread[http-bio-8080-exec-57,5,main]
    WARNING: Search for User error: 
    org.forgerock.opendj.ldap.ConnectionException: Connect Error: No operational connection factories available
        at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:210)
    ...
    Caused by: org.forgerock.opendj.ldap.ConnectionException: Server Connection Closed: Heartbeat failed
        at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:210)
    ...
    Caused by: org.forgerock.opendj.ldap.ErrorResultException: Unwilling to Perform: Rejecting the requested operation  because the connection has not been authenticated
        at org.forgerock.opendj.ldap.ErrorResultException.newErrorResult(ErrorResultException.java:232)
    
  • Data Store authentication module:
    amAuth:12/06/2015 03:28:39:704 PM BST: Thread[http-bio-18080-exec-2,5,main]
    Exception : 
    com.sun.identity.authentication.spi.AuthLoginException: Authentication Failed
    Plug-in org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo encountered a ldap exception.  ldap errorcode=91
            at com.sun.identity.authentication.modules.datastore.DataStore.process(DataStore.java:165)
            at com.sun.identity.authentication.spi.AMLoginModule.wrapProcess(AMLoginModule.java:1023)
            at com.sun.identity.authentication.spi.AMLoginModule.login(AMLoginModule.java:1197)
    ...
    Caused by: Message:Plug-in org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo encountered a ldap exception.  ldap errorcode=91
    
            at org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo.newIdRepoException(DJLDAPv3Repo.java:2478)
            at org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo.handleErrorResult(DJLDAPv3Repo.java:2451)
            at org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo.getDN(DJLDAPv3Repo.java:2316)
            at org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo.findDNForAuth(DJLDAPv3Repo.java:2266)
    

Correspondingly, the DS/OpenDJ access log shows a CONNECT and DISCONNECT without a SEARCH occurring:

[12/Jun/2015:15:46:31 +0100] CONNECT conn=907 from=127.0.0.1:35590 to=127.0.0.1:20389 protocol=LDAP 
[12/Jun/2015:15:46:31 +0100] UNBIND REQ conn=907 op=2 msgID=3 
[12/Jun/2015:15:46:31 +0100] DISCONNECT conn=907 reason="Client Unbind"  

Recent Changes

Disabled anonymous access in DS/OpenDJ using the following command:

$ ./dsconfig --hostname server.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password set-global-configuration-prop --set reject-unauthenticated-requests:true --trustAll --no-prompt

Causes

Completely disabling anonymous access in DS/OpenDJ prevents the SEARCH request from succeeding if AM/OpenAM uses the heartbeat mechanism (which it does by default) and causes connections from AM/OpenAM to fail. 

See How does AM/OpenAM (All versions) use anonymous access calls to DS/OpenDJ? for further information.

Solution

This issue can be resolved by allowing anonymous access in DS/OpenDJ and then using Access Control Instruction (ACI) to ALLOW anonymous binds for the heartbeat search but disallow other anonymous searches. See How do I prevent anonymous access in DS/OpenDJ (All versions)? for further information.

  • You can use the following command to allow anonymous access:
    $ ./dsconfig --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password set-global-configuration-prop --set reject-unauthenticated-requests:false --trustAll --no-prompt
    
  • You should construct an ACI that allows anonymous binds for the heartbeat search, but only allows access to the root DSE; the following is an example SEARCH request AM/OpenAM normally sends for the heartbeat:
    SEARCH REQ conn=6 op=2468 msgID=2469 base="" scope=baseObject filter="(objectClass=*)" attrs="1.1"
    
  • You can reproduce the heartbeat request using an ldapsearch such as the following:
    $ ./ldapsearch --port 50389 --bindDN "cn=Directory Manager" --bindPassword password --baseDN "" --searchScope base "objectClass=*" "1.1"
    

Alternatively, you can disable heartbeats in AM/OpenAM, although this approach is not recommended as it degrades failover handling. You can disable heartbeats in the data store as follows; you must disable heartbeats for data stores in all affected realms:

  • AM / OpenAM 13.x console: navigate to: Realms > [Realm Name] > Data Stores > [Data Store Name] > LDAP Connection Heartbeat Interval and enter 0.
  • Pre-OpenAM 13 console: navigate to: Access Control > [Realm Name] > Data Stores > [Data Store Name] > LDAP Connection Heartbeat Interval and enter 0.
  • ssoadm: enter the following command:
    $ ./ssoadm update-datastore -e [realmname] -m [datastorename] -u [adminID] -f [passwordfile] -a openam-idrepo-ldapv3-heartbeat-interval=0
    replacing [realmname], [datastorename], [adminID] and [passwordfile] with appropriate values.
  • Amster: follow the steps in How do I update property values in AM (All versions) using Amster?with these values:
    • Entity: OpenDJ
    • Property: openam-idrepo-ldapv3-heartbeat-interval
Caution

If you choose to disable heartbeats rather than using access controls, you should test this in your pre-production environment first to ensure there are no unwanted side effects.

See Also

How do I prevent anonymous access in DS/OpenDJ (All versions)?

How does AM/OpenAM (All versions) use anonymous access calls to DS/OpenDJ?

Data stores in AM/OpenAM

Setup and Maintenance Guide › Setting Up Identity Data Stores

Related Training

N/A

Related Issue Tracker IDs

OPENAM-3498 (IdRepo connection pool fails if anonymous access is disabled)


Data validation failed error when creating or updating user data store LDAP attributes after upgrading to AM 6.5.1

The purpose of this article is to provide assistance if you encounter a "Data validation failed for the attribute, sun-idrepo-ldapv3-config-ldap-server" error in AM 6.5.1 when creating or updating the user data store LDAP attributes via ssoadm.

Symptoms

The following error is shown when executing a ssoadm command that updates or creates a data store, for example, update-datastore or create-datastore:

com.sun.identity.cli.CLIException: Message:Data validation failed for the attribute, sun-idrepo-ldapv3-config-ldap-server

The following error is shown in the Configuration debug log when this happens:

amCLI:05/08/2019 10:41:29:750 AM BST: Thread[main,5,main]: TransactionId[unknown]
ERROR: CommandManager.<init>
com.sun.identity.cli.CLIException: Message:Data validation failed for the attribute, sun-idrepo-ldapv3-config-ldap-server
   at com.sun.identity.cli.datastore.UpdateDataStore.handleRequest(UpdateDataStore.java:111)
   at com.sun.identity.cli.SubCommand.execute(SubCommand.java:296)
   at com.sun.identity.cli.CLIRequest.process(CLIRequest.java:217)
   at com.sun.identity.cli.CLIRequest.process(CLIRequest.java:139)
   at com.sun.identity.cli.CommandManager.serviceRequestQueue(CommandManager.java:604)
   at com.sun.identity.cli.CommandManager.<init>(CommandManager.java:186)
   at com.sun.identity.cli.CommandManager.main(CommandManager.java:163)
Caused by: Message:Data validation failed for the attribute, sun-idrepo-ldapv3-config-ldap-server
   at com.sun.identity.sm.ServiceSchemaImpl.throwInvalidAttributeValuesException(ServiceSchemaImpl.java:757)
   at com.sun.identity.sm.ServiceSchemaImpl.clientEndAttrValidation(ServiceSchemaImpl.java:691)
   at com.sun.identity.sm.ServiceSchemaImpl.validatePlugin(ServiceSchemaImpl.java:669)
   at com.sun.identity.sm.ServiceSchemaImpl.validateAttrValues(ServiceSchemaImpl.java:606)
   at com.sun.identity.sm.ServiceSchemaImpl.validateAttributes(ServiceSchemaImpl.java:354)
   at com.sun.identity.sm.ServiceSchemaImpl.validateAttributes(ServiceSchemaImpl.java:323)
   at com.sun.identity.sm.ServiceConfig.setAttributes(ServiceConfig.java:544)
   at com.sun.identity.cli.datastore.UpdateDataStore.handleRequest(UpdateDataStore.java:92)
   ... 6 more

Recent Changes

Upgraded to AM 6.5.1.

Causes

The sun-idrepo-ldapv3-config-ldap-server ssoadm property has been renamed to sun-idrepo-ldapv3-config-ldap-server=[n] to allow the priority order of LDAP servers to be specified. This change occurred as a result of OPENAM-5867 (Data Store LDAP server (admin-ordered) list is reordered by OpenAM).

Previously you could specify multiple servers as follows, but the order in which they were entered was not saved:

sun-idrepo-ldapv3-config-ldap-server=localhost:50389
sun-idrepo-ldapv3-config-ldap-server=example.com:1389

Now you can specify them as follows, which sets the priority order:

sun-idrepo-ldapv3-config-ldap-server=[0]=localhost:50389
sun-idrepo-ldapv3-config-ldap-server=[1]=example.com:1389
Note

The change in property name does not affect Amster or REST because this attribute is already multi-valued, for example:  

"sun-idrepo-ldapv3-config-ldap-server":["localhost:51389","example.com:1389"],

However, the order in which they are specified is now respected.

Solution

This issue can be resolved by updating your ssoadm scripts or commands to use sun-idrepo-ldapv3-config-ldap-server=[0], sun-idrepo-ldapv3-config-ldap-server=[1] and so on, instead of sun-idrepo-ldapv3-config-ldap-server.

See Also

How do I create a user data store in AM/OpenAM (All versions) using ssoadm?

Setup and Maintenance Guide › Setting Up Identity Data Stores

Related Training

N/A

Related Issue Tracker IDs

OPENAM-5867 (Data Store LDAP server (admin-ordered) list is reordered by OpenAM)


Insufficient Access Rights error in AM (All versions) and OpenAM 13.x when using a Generic LDAPv3 data store

The purpose of this article is to provide assistance if you encounter an "Insufficient Access Rights: The request control with Object Identifier (OID) '1.3.6.1.4.1.36733.2.1.5.1' cannot be used due to insufficient access rights" error in AM/OpenAM when using a Generic LDAPv3 (LDAPv3) data store, such as Oracle® Unified Directory (OUD).

Symptoms

The following error is shown when using a Generic LDAPv3 (LDAPv3) data store:

amAuthLDAP:27/10/2016 11:12:03:599 AM CST: Thread[ajp-apr-8009-exec-5,5,main]: TransactionId[475c572e-f336-42ff-b6cd-85037de38dfa-94] 
WARNING: resultCode: Connect Error 
amAuthLDAP:27/10/2016 11:12:03:599 AM CST: Thread[ajp-apr-8009-exec-5,5,main]: TransactionId[475c572e-f336-42ff-b6cd-85037de38dfa-94] 
WARNING: Cannot connect to [host2.example.com:389] 
org.forgerock.opendj.ldap.ConnectionException: Connect Error: No operational connection factories available 
   at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:163) 
   at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:124) 
   at org.forgerock.opendj.ldap.AbstractLoadBalancingAlgorithm.getMonitoredConnectionFactory(AbstractLoadBalancingAlgorithm.java:343) 
   at org.forgerock.opendj.ldap.AbstractLoadBalancingAlgorithm.access$100(AbstractLoadBalancingAlgorithm.java:59) 
   at org.forgerock.opendj.ldap.AbstractLoadBalancingAlgorithm$MonitoredConnectionFactory.getConnection(AbstractLoadBalancingAlgorithm.java:88) 
   at org.forgerock.opendj.ldap.LoadBalancer.getConnection(LoadBalancer.java:55) 
   at org.forgerock.openam.ldap.LDAPAuthUtils.getAdminConnection(LDAPAuthUtils.java:459) 
   at org.forgerock.openam.ldap.LDAPAuthUtils.searchForUser(LDAPAuthUtils.java:707) 
   at org.forgerock.openam.ldap.LDAPAuthUtils.authenticateUser(LDAPAuthUtils.java:399) 
   at com.sun.identity.authentication.modules.ldap.LDAP.process(LDAP.java:335) 
... 
Caused by: org.forgerock.opendj.ldap.ConnectionException: Server Connection Closed: Heartbeat failed 
   at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:163) 
   at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:124) 
   at org.forgerock.opendj.ldap.LDAPConnectionFactory$4.handleException(LDAPConnectionFactory.java:510) 
... more 
Caused by: org.forgerock.opendj.ldap.AuthorizationException: Insufficient Access Rights: The request control with Object Identifier (OID) '1.3.6.1.4.1.36733.2.1.5.1' cannot be used due to insufficient access rights 
   at org.forgerock.opendj.ldap.LdapException.newLdapException(LdapException.java:155) 
... 20 more

Recent Changes

Upgraded to, or installed AM 5 or later.

Upgraded to, or installed OpenAM 13.x.

Implemented a Generic LDAPv3 type data store.

Causes

The Generic LDAPv3 type data store sends a TransactionID control with all LDAP requests, which has the criticality field set to FALSE. The TransactionID is specific to DS/OpenDJ; however, all LDAP V3 compliant data stores will ignore this control since the criticality field is set to FALSE.

This error will only present itself if your data store is not compliant per RFC4511 section 4.1.11.

Solution

This issue can be resolved by adding a global ACI to allow access to the '1.3.6.1.4.1.36733.2.1.5.1' OID such as the following:

dsconfig set-access-control-handler-prop --add global-aci:'(targetcontrol="1.3.6.1.4.1.36733.2.1.5.1")(version 3.0; acl "TransactionIdControl OpenAM control"; allow (read) userdn="ldap:///anyone";)'
Note

The exact way in which this is done will be specific to your data store, which is outside the scope of ForgeRock support; if you want more tailored advice, consider engaging Professional Services

See Also

How do I know what the default Global ACIs are used for in DS/OpenDJ (All versions)?

Related Training

N/A

Related Issue Tracker IDs

OPENAM-8629 (TransactionID control should not be sent when using 'Generic LDAP' datastore)


Unable to load schema for plug-in error when configuring an LDAP server as a data store in AM/OpenAM (All versions)

The purpose of this article is to provide assistance if you are trying to configure an LDAP server as a data store in the AM/OpenAM console and it fails to connect with a schema error: "Unable to load schema for plug-in"; this happens when the Load schema option is selected. You may encounter this error with any of the data stores, such as DS/OpenDJ, Active Directory® or Tivoli.

Symptoms

You will see an error in the AM/OpenAM console when trying to save the configuration settings for a data store if Load Schema is enabled (Load Schema when saved option is selected in pre-AM 6) . The exact error will vary according to which data store you are configuring, for example:

  • DS/OpenDJ:
    Unable to load schema for plug-in OpenDj Directory Server for realm /. Connect Error: No operational connection factories available
    
  • Active Directory:
    Unable to load schema for plug-in Active Directory for realm /. Connect Error: No operational connection factories available
    
  • Tivoli:
    Unable to load schema for plug-in Tivoli for realm /. Connect Error: No operational connection factories available
    
Note

The description after Connect Error: may vary depending on which version of AM/OpenAM you are using and which details are incorrect; however, the "No operational connection factories available" description is the most common. Additionally, you may not see an error in the console if you have multiple servers and at least one of them is correct.

An error similar to the following is shown in the IdRepo log when this happens:

ERROR: An error occurred while trying to initiate persistent search connection
org.forgerock.openam.sm.datalayer.api.LdapOperationFailedException: 
CTS: Operation failed:
Result Code: Connect Error
Diagnostic Message: No operational connection factories available
Matched DN: 
   at org.forgerock.openam.sm.datalayer.providers.LdapConnectionFactoryProvider$LdapConnectionFactory.create(LdapConnectionFactoryProvider.java:169)
   at org.forgerock.openam.sm.datalayer.providers.LdapConnectionFactoryProvider$LdapConnectionFactory.create(LdapConnectionFactoryProvider.java:137)
   at com.iplanet.services.ldap.event.LDAPv3PersistentSearch.startQuery(LDAPv3PersistentSearch.java:168)
   at org.forgerock.openam.idrepo.ldap.DJLDAPv3Repo.addListener(DJLDAPv3Repo.java:2088)
   at com.sun.identity.idm.server.IdRepoPluginsCache.constructIdRepoPlugin(IdRepoPluginsCache.java:489)
   at com.sun.identity.idm.server.IdRepoPluginsCache.addIdRepo(IdRepoPluginsCache.java:355)
   at com.sun.identity.idm.server.IdRepoPluginsCache.removeIdRepo(IdRepoPluginsCache.java:268)
   at com.sun.identity.idm.server.IdRepoPluginsCache.organizationConfigChanged(IdRepoPluginsCache.java:648)
   at com.sun.identity.sm.ServiceConfigManagerImpl.notifyOrgConfigChange(ServiceConfigManagerImpl.java:505)
   at com.sun.identity.sm.ServiceConfigManagerImpl.objectChanged(ServiceConfigManagerImpl.java:465)
   at com.sun.identity.sm.SMSNotificationManager.sendNotifications(SMSNotificationManager.java:294)
   at com.sun.identity.sm.SMSNotificationManager$LocalChangeNotifcationTask.run(SMSNotificationManager.java:370)
   at org.forgerock.openam.audit.context.AuditRequestContextPropagatingRunnable.run(AuditRequestContextPropagatingRunnable.java:34)
   at com.iplanet.am.util.ThreadPool$WorkerThread.run(ThreadPool.java:314)
Caused by: org.forgerock.opendj.ldap.ConnectionException: Connect Error: Connection refused

Recent Changes

Configured a new external data store.

Changed the credentials for an existing data store.

Made network changes.

Causes

AM/OpenAM cannot communicate with, or connect to the LDAP server.

Solution

This issue can be resolved as follows:

  • Ensure you have the correct server credentials.
  • Ensure the LDAP server name is correct.
  • Ensure all the communications ports between AM/OpenAM and LDAP server are open.
  • Review firewall communications between AM/OpenAM and LDAP server.

See Also

AM/OpenAM (All versions) fails to connect to the user data store when anonymous access is disabled in DS/OpenDJ

Data stores in AM/OpenAM

​​​​​​​Setup and Maintenance Guide › Introducing Identity Data Stores

Related Training

N/A

Related Issue Tracker IDs

N/A


Your account has been locked error when authentication fails in OpenAM 13.5.1

The purpose of this article is to provide assistance if you encounter a "Your account has been locked" error when authentication fails in OpenAM 13.5.1, even though the user's account is not actually locked.

Symptoms

The user sees one of the following messages in the browser when they cannot authenticate:

Your account has been locked
This user is not active. Contact your system administrator

The following error is shown in the Authentication debug log when this happens:

amAuth:02/12/2018 10:07:29:742 AM GMT: Thread[http-nio-8080-exec-5,5,main]: TransactionId[ab90253e-462e-4225-8a20-97d329775296-384]
Exception : Your account has been locked.|user_inactive.jsp
amAuth:02/12/2018 10:07:29:743 AM GMT: Thread[http-nio-8080-exec-5,5,main]: TransactionId[ab90253e-462e-4225-8a20-97d329775296-384]
Error retrieving SSOToken :
com.iplanet.sso.SSOException: Session state is invalid. AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.* 
   at com.iplanet.sso.providers.dpro.SSOProviderImpl.createSSOToken(SSOProviderImpl.java:220)
   at com.iplanet.sso.providers.dpro.SSOProviderImpl.createSSOToken(SSOProviderImpl.java:184)
   at com.iplanet.sso.providers.dpro.SSOProviderImpl.createSSOToken(SSOProviderImpl.java:236)
   at com.iplanet.sso.SSOTokenManager.createSSOToken(SSOTokenManager.java:369)
   at com.sun.identity.authentication.service.LoginState.getSSOToken(LoginState.java:1862)
   at com.sun.identity.authentication.service.LoginState.logFailed(LoginState.java:4405)
   at com.sun.identity.authentication.service.LoginState.logFailed(LoginState.java:4352)
   at com.sun.identity.authentication.service.AMLoginContext.runLogin(AMLoginContext.java:753)
   at com.sun.identity.authentication.server.AuthContextLocal.submitRequirements(AuthContextLocal.java:617)
   at org.forgerock.openam.core.rest.authn.core.wrappers.AuthContextLocalWrapper.submitRequirements(AuthContextLocalWrapper.java:115)
   at org.forgerock.openam.core.rest.authn.core.LoginProcess.next(LoginProcess.java:173)
   at org.forgerock.openam.core.rest.authn.RestAuthenticationHandler.processAuthentication(RestAuthenticationHandler.java:262)
   at org.forgerock.openam.core.rest.authn.RestAuthenticationHandler.authenticate(RestAuthenticationHandler.java:167)
   at org.forgerock.openam.core.rest.authn.RestAuthenticationHandler.continueAuthentication(RestAuthenticationHandler.java:114)
   at org.forgerock.openam.core.rest.authn.http.AuthenticationServiceV1.authenticate(AuthenticationServiceV1.java:145)
...
Caused by: com.iplanet.dpro.session.SessionException: Session state is invalid. AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.* 
...

The user is actually unlocked despite the error (inetUserStatus = Active in DS/OpenDJ). You can check the physical account lockout status of a user in the DS/OpenDJ user store by querying the inetUserStatus attribute using the ldapsearch command as demonstrated in How do I enable account lockout in AM/OpenAM (All versions)? (Checking physical account lockout status).

Note

amadmin can log in because they are stored in the configuration data store rather than a user store.

Recent Changes

Upgraded to, or installed OpenAM 13.5.1.

Added an additional user store(s) to a realm.

Causes

This is known issue OPENAM-10233 (Authentication failing when multiple datastores in realm).

A recent change to fix OPENAM-9849 (isActive check should fail if the user is inactive in any of the configured data stores) did not take account of multiple user stores; this meant a user was treated as inactive if they were not found in the first user store; they were subsequently not searched for in the other user store(s). Users who exist in all user stores can authenticate.

Solution

This issue can be resolved by upgrading to OpenAM 13.5.2 or later; you can download this from BackStage.

Workaround

You can workaround this issue using one of the following approaches:

  • Ensure users exist in all user stores.
  • Remove any additional user stores to leave only one user store.

See Also

How do I remove the embedded DS/OpenDJ (All versions) after migrating to an external instance?

How do I enable account lockout in AM/OpenAM (All versions)?

How do I understand what the user data store is used for in AM/OpenAM (All versions)?

FAQ: Users in AM/OpenAM

Administrator and user accounts in AM/OpenAM

Related Training

N/A

Related Issue Tracker IDs

OPENAM-11791 (Authenticating with two or more datastore in OpenAM 13.5.1 , will result in "Your account has been locked" error message)

OPENAM-10233 (Authentication failing when multiple datastores in realm)

OPENAM-7871 (Document the use of multiple identity repository in the same realm)


CTS Store


Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)

The purpose of this article is to provide best practice advice on configuring an external DS/OpenDJ instance for CTS in AM/OpenAM as a non-admin user. CTS is used to provide session failover in AM/OpenAM.

Introduction

AM/OpenAM uses the CTS as the enabling technology for Session Failover (SFO). CTS leverages DS/OpenDJ to provide persistent and highly available storage for sessions, OAuth2 tokens and SAML2 tokens.

DS/OpenDJ is the only supported directory for CTS. By default CTS uses the same instances of DS/OpenDJ that holds the AM/OpenAM configuration. However, due to the very different profile of token data, an external DS/OpenDJ instance dedicated to CTS traffic is highly recommended as CTS traffic is ~90%+ writes and ~10% reads (unlike most LDAP traffic). Furthermore, the data is highly volatile and hence requires different sizing and tuning parameters to typical LDAP servers, which tend to host fairly static data optimized for reads.

This article guides you on how to configure such an external repository for CTS and integrate with AM/OpenAM using a non-admin user (that is, not the Directory Manager).

Note

Tuning, replication design and sizing is considered out of scope of this article, but must be considered in your environments.

AM 5.x

Key changes to be aware of:

You should refer to Installation Guide › General Recommendations for CTS Configuration for general recommendations about CTS configuration.

OpenAM 13.5.x and earlier

The Affinity type deployment architecture noted above is also available in OpenAM 13.5.1.

You should refer to OpenAM Installation Guide › General Recommendations for CTS Configuration for general recommendations about CTS configuration.

Overview of deployment steps

Note

The following steps were tested against OpenAM 13 with OpenDJ 3 and OpenAM 12 with OpenDJ 2.6.3.

The deployment is broken down into the following steps:

  1. DS/OpenDJ base build and CTS data (container and schema) import
  2. Non-admin user creation and ACI import
  3. CTS index import and rebuild
  4. Console configuration
  5. Testing session failover

In a replicated setup, you need to complete steps 2 and 3 on all servers as the ACI is a global-aci and the index is local to each server. Step 1 is only needed on one server as the schema is replicated by default.

Note

If you are using Amazon Web Services (AWS) to host AM/OpenAM and DS/OpenDJ, you should ensure your DS/OpenDJ servers are replicating prior to installing and configuring AM/OpenAM. If you configure replication after installing AM/OpenAM, you will need to restart AM/OpenAM before your setup functions as expected.

DS/OpenDJ base build and CTS data import

  1. Download the latest DS/OpenDJ zip file from BackStage.
  2. Unzip the DS/OpenDJ zip file and run the setup program. The following parameters have been used for the purpose of this article:
    Initial Root User DN for the Directory Server: cn=Directory Manager
    Password for the Initial Root User: [password]
    Fully Qualified Hostname: cts.example.com
    LDAP Listening Port: 3389
    Administration Connector Port: 5444
    Create Base DNs: Yes
    Provide the backend type: JE
    Base DN for Directory Data: dc=cts,dc=example,dc=com    
    Option for Populating Database: Only create base entry
    Do You Want to Enable SSL: no
    Do You Want to Enable Start TLS: no
    Do You Want To Start The Server: yes
    What Would You Like To Do: Set up server with parameters above
    Providing the backend type is necessary in OpenDJ 3.x; you can use either JE or PDB backend type. The PDB backend type is deprecated in DS 5 and removed in DS 6.
Note

Depending on your requirements, you may need to enable SSL and/or Start TLS.

  1. Check that the DS/OpenDJ instance is operational once the installation has finished.
  2. Navigate to: /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha, open the cts-container.ldif file and do a find and replace; find: @SM_CONFIG_ROOT_SUFFIX@ and replace with the Base DN defined during the DS/OpenDJ installation procedure (for example, dc=cts,dc=example,dc=com).
  3. For DS 5 and later: Update each entry in the cts-container.ldif file to include the following:
    changetype: add
    For example, an updated entry would look similar to this:
    dn: ou=tokens,dc=cts,dc=example,dc=com
    changetype: add
    objectClass: organizationalUnit
    objectClass: top
    ou: tokens
  4. Import the CTS container configuration using the following command depending on your version:
    • DS 5 and later:
      $ ./ldapmodify --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha/cts-container.ldif
    • Pre-DS 5:
      $ ./ldapmodify --defaultAdd --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL --filename /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha/cts-container.ldif
    You should see the following output:
    Processing ADD request for ou=tokens,dc=cts,dc=example,dc=com
    ADD operation successful for DN ou=tokens,dc=cts,dc=example,dc=com
    Processing ADD request for ou=openam-session,ou=tokens,dc=cts,dc=example,dc=com
    ADD operation successful for DN ou=openam-session,ou=tokens,dc=cts,dc=example,dc=com
    Processing ADD request for ou=famrecords,ou=openam-session,ou=tokens,dc=cts,dc=example,dc=com
    ADD operation successful for DN ou=famrecords,ou=openam-session,ou=tokens,dc=cts,dc=example,dc=com
    
  5. Add the CTS schema into the repository using the following command depending on your version:
    • DS 5 and later:
      $ ./ldapmodify --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha/cts-add-schema.ldif
    • Pre-DS 5:
      $ ./ldapmodify --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL --filename /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha/cts-add-schema.ldif
    You should see the following output:
    Processing MODIFY request for cn=schema
    MODIFY operation successful for DN cn=schema
    
Note

If AM/OpenAM is binding to CTS as the admin user (Directory Manager), you can skip the next section and proceed straight to the section: CTS index import and rebuild.

Non-admin user creation and ACI import

For best practice the use of the admin user is not recommended. Instead, a new and less privileged user than Directory Manager should be created and ACIs applied to allow the user read, search, modify and delete entries. You can also create a different user per AM/OpenAM instance if required.

The following section documents these step.

  1. Create an LDIF file called cts_user.ldif with the CTS user defined. The following sample LDIF creates a user called openam_cts:
    dn: ou=admins,dc=cts,dc=example,dc=com
    changetype: add
    objectClass: top
    objectClass: organizationalunit
    ou: OpenAM Administrator
    
    dn: uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com
    changetype: add
    objectClass: top
    objectClass: person
    objectClass: organizationalPerson
    objectClass: inetOrgPerson
    cn: OpenAM Administrator
    sn: OpenAM
    userPassword: [password]
    ds-privilege-name: update-schema
    ds-privilege-name: subentry-write
    ds-privilege-name: password-reset
    Ensure you replace the [password] tag with the actual password value if you use this sample as the basis of your LDIF file.
  2. Add the new user to the CTS repository using the following command depending on your version:
    • DS 5 and later:
      $ ./ldapmodify --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL .../cts_user.ldif
    • Pre-DS 5:
      $ ./ldapmodify --defaultAdd --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL --filename .../cts_user.ldif
    You should see the following output:
    Processing ADD request for ou=admins,dc=cts,dc=example,dc=com
    ADD operation successful for DN ou=admins,dc=cts,dc=example,dc=com
    Processing ADD request for uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com
    ADD operation successful for DN uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com
    
  3. Add a global ACI to allow the openam_cts user to modify schema using the following command:
    $ ./dsconfig set-access-control-handler-prop --hostname cts.example.com --port 5444 --bindDN "cn=Directory Manager" --bindPassword password --no-prompt --trustAll --add 'global-aci:(target = "ldap:///cn=schema")(targetattr = "attributeTypes || objectClasses")(version 3.0; acl "Modify schema"; allow (write) userdn = "ldap:///uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com";)'
  4. Check the Global ACI has been applied using the following command:
    $ ./dsconfig get-access-control-handler-prop  --hostname cts.example.com --port 5444 --bindDN "cn=Directory Manager" --bindPassword password --no-prompt --trustAll --property global-aci
    The following entry should be present:
    "(target = "ldap:///cn=schema")(targetattr = "attributeTypes || objectClasses")(version 3.0; acl "Modify schema"; allow (write) userdn = "ldap:///uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com";)",
  5. Create an LDIF file called cts_acis.ldif to add the ACIs to allow the CTS user to create, search, modify, delete and allow persistent search to the CTS repository. A sample LDIF file for this is:
    dn: dc=cts,dc=example,dc=com
    changetype: modify
    add: aci
    aci: (targetattr="*")(version 3.0;acl "Allow entry search"; allow (search, read)(userdn = "ldap:///uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com");)
    aci: (targetattr="*")(version 3.0;acl "Modify entries";  allow (write)(userdn = "ldap:///uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com");)
    aci: (targetcontrol="2.16.840.1.113730.3.4.3")(version 3.0;acl "Allow  persistent search"; allow (search, read)(userdn = "ldap:///uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com");)
    aci: (version 3.0;acl "Add config entry"; allow (add)(userdn = "ldap:///uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com");)
    aci: (version 3.0;acl "Delete entries"; allow (delete)(userdn = "ldap:///uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com");)
  6. Import the ACIs into the CTS repository using the following command depending on your version:
    • DS 5 and later:
      $ ./ldapmodify --hostname cts.example.com --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL .../cts_acis.ldif
      
    • Pre-DS 5:
      $ ./ldapmodify --hostname cts.example.com --port 1636 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL --filename .../cts_acis.ldif
      
    You should see the following output:
    Processing MODIFY request for dc=cts,dc=example,dc=com
    MODIFY operation successful for DN dc=cts,dc=example,dc=com
    

CTS index import and rebuild

  1. Navigate to: /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha, open the cts-indices.ldif file and do a find and replace; find: @DB_NAME@ and replace with the actual DB name (for example, userRoot).
  2. For DS 5 and later: Update each entry in the cts-indices.ldif file to include the following:
    changetype: add
    For example, an updated entry would look similar to this:
    dn: ds-cfg-attribute=coreTokenExpirationDate,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    changetype: add
    objectClass: top
    objectClass: ds-cfg-backend-index
    ds-cfg-attribute: coreTokenExpirationDate
    ds-cfg-index-type: ordering
  3. Apply the new indexes to the CTS repository depending on your version:
    • DS 5 and later:
      $ ./ldapmodify --port 5444 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha/cts-indices.ldif
    • Pre-DS 5:
      $ ./ldapmodify --defaultAdd --port 5444 --bindDN "cn=Directory Manager" --bindPassword password --UseSSL --trustALL --filename /path/to/tomcat/webapps/openam/WEB-INF/template/ldif/sfha/cts-indices.ldif
    You should see the following output:
    Processing ADD request for ds-cfg-attribute=coreTokenExpirationDate,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    ADD operation successful for DN ds-cfg-attribute=coreTokenExpirationDate,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    Processing ADD request for ds-cfg-attribute=coreTokenUserId,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    ADD operation successful for DN ds-cfg-attribute=coreTokenUserId,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    ……………
    ……………
    ……………
    ……………
    ds-cfg-attribute=coreTokenDate04,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    Processing ADD request for ds-cfg-attribute=coreTokenDate05,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    ADD operation successful for DN ds-cfg-attribute=coreTokenDate05,cn=Index,ds-cfg-backend-id=userRoot,cn=Backends,cn=config
    
Note

 These indexes may require further tuning subject to environmental load testing when used in production.

  1. Stop the DS/OpenDJ server so that you can perform the next step offline.
  2. Rebuild all indexes and verify using the following commands depending on your version:
    • DS 5 and later:
      $ ./rebuild-index --baseDN "dc=cts,dc=example,dc=com" --rebuildAll --offline
      
      $ ./verify-index --baseDN "dc=cts,dc=example,dc=com"
    • Pre-DS 5:
      $ ./rebuild-index --baseDN "dc=cts,dc=example,dc=com" --rebuildAll
      
      $ ./verify-index --baseDN "dc=cts,dc=example,dc=com"
  3. Restart the DS/OpenDJ server and continue on to the AM/OpenAM console configuration.

AM/OpenAM console configuration

Complete the following configuration steps in the console to integrate the new CTS repository with AM/OpenAM. Alternatively, you can perform these configuration steps using ssoadm as described in How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm?

Assumptions

Configuration

  1. Configure the CTS repository in AM/OpenAM:
    • AM / OpenAM 13.5 console: navigate to: Configure > Server Defaults > CTS and complete the required details on both tabs.
    • Pre-OpenAM 13.5 console: navigate to: Configuration > Servers and Sites > Default Server Settings > CTS, select the External Token Store option and complete the required details.
    The following settings have been used for the purpose of this article; see Reference › Configuration Reference > CTS Token Store (AM 5 and later) or OpenAM Reference › Configuration Reference > CTS (OpenAM 13.5.x and earlier) for further information on these settings:
    Root Suffix: dc=cts,dc=example,dc=com 
    SSL/TLS Enabled: leave this option switched off
    Connection String(s): cts.example.com:3389
    Login Id: uid=openam_cts,ou=admins,dc=cts,dc=example,dc=com
    Password: password
    Max Connections: 10 (this is the default value and should be tuned for Production depending on the authentication profile)
    Heartbeat: 10 (this is the default value and should be tuned for Production depending on the network and physical architecture)
    Affinity Enabled: leave this option switched off (only available in OpenAM 13.5.1 / AM 5 and later)
    
    Click Save to save these changes.
Note

By default, the AM/OpenAM to DS/OpenDJ CTS connection string is inherited by all AM/OpenAM instances. It can also be configured on an instance by instance basis as discussed below, although in an Affinity deployment, all instances must share the same connection string.

  1. In OpenAM 13.5.x and earlier: Configure the secondary configuration instance in OpenAM: 
    • OpenAM 13.5: navigate to: Configure > Global Services > Session > Secondary Configuration Instance. Click New and select the Site from the drop-down list.
    • Pre-OpenAM 13.5 console: navigate to: Configuration > Global > Session > Secondary Configuration Instance. Click New and select the Site from the drop-down list.
    Complete the following settings:
    Session persistence and High Availability Failover: select the Enabled option
    Reduced Crosstalk: select the Enabled option
    Session Logout/Destroy Broadcast: Disabled
    Reduced Crosstalk Purge Delay: 1
    
    Click Add to save these changes.
  2. Restart all AM/OpenAM servers in the site and test your configuration.

Instance specific Connection String (active/passive deployments only)

You can configure the connection string on an instance by instance basis as follows: you must repeat this on all instances:

  • AM / OpenAM 13.5 console: navigate to: Deployment > Servers > [Server Name] > CTS > External Store Configuration, click the padlock against Connection String(s) to remove inheritance and set the connection string for this server. The connection string can also be multi-valued and ordered (using commas) to split CTS traffic by instances, limit traffic by geography and/or specify failover servers.
  • Pre-OpenAM 13.5 console:
    1. Navigate to: Configuration > Servers and Sites > [Server Name] > Inheritance Settings and deselect the Connection String property. Click Save to save these changes. The connection string is now specific to this server.
    2. Navigate to: Configuration > Servers and Sites > [Server Name] > CTS and amend the connection string for this server as required. The connection string can also be multi-valued and ordered (using commas) to split CTS traffic by instances, limit traffic by geography and/or specify failover servers.

For example:

  • Single CTS DS/OpenDJ instance or LB:
    cts.example.com:3389
    In this scenario, all instances of AM/OpenAM will use this address - this address could be a server or a load balancer intelligently managing traffic to multiple DS/OpenDJ CTS instances.
  • Single CTS DS/OpenDJ Instance no LB:
    cts-1.example.com:3389, 
    cts-2.example.com:3389
    
    In this scenario, all instances of AM/OpenAM will go to cts-1 unless that DS/OpenDJ server goes down; in which case, all AM/OpenAM instances will go to cts-2.
  • Multi-DS/OpenDJ Instance per AM/OpenAM Limited to Site:
    cts-1.example.com:3389|server1|site1,
    cts-2.example.com:3389|server1|site1,
    cts-2.example.com:3389|server2|site1,
    cts-1.example.com:3389|server2|site1,
    cts-3.example.com:3389|server1|site2,
    ...
    In this scenario, server 1 of site 1 will go to cts-1 as it's the primary DS/OpenDJ CTS server with cts-2 as its failover. Server 2 in site 1 will go to cts-2 as it's the primary server with cts-1 as its failover. Server 1 in site 2 will go to cts-3 as it's the primary server and so on.
Note

It is clear from these examples that the connection string value is highly configurable and flexible depending on the target environment.

Testing session failover

This article uses two browsers to test failover: Chrome™ and Firefox®. You can use any two browsers or browsers running in incognito mode. Other approaches, such as viewing tokens using an LDAP browser, could also be used.

  1. In Chrome, log into the second AM/OpenAM instance with the amadmin user and click on Sessions.
  2. In Firefox, log into the first AM/OpenAM instance with a test user.
  3. In Chrome, verify the test user exists in the first AM/OpenAM instance’s session list and not in the second instance.
  4. Shutdown the first AM/OpenAM instance.
  5. In Firefox, rewrite the URL to point to the second AM/OpenAM instance. If successful, the browser should not prompt for login.
  6. Confirm the session has failed over - in Chrome, list the sessions on the second instance and verify that the test user’s session is present.
  7. Restart the first AM/OpenAM instance to complete the testing.

See Also

Best practice for using Core Token Service (CTS) Affinity based load balancing in AM (All versions) and OpenAM 13.5.1, 13.5.2

How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm?

FAQ: Core Token Service (CTS) and session high availability in AM/OpenAM

Core Token Service (CTS) and sessions in AM/OpenAM

Installation Guide › Implementing the Core Token Service

Related Training

ForgeRock Access Management Core Concepts (AM-400)

Related Issue Tracker IDs

OPENAM-10383 (Validation of External CTS store using OpenAM GUI)

OPENAM-10382 (If we set up External CTS wrongly then OpenAM fails catastrophically on start up.)

OPENAM-9762 (Introduce Explicit Health Monitoring for the CTS Reaper)

OPENAM-9738 (Enable CTS segregation to allow each token type to write to a different CTS instance)

OPENAM-8210 (Enhance CTS to persist tokens across multiple OpenDJ instances rather than a single primary OpenDJ instance by some form of sharding)

OPENAM-8202 (If the "Login Id" in the External Store Configuration(CTS) is set to incorrect value,CoreSystem debug log is full of duplicate error)

OPENAM-7466 (Get CTS total tokens using SNMP monitoring fails )

OPENAM-5478 (CTS Reaper auto negotiation)

OPENAM-3510 (all tokens (session,saml,oauth) are stored below the same entry when external CTS is used)

OPENAM-3341 (CTS Root Suffix Issue)


Best practice for using Core Token Service (CTS) Affinity based load balancing in AM (All versions) and OpenAM 13.5.1, 13.5.2

The purpose of this article is to provide best practice advice on using CTS Affinity based load balancing in AM/OpenAM. Affinity deployments allow AM/OpenAM to connect to multiple master directory server instances, with each instance acting as the master for a subset of CTS tokens. In this architecture, CTS tokens are described as having an affinity for a given directory server instance.

Overview

The ForgeRock Core Token Service (CTS) is a specific DS/OpenDJ repository used to store and persist AM/OpenAM SSO, OAuth2/OIDC and SAML tokens. In terms of architecture, typical primary/secondary aka active/standby or all active 1:1 AM/OpenAM topologies are adopted for AM/OpenAM to CTS connections (described in the Traditional AM/OpenAM - CTS topology section). Both of these approach have inherent limitations, namely:

  • Active/standby topologies do not maximize available hardware (only the primary node services requests) and horizontal scaling of the CTS pool of servers is not possible.
  • Active/standby topologies require expensive vertically scaled instances under high load.
  • 1:1 all active CTS topologies require stickiness for all session requests to the server which created the token or replication delay becomes a functional issue.

Affinity based load balancing for CTS elegantly resolves these problems.

Topics

This article covers the following topics:

Traditional AM/OpenAM - CTS topology

Assuming CTS connections strings are used; the recommended approach to avoid the need for a load balancer and allow AM/OpenAM to manage its own connections is to typically use one of two AM/OpenAM - CTS topologies - Primary/Secondary (aka Active/Standby) or 1:1 all active. These are described below.

Mode 1 - Primary/Secondary 

All AM/OpenAMs in the pool communicate with a single CTS server. If this server fails, all connections failover to the secondary CTS server and so on. The following topology depicts this architecture:

In this mode, replication delay is not an issue as all traffic hits the primary and as long as replication happens successfully over time; functionality is unaffected.

A connection string for this mode would look like this:

cts1.example.com:1636,cts2.example.com:1636

Issues:

  • In an high load environment, CTS needs to be vertically scaled to ensure a single server can handle all requests. This is extremely costly for both on-prem and cloud based deployments.
  • It does not make efficient use of the available hardware; only one server is handling traffic, while the others are sat idle awaiting primary failure.

Mode 2 - All Active CTS with 1:1 Mapping between AM/OpenAM and CTS nodes

Each AM/OpenAM communicates with its own CTS server, if this node fails it connects to its failover CTS. For example, AM1 connects to CTS1 as its primary with CTS2 as it secondary, AM2 connects to CTS2 as its primary and CTS1 as its secondary. The following topology depicts this architecture:

This mode allows an all active CTS farm to maximize hardware usage. This approach must ensure end-to-end stickiness to the AM/OpenAM node that created the session/token for all interactions to prevent functional issues. If this cannot be ensured, this is not a viable option.

The connection string would look like the following, where the serverID for AM1=01 and AM2=02.

cts1.example.com:1636|01,cts2.example.com:1636|01,cts2.example.com:1636|02, 
cts1.example.com:1636|02 

Issues:

  • An all active architecture like this is highly susceptible to replication delay if stickiness to the AM/OpenAM node that created a given session cannot be guaranteed. DS/OpenDJ replication is based on a “loose” algorithm where all CTS nodes will fully converge over time, but not instantly. This means under high load a token create may hit AM1 and CTS1, however AM2 (which is connected to CTS2) may be hit for the token validate request. If replication has not happened fast enough, the request would error.

What is CTS Affinity based load balancing?

Affinity based load balancing for CTS addresses the key issues highlighted with the more traditional approaches; namely, unlimited horizontal scaling of the CTS pool using smaller, cheaper instances and eliminating functional issues as a result of replication delay.

It does this by making use of the new affinity based load balancing algorithm built into the DS/OpenDJ SDK deployed with AM/OpenAM. For each and every inbound token DN, the SDK creates a hash and allocates the result to a specific CTS DS/OpenDJ instance in the CTS connection string. All AM/OpenAM servers in the pool compute the same hash and thus send the request to the same CTS instance. The next request for another token DN is hashed and may be sent to another CTS instance in the pool and so on. The end result of this is, for each and every token a create occurs on a specific CTS instance (the token origin server) and from that point on all read, update and delete operations will be sent to this same origin server from any AM/OpenAM node. The following topology depicts this architecture:

The DS/OpenDJ SDK also makes sure token creates are spread close to evenly across all CTS nodes in the pool to ensure one CTS server is not overloaded with requests, while the others remain idle. Finally, the DS/OpenDJ SDK is instance aware; if the origin server goes down, the request goes to another in the pool and remains sticky to that CTS instance from then on. Assuming replication has happened there will be no functional impact. When the original CTS server comes back online, requests fail back to that server for any requests where it is the origin servers.

The connection string for affinity simply lists all CTS instances as a comma separate list:

cts1.example.com:1636,cts2.example.com:1636
Caution

It is imperative that all AM/OpenAM servers share the same connection string; affinity will fail if each AM/OpenAM server changes the ordering of the connection string.

What are the advantages of CTS Affinity based load balancing?

The primary advantages of CTS Affinity based load balancing over the traditional topologies are:

  • Allows horizontal scaling of the CTS server farm, this is not possible with the traditional topologies.
  • Maximizes available resources by introducing an all active CTS pool of servers.
  • Allows smaller, cheaper nodes for handling CTS traffic with the all active CTS architecture, rather than expensive vertically scaled nodes.
  • Combats replication delay between CTS nodes.
  • Removes the need for end-to-end stickiness to AM/OpenAM for session activities (stickiness is still required for authentication as the AuthID is held in the memory of the AM/OpenAM server that was first hit for the ../json/authenticate call; see OPENAM-8336 (XUI+REST authentication with chains must have sticky load balancing) for further information).

How do you configure CTS Affinity based load balancing?

CTS Affinity based load balancing can be configured using Amster, ssoadm and the console. The console approach is described below:

  1. Build an external CTS repository per the following guides: Installation Guide › CTS Deployment Steps and Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions) .
  2. As the connection string for CTS Affinity needs to be exactly the same across all AM/OpenAM nodes, it is recommended to make the necessary CTS changes at the Server Default level rather than at an instance level to ensure all AM/OpenAM nodes inherit the settings from these defaults. Navigate to: Configure > Server Defaults > CTS.
  3. Configure the following settings:
    • Change the Store Mode to External Token Store.
    • Set the Root Suffix to that configured in step 1.
    • Set Max Connections as appropriate depending on your version: 
      • AM 5 and later: Each AM will create a small number of connections on startup to each CTS node in the pool. As load increases, the number of connections from each AM to each CTS node will increase up to this Max Connection setting. In addition, the first CTS node in the connection string will receive additional connections for the AM Reaper and Blacklisting threads. For example, if Max Connections is set to 20 and there are 2 AM nodes and 2 CTS nodes:
        • AM1 will create a maximum of 20 connections to CTS1* and a maximum of 20 connections to CTS2.
        • AM2 will create a maximum of 20 connections to CTS1* and a maximum of 20 connections to CTS2.
        * CTS1 will however receive additional connections from each AM for the AM Reaper and AM Blacklisting threads, so total connections from AM1 and AM2 to CTS1 will be greater than 20. The Reaper and Blacklisting threads execute on the first CTS node in the connection list only, unless it is down in which case it will create a new set of connections to the next CTS in the CTS connection string list.
      • Pre-AM 5: The Max connections is shared as follows; 1 connection for CTS cleanup Reaper and the rest shared equally between the nodes specified in the connection string. For example, if there are 2 CTS nodes and the connection string is set to 21; 10 connections will be allocated to CTS1; 10 to CTS2 and 1 reserved for the reaper. Load testing will determine optimal value - increase as appropriate. 

 

  1. Click Save Changes
  2. Select the External Store Configuration tab and configure the following settings: 
    • Enable SSL/TLS Enabled if CTS is LDAPS enabled.
    • Define the Connection String with a comma separated list in the format <server FQDN>:<port>. An example connection string is:
      cts1.example.com:1636,cts2.example.com:1636
    • Set the Login Id and Password as appropriate.
    • Enable Affinity Enabled.

  1. Click Save Changes.
  2. Restart the AM/OpenAM node(s). Configuration complete.
Note

If there are problems be sure to check out AM/OpenAM’s debug logs - usually it is something like the bind credentials to connect to CTS are wrong. See How do I enable Message level debugging in AM/OpenAM (All versions) debug files?

What about multi-data center deployments?

As always the optimal AM/OpenAM to CTS topology for a multi-data center (DC) deployment is it depends. For example, is failover across DCs really required? The worst case is a user needs to re-authenticate, there is cost and complexity for DC SSO and OAuth2 session failover. Another consideration may be latency; for example, some customers have dedicated communication lines which guarantee low latency between their DCs; others are subject to public internet conditions. These considerations and more should be factored in when deciding on which CTS architecture to go with.

However, if for example, inter-DC failover for SSO and OAuth2 sessions is required and latency levels are guaranteed, the standard affinity approach could be used, whereby all CTS nodes from both sites are set in the connection string and AM/OpenAM sees the CTS pool as one collection of servers and is unaware that some CTS nodes are local and some remote. Example connection string: 

cts1-dc1.example.com:1636,cts2-dc1.example.com:1636,cts1-dc2.example.com:1636,cts2-dc2.example.com:1636

If however latency levels cannot be guaranteed, then perhaps affinity within a site would be more appropriate, whereby the connection string is set at an instance level for all CTS nodes within a particular DC. However, this approach would be subject to replication delay for token replication between DCs; if the data does not make the remote DC quick enough and stickiness to a particular DC cannot be guaranteed, there is a chance for failure at the token validation phase. Example connection string:

AM1 and AM2 in DC1: cts1-dc1.example.com:1636,cts2-dc1.example.com:1636
AM1 and AM2 in DC2: cts1-dc2.example.com:1636,cts2-dc2.example.com:1636

Multi-DC deployments are complex; ForgeRock Professional Services are available to help customers to architect, design and build in the best way. For more information, please contact ps@forgerock.com   

See Also

Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)

How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm?

FAQ: Core Token Service (CTS) and session high availability in AM/OpenAM

Installation Guide › Implementing the Core Token Service

Installation Guide › General Recommendations for CTS Configuration

Related Training

N/A

Related Issue Tracker IDs

N/A


How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm?

The purpose of this article is to provide information on configuring an external Core Token Service (CTS) token store in AM/OpenAM using ssoadm.

Prerequisites

The external CTS token store must already exist (and you just want to modify its configuration using ssoadm) or you must take steps to prepare the external OpenDJ instance for CTS first if you want to automate the entire process.

The Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)  article guides you through all the steps needed to configure an external OpenDJ instance for CTS. In summary, these steps are:

  1. OpenDJ base build and CTS data (container and schema) import.
  2. Non-admin user creation and ACI import.
  3. CTS index import and rebuild.
  4. AM/OpenAM console configuration.
  5. Testing session failover.

You must complete steps 1 to 3 prior to configuring an external CTS token store in AM/OpenAM. You can then use these instructions to complete step 4 using ssoadm instead of using the console if you wish.

See Installation Guide › Implementing the Core Token Service for further information.

Configuring an external CTS token store

Note

The CTS token store is the authoritative source for sessions (as of AM 5). This means you will lose access to the console if you misconfigure the CTS store. As such, it is highly recommended that you have up to date backups prior to configuring the external CTS store in case of misconfiguration. See How do I make a backup of configuration data in AM/OpenAM (All versions)? for further information.

You can configure an external CTS token store using ssoadm as follows:

  1. Create a data file (called DATA_FILE to match the next command) and include the following properties with appropriate values (example values are shown): 
    ​org.forgerock.services.cts.store.directory.name=host.domain:60589
    org.forgerock.services.cts.store.heartbeat=10
    org.forgerock.services.cts.store.location=external
    org.forgerock.services.cts.store.loginid=uid=user
    org.forgerock.services.cts.store.max.connections=10
    org.forgerock.services.cts.store.password=password
    org.forgerock.services.cts.store.root.suffix=dc=example,dc=com
    org.forgerock.services.cts.store.ssl.enabled=false
  2. Enter the following command:
    $ ./ssoadm update-server-cfg -s [serverName] -u [adminID] -f [passwordfile] -D DATA_FILE
    
    replacing [serverName], [adminID] and [passwordfile] with appropriate values. You can use default for [serverName] if you want to change the Default Server Settings.
  3. Restart the web application container in which AM/OpenAM runs.

If you only want to update some of the configuration settings, you can specify just the attributes you want to change. For example, to change the maximum number of connections, you could use the following ssoadm command:

$ ./ssoadm update-server-cfg -s default -u amadmin -f pwd.txt -a org.forgerock.services.cts.store.max.connections=17

See Also

Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)

FAQ: Core Token Service (CTS) and session high availability in AM/OpenAM

Login to AM/OpenAM console (All versions) fails for amadmin user

FAQ: Installing and using ssoadm in AM/OpenAM

Installation Guide › CTS Configuration

Installation Guide › Implementing the Core Token Service

Setup and Maintenance Guide › Tuning LDAP CTS and Configuration Store Settings

Related Training

N/A

Related Issue Tracker IDs

OPENAM-4673 (External CTS Root DN is hardcoded)


FAQ: Session crosstalk and the Core Token Service (CTS) in OpenAM

The purpose of this FAQ is to provide answers to commonly asked questions regarding session crosstalk and the CTS in OpenAM. Crosstalk only applies to OpenAM 13.5.x and earlier. As of AM 5.0, crosstalk and the concept of a home server have been removed.

Frequently asked questions

Q. What is session crosstalk?

A. When a user authenticates, the session is created on the OpenAM server and is cached in memory on that server; this means that a session can only be validated on the originating (Home) server. In a multi-server deployment, a session may need to be validated on a different server to the one receiving the request. In this situation, the server receiving the request will make a HTTP request via a back channel to the Home server to retrieve the session. This mode of communication between servers for the purpose of validating sessions is referred to as session crosstalk. 

Any requests which update session objects such as log out, reset idle times or set a session attribute always use crosstalk to ensure the integrity. 

Q. What is the Home server?

A. The Home server is the server on which the session was created (the originating server). This may be the local server or a remote server depending on which server is receiving the request.

Note

As of AM 5.0, the concept of a Home server no longer applies; see Release Notes › What's New › New Features for further information on these changes. 

Q. How does OpenAM identify the Home server?

A. OpenAM uses the information contained in the session cookie (iPlanetDirectoryPro cookie) to identify the originating site and server as described in FAQ: Cookies in AM/OpenAM (Q. What information is contained in the AM/OpenAM session cookie?).

Q. When is session crosstalk used?

A. Session crosstalk happens when the session originates from a different OpenAM instance to one receiving the request.

Typically, OpenAM tries to resolve sessions in the following order upon receiving a request:

  1. Retrieves the session from the Home server as follows, in order:
    1. Looks up the cached session in memory if either it originates from the OpenAM instance receiving the request or it has been cached from a previous crosstalk attempt.
    2. Uses crosstalk to resolve the session on the remote server where the session originated if the remote server is up and running. The check of whether a server instance is alive is done by ClusterStateService. If the remote server is not running, OpenAM will try to find the next candidate. The retrieved session is then cached on the requesting server (non-Home server) to enable future requests to be resolved quickly. This session is stored per the Maximum Caching Time setting as detailed in How do I change the Maximum Caching Time in AM 5.x, 6 and OpenAM 12.x, 13.x? Additionally, a callback listener is registered with the Home server to enable notifications about session changes to be received to ensure the cached session does not become out-of-date.
  2. Looks up the session in the CTS store if all the other remote servers are down. If the other remote servers are running, AM/OpenAM will try to find the next candidate and perform crosstalk. This behavior is different when the Reduced Crosstalk option is set:
    1. When Reduced Crosstalk option is set, AM/OpenAM first tries to retrieve the session from the CTS for read operations. Any operation that requires the session state to be changed (e.g. Logout, set property , or reset idle time) will need to be delegated to the authoritative server (home server) and this is achieved using crosstalk.
    2. You should not disable reduced crosstalk unless advised to do so by ForgeRock Support. 
Note

Requests to update sessions, such as requests to log out, reset the session idle time, or set a session attribute, always use crosstalk to ensure the integrity of the update requests. See  Request Forward Caveats for further details including load balancer considerations. 

Q. How do OpenAM servers communicate with each other for crosstalk?

A. OpenAM uses the server URL for back channel communication. You should ensure that all OpenAM instances can communicate with each other via this URL. The URIs are:

  • /sessionservice for session related requests.
  • /namingservice for ClusterStateService.

You can check what port etc they are communicating on as follows:

  • OpenAM 13.5 console: navigate to Deployment > Servers.
  • Pre-OpenAM 13.5 console: navigate to Configuration > Servers and Sites.

For crosstalk, OpenAM performs a HTTP request against the remote Home server's sessionservice PLL endpoint (sent as XML) and receives back the session details in XML format (SessionInfo).

Q. How can I tell if crosstalk is being used?

A. You will see POST requests being made to the sessionservice endpoint from one OpenAM server to another. For example, you will see something similar to the following in the Apache Tomcat™ access logs, which in this example shows the call originating from IP address: 198.51.100.20:

198.51.100.20 - - [21/Jun/2017:10:07:11 +0100] "POST /openam/sessionservice HTTP/1.1" 200 403

You will also see requests being sent and received in the Session logs, for example:

PLLClient:03/06/2017 09:22:34:656 PM GMT: Thread[http-bio-8443-exec-23,5,main]
Send RequestSet Session XML :
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<RequestSet vers="1.0" svcid="session" reqid="198">
<Request><![CDATA[<SessionRequest vers="1.0" reqid="127">
<GetSession reset="true">
<SessionID>AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.*</SessionID>
</GetSession>
</SessionRequest>]]></Request>
</RequestSet>
PLLClient:03/06/2017 09:22:34:656 PM GMT Thread[http-bio-8443-exec-23,5,main]
Response XML :
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<ResponseSet vers="1.0" svcid="session" reqid="198">
<Response dtdid=""><![CDATA[<SessionResponse vers="1.0" reqid="127">
<GetSession>
<Session sid="AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.*" stype="user" cid="uid=user1, ou=employees, ou=People, dc=example, dc=com" cdomain="o=users,ou=services,dc=amconfig,dc=example,dc=com" maxtime="900" maxidle="120" maxcaching="3" timeidle="2" timeleft="53980" state="valid">
<Property name="CharSet" value="UTF-8"></Property>
<Property name="UserId" value="user1"></Property>
<Property name="FullLoginURL" value="/openam/XUI/#login"></Property>
<Property name="successURL" value="/openam/XUI/#profile/></Property>
<Property name="cookieSupport" value="true"></Property>
<Property name="AuthLevel" value="0"></Property>
<Property name="SessionHandle" value="shandle:AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.*"></Property>
<Property name="UserToken" value="user1"></Property>
<Property name="loginURL" value="/openam/XUI/#login"></Property>
<Property name="maxTokenLifeMinutes" value="10080"></Property>
<Property name="IndexType" value="service"></Property>
<Property name="Principals" value="uid=user1, ou=employees, ou=People, dc=example, dc=com"></Property>
<Property name="Service" value="ldap"></Property>
<Property name="PostAuthProcessInstance" value="org.forgerock.openam.authentication.modules.persistentcookie.PersistentCookieAuthModule"></Property>
<Property name="sun.am.UniversalIdentifier" value="id=user1,ou=user,o=users,ou=services,dc=amconfig,dc=example,dc=com"></Property>
<Property name="amlbcookie" value="02"></Property>
<Property name="Organization" value="o=users,ou=services,dc=amconfig,dc=example,dc=com"></Property>
<Property name="Locale" value="en_GB"></Property>

A key line to note in the above log extract is the following, since reset="true" is what triggers crosstalk:

<GetSession reset="true">

If you see reset="false" instead, this means the CTS store will be used.

Q. How can I tell if the session is being retrieved from a local or remote server?

A. You can tell from the Session logs if the session is local or remote as follows:

  • Local:
    amSession:06/11/2017 03:29:01:534 AM GMT: Thread[tomcat-http--13,5,main]: TransactionId[d9f5bb96-71e6-463e-11755-2f7ea3950543-63]
    Local fetch SessionInfo for AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.*
    
  • Remote:
    amSession:06/11/2017 03:29:01:534 AM GMT: Thread[tomcat-http--13,5,main]: TransactionId[d9f5bb96-71e6-463e-11755-2f7ea3950543-63]
    Remote fetch SessionInfo for AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.*
    

Q. What impact does crosstalk have on network traffic?

A. Crosstalk generates network traffic, which can have an impact on performance. The HTTP requests made between servers are blocking calls, so you have an outgoing and incoming request when crosstalk is happening, and potentially a third request if the crosstalk is occurring between sites as well. This means a single user request can keep two or three request processing threads busy at a time. 

It is important to tune your timeout settings according to your network speed or latency; in particular, you need to set the following advanced server properties (in milliseconds) appropriately for your deployment:

com.sun.identity.url.readTimeout              (default is 30000 ms)
org.forgerock.openam.url.connectTimeout       (default is 10000 ms)

You can set these properties using either the console or ssoadm:

  • OpenAM 13.5 console: navigate to Deployment > Servers > [Server Name] > Advanced and add the required property and value.
  • Pre-OpenAM 13.5 console: navigate to Configuration > Servers and Sites > [Server Name] > Advanced and add the required property and value.
  • ssoadm: enter the following command:
    $ ./ssoadm update-server-cfg -s [serverName] -u [adminID] -f [passwordfile] -a [property=value]
    replacing [serverName], [adminID], [passwordfile] and [property=value] with appropriate values.

If these timeouts are set too low to allow a remote server to respond, crosstalk will fail and you will see errors such as the following in your logs:

  • CoreSystem log:
    amXMLHandler:06/11/2017 03:29:01:532 AM GMT: Thread[http-nio-8080-exec-41,5,main]
    Error Response String : <Exception errorCode="Read timed out"></Exception>
    PLLClient:06/11/2017 03:29:01:532 AM GMT: Thread[http-nio-8080-exec-34,5,main]
    WARNING: PLLClient.send: exception: 
    java.net.SocketTimeoutException: Read timed out
  • Sessions log:
    amSession:06/11/2017 03:29:01:532 AM GMT: Thread[SystemTimer,5,main]: TransactionId[d9f5bb96-71e6-463e-11755-2f7ea3950543-63]
    ERROR: 20218:ClusterStateService.checkServerUp():: timeout too small 10000
    
    amSession:06/11/2017 03:29:01:534 AM GMT: Thread[SystemTimer,5,main]: TransactionId[d9f5bb96-71e6-463e-11755-2f7ea3950543-63]
    ERROR: 20218:ClusterStateService.checkServerUp():: error while checking server 01
    java.net.SocketTimeoutException: connect timed out
    amSession:06/11/2017 03:29:01:534 AM GMT: Thread[tomcat-http--13,5,main]: TransactionId[d9f5bb96-71e6-463e-11755-2f7ea3950543-63]
    processSessionRequest caught exception: connect timed out
    com.iplanet.dpro.session.SessionException: connect timed out

Q. What happens if the CTS store is down?

A. This depends on whether the Reduced Crosstalk option is set or not: 

  • If it is not set, OpenAM should retrieve the session from the Home server as described in Q. When is session crosstalk used? 
  • If it is set, OpenAM will try to retrieve the session from the CTS store but will fail.

Q. Are policy agent sessions stored in the CTS store?

A. No, policy agent sessions (Application SSOSession) are not stored in the CTS store by default as they are not subject to session failover. Therefore, crosstalk is always used with policy agent sessions.

However, if the Reduced Crosstalk option is set, policy agent session validation will always fail since the session does not exist in the CTS store and crosstalk is not used for agent sessions when this option is set. When this happens, you will see errors such as the following in the agent logs (debug.log or amAgent):

2017-06-09 11:03:47.002 Error 4786:7fc8f789bae0 AM_SSO_SERVICE: SSOTokenService::getSessionInfo(): Error 35 for sso token ID AQIC5wM2LY4Sfcxwo1cOJAMRkbIsn0bS8Pm1IB623MLBCnM.*AAJTSQACMDIAAlNLABMxOTM5OTEyMDI1MjY3NzE4OTc0AAJTMQACMDE.* 
2015-08-12 12:59:59.002 Error 4786:7fc8f789bae0 PolicyEngine: am_policy_evaluate: InternalException in Service::update_policy with error message:Session query failed. and code:35

Storing policy agent sessions in the CTS store

You can change the configuration to store policy agent sessions in the CTS store by setting the com.iplanet.am.session.agentSessionIdleTime property to force idle policy agent sessions to expire.

The default is 0 (policy agent sessions never expire) but you can also set it to a value of 30 or above (no maximum) to indicate the number of minutes a session can be idle. When set to 30 or above, the agent application token is stored in the CTS token store rather than locally. 

You can set this property using either the console or ssoadm:

  • AM / OpenAM 13.5 console: navigate to: Configure > Server Defaults > Advanced > com.iplanet.am.session.agentSessionIdleTime and amend the required number of minutes.
  • Pre-OpenAM 13.5 console: navigate to: Configuration > Servers and Sites > Default Server Settings > Advanced > com.iplanet.am.session.agentSessionIdleTime and enter the required number of minutes.
  • ssoadm: enter the following command:
    $ ./ssoadm update-server-cfg -s default -u [adminID] -f [passwordfile] -a com.iplanet.am.session.agentSessionIdleTime=[minutes]
    replacing [adminID], [passwordfile] and [minutes] with appropriate values.
Note

You must restart the web application container in which AM/OpenAM runs to apply these configuration changes. 

See Also

FAQ: Core Token Service (CTS) and session high availability in AM/OpenAM

Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)

Sessions

Related Training

N/A

Related Issue Tracker IDs

OPENAM-9731 (Occasional failure to find server from ID in Webtop/Naming )

OPENAM-5632 (Occasional failure of OpenAM configurator tool)


FAQ: Core Token Service (CTS) and session high availability in AM/OpenAM

The purpose of this FAQ is to provide answers to commonly asked questions regarding CTS and session high availability in AM/OpenAM. Session high availability was formerly referred to as session failover.

Frequently asked questions

Q. Can I use any LDAP server for the CTS store?

A. No, DS/OpenDJ is the only supported backend for the CTS store. See Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)  for further information.

Q. Can I used the embedded CTS store in production?

A. We recommend that you use an external CTS store in production as it allows you to tune the LDAP server containing the CTS store separately to the LDAP server containing the configuration store. Typically, a directory containing tokens that change frequently and are relatively large (CTS store) needs different tuning to a directory containing data that is relatively stable (configuration store). Being able to tune these stores separately gives you greater control over performance. 

However, if you have a small scale deployment that is relatively simple, the embedded CTS store may be suitable for your needs; you should performance test this option to check it is appropriate to use the embedded CTS store.

This is discussed in more detail in Installation Guide › General Recommendations for CTS Configuration.

Q. Is the CTS store only used for session tokens?

A. No, the CTS store is used for a variety of tokens, such as OAuth, SAML and REST.

See Understanding CTS token types in AM/OpenAM for further information.

Q. Is it normal for a session token to be a negative number?

A. Yes it is normal due to the way in which session tokens (or coreTokenId attributes) are generated; it does not indicate an error. Session tokens are generated as follows:

  • They are composed of:
    <storage key (generated using SecureRandom)> + <legacy sessionID extension>
  • This value is then encrypted to a string.
  • This string is then converted to a String object using Hex.decodeHex() when the session token is generated. This step can sometimes produce a negative number.

Example

You may see session tokens with negative numbers, for example:

CTS: Create: Created SESSION Token -662229285778566861

Or see them represented as a coreTokenId attribute in your LDAP browser, for example:

Attribute Description                   Value
---------------------                   -----
coreTokenId                             -662229285778566861

Q. Can I use a load balancer with my CTS deployment?

A. You should avoid using a load balancer in front of the CTS stores as detailed in Installation Guide › General Recommendations for CTS Configuration.

See Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)  for further information.

Q. Can I improve CTS performance?

A. Yes, you can tune the DS/OpenDJ server to improve CTS performance as discussed in FAQ: DS/OpenDJ performance and tuning.

All requests are made asynchronously in the background, which increases performance as CPU and memory are better utilized. There are also two properties available (queue size and queue timeout) which can be set to tune CTS further as detailed in Installation Guide › CTS Tuning Considerations.

As of OpenAM 13.0, indexes have been optimized to further improve the performance of CTS.

Q. Can I configure the external CTS token store using ssoadm?

A. Yes you can providing the external CTS token store already exists (and you just want to modify its configuration using ssoadm) or you have taken steps to prepare the external DS/OpenDJ instance for CTS first. See How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm? for further information.

Q. What does session failover mean?

A. Session failover refers to the concept of persisting SSO sessions, which are stored in the CTS store. This store is shared with other servers in the same site to allow for persisting sessions. If a server (AM1) stops responding, other servers in the site can access the session information in the CTS; this means that a user who authenticated to AM1 does not have to log in again.

Session failover has been replaced in AM 5 with the concept of session high availability; this is enabled by default for all AM deployments. See AM 5 Release Notes › What's New › Cloud for further information.

Q. Do I need to configure session failover in sub-realms or will they inherit the settings from the top level realm?

A. Session failover is a system-wide setting so is not something that needs to be adjusted on a per-realm basis.

However, session timeouts can be adjusted on a per-realm basis as detailed in How do I configure realm level session timeouts in AM/OpenAM (All versions)?

Q. How do I enable session failover using ssoadm?

A. You can enable session failover in pre-AM 5 using one of the following ssoadm commands:

  • OpenAM 13.x
    $ ./ssoadm create-sub-cfg -u [adminID] -f [passwordfile] -s iPlanetAMSessionService -b [sitename] -g Site -a iplanet-am-session-sfo-enabled=true
    replacing [adminID], [passwordfile] and [sitename] with appropriate values.
  • Pre-OpenAM 13
    $ ./ssoadm create-sub-cfg -u [adminID] -f [passwordfile] -s iPlanetAMSessionService -b Site -g [sitename] -a iplanet-am-session-sfo-enabled=true
    replacing [adminID], [passwordfile] and [sitename] with appropriate values.

In summary, the difference between these commands is that the -b and -g options have swapped round in OpenAM 13.x. 

Note

Session failover has been replaced in AM 5 with the concept of session high availability; this is enabled by default for all AM deployments. See AM 5 Release Notes › What's New › Cloud for further information.

Q. Are sessions replicated across AM/OpenAM servers?

A. No, sessions are not replicated across AM/OpenAM servers; the CTS tokens that represent the user's session are replicated to all DS/OpenDJ nodes for which you have configured replication.

This is discussed in more detail in Installation Guide › CTS Backups and Directory Services Replication Purge Delay.

Q. Can I disable replication-purge-delay for the CTS backend?

A. No, the replication-purge-delay property is a system-wide setting and applies to the entire replication server; it is not possible to set this for an individual backend.

See Also

Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)

Best practice for using Core Token Service (CTS) Affinity based load balancing in AM (All versions) and OpenAM 13.5.1, 13.5.2

How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm?

Installation Guide › Implementing the Core Token Service

Core Token Service (CTS) and sessions in AM/OpenAM

Related Training

ForgeRock Access Management Core Concepts (AM-400)

Related Issue Tracker IDs

OPENAM-4673 (External CTS Root DN is hardcoded)

OPENAM-3447 (CTS update fails due to attribute conflict)


Known Issues


CoreSystem debug log grows rapidly in OpenAM 13.0 and 13.5 when CTS connection fails

The purpose of this article is to provide assistance if the CoreSystem debug log fills up quickly in OpenAM 13.0 and 13.5 when the CTS connection fails. You will see "Cannot start task executor" and "Diagnostic Message: No operational connection factories available" errors repeating continually in the debug log. This log growth can fill the disk up, bypassing the log rotation maximum value and eventually bring OpenAM down.

Symptoms

The following error is shown in the CoreSystem debug log when the log fills up:

amThreadManager:01/26/2016 02:14:47:633 PM GMT: Thread[amThreadManager-1,5,ServerService ThreadGroup]: TransactionId[300c2122-7500-4b31-abdc-39a19681ff30-2]
ERROR: ThreadMonitor: Thread WatchDog detected error, restarting
java.util.concurrent.ExecutionException: java.lang.IllegalStateException: Cannot start task executor
   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
   at org.forgerock.openam.shared.concurrency.ThreadMonitor$WatchDog.run(ThreadMonitor.java:231)
   at org.forgerock.openam.audit.context.AuditRequestContextPropagatingRunnable.run(AuditRequestContextPropagatingRunnable.java:42)
   at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
   at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalStateException: Cannot start task executor
   at org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutorThread.run(SeriesTaskExecutorThread.java:85)
   ... 6 more
Caused by: org.forgerock.openam.sm.datalayer.api.LdapOperationFailedException: 
CTS: Operation failed:
Result Code: Connect Error
Diagnostic Message: No operational connection factories available
Matched DN: 
   at org.forgerock.openam.sm.datalayer.providers.LdapConnectionFactoryProvider$LdapConnectionFactory.create(LdapConnectionFactoryProvider.java:158)
   at org.forgerock.openam.sm.datalayer.providers.LdapConnectionFactoryProvider$LdapConnectionFactory.create(LdapConnectionFactoryProvider.java:126)
   at org.forgerock.openam.cts.monitoring.impl.connections.MonitoredCTSConnectionFactory.create(MonitoredCTSConnectionFactory.java:71)
   at org.forgerock.openam.sm.datalayer.impl.SimpleTaskExecutor.start(SimpleTaskExecutor.java:59)
   at org.forgerock.openam.sm.datalayer.impl.SeriesTaskExecutorThread.run(SeriesTaskExecutorThread.java:83)
   ... 6 more
Caused by: org.forgerock.opendj.ldap.ConnectionException: Connect Error 

The final Caused by statement varies depending on the root cause, but you may see one of the following causes instead:

Caused by: org.forgerock.opendj.ldap.ConnectionException: Connect Error: Connection refused

Caused by: org.forgerock.opendj.ldap.ConnectionException: Server Connection Closed 

Caused by: org.forgerock.opendj.ldap.AuthenticationException: Invalid Credentials
Note

You will still encounter this issue even if you have debug log rotation set up since OpenAM does not currently delete old log files once they are rotated.

Recent Changes

Upgraded to, or installed OpenAM 13.0 or 13.5.

Made configuration changes to CTS.

CTS Configuration is set to incorrect value for "Login Id".

Causes

OpenAM cannot connect to the CTS token store; this may be because CTS is misconfigured or there is a connection issue, such as the CTS token store is down or a firewall is refusing the connection between the two servers.

There is a known issue where this situation results in the errors being continually logged in the CoreSystem debug log, which causes it to grow rapidly: OPENAM-8202 (If the "Login Id" in the External Store Configuration(CTS) is set to incorrect value,CoreSystem debug log is full of duplicate error).

Solution

This issue can be resolved by upgrading to OpenAM 13.5.1 or later; you can download this from BackStage.

Workaround

Alternatively, this issue can be resolved as follows:

  1. Clear your CoreSystem debug log.
  2. Correct the root cause.

Clear your CoreSystem debug log: You can clear the debug file as detailed in How do I clear debug logs in AM/OpenAM (All versions)? This should make the system responsive again, although the CoreSystem debug log will immediately start filling again so you need to be quick. If you do not have time to rectify your issue before the log fills up again, you can run the following command to continually clear the debug file on a loop to give you more time:

$ while true; do cat /dev/null > CoreSystem; done

Alternatively, you can temporarily switch OpenAM back to using the default embedded CTS token store (rather than external) using the following ssoadm command to give you more time: 

$ ./ssoadm update-server-cfg -s [serverName] -u [adminID] -f [passwordfile] -a org.forgerock.services.cts.store.location=default

replacing [serverName], [adminID] and [passwordfile] with appropriate values. You can use default for [serverName] if you want to change the Default Server Settings. 

Correct the root cause: Depending on whether you made any configuration changes or not, you should:

  • Ensure the CTS token store is up and running prior to starting OpenAM.
  • Ensure OpenAM can communicate with the OpenDJ server running CTS.
  • Correct your misconfiguration / revert the recent configuration change you made. A good way of doing this is via ssoadm using the commands detailed in How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm? or by restoring a backup taken prior to making your configuration changes.

See Also

How do I configure an external CTS token store in AM/OpenAM (All versions) using ssoadm?

Best practice for configuring an external DS/OpenDJ instance for the Core Token Service (CTS) in AM/OpenAM (All versions)

FAQ: Core Token Service (CTS) and session high availability in AM/OpenAM

Installation Guide › Implementing the Core Token Service

Related Training

N/A

Related Issue Tracker IDs

OPENAM-8202 (If the "Login Id" in the External Store Configuration(CTS) is set to incorrect value,CoreSystem debug log is full of duplicate error)


How do I remove the embedded DS/OpenDJ (All versions) after migrating to an external instance?

The purpose of this article is to provide assistance with removing the embedded DS/OpenDJ to prevent it from starting up with AM/OpenAM once you have migrated to an external instance. The embedded DJ instance is started and stopped with internal calls rather than through the standard DS/DJ stop/start scripts. The procedure in this article assumes you are not using the embedded instance for any of your data stores.

Removing the embedded DS/OpenDJ

Warning

You must be certain that you are not using the embedded DS/OpenDJ for any of your data stores before proceeding.

You can remove the embedded DS/OpenDJ as follows:

  1. Stop the web application container in which AM/OpenAM runs. 
  2. Make a backup of the $HOME/[am_instance]/opends directory:
    $ cp -r $HOME/[am_instance]/opends /path/to/backupFiles/opends 
  3. Remove the $HOME/[am_instance]/opends directory:
    $ rm -rf $HOME/[am_instance]/opends
    
  4. Restart the web application container in which AM/OpenAM runs.

Restore

To restore the embeded DS/OpenDJ instance copy back the opends backup folder to the AM instance directory as follows:

$ cp -r /path/to/backupFiles/opends $HOME/[am_instance]/opends

Restart the web application container in which AM/OpenAM runs.

See Also

How do I migrate from an embedded to external DS/OpenDJ in AM/OpenAM (All versions)?

How do I add a second configuration store or edit an existing configuration store in AM/OpenAM (All versions)?

Data stores in AM/OpenAM

Related Training

N/A

Related Issue Tracker IDs

N/A


Copyright and TrademarksCopyright © 2018 - 2019 ForgeRock, all rights reserved.

This content has been optimized for printing.

Loading...