How To
ForgeRock Identity Platform
Does not apply to Identity Cloud

How do I migrate OAuth 2.0 CTS-based tokens to AM 6.5.x from an earlier version?

Last updated Apr 13, 2021

The purpose of this article is to provide information on migrating OAuth 2.0 CTS-based tokens (Access and Refresh) from an earlier version to DS 6.5.x for use with AM 6.5.x. CTS-based tokens were called Stateful in AM 5.x.

1 reader recommends this article


This article describes how to export CTS-based tokens into DS 6.5.x for use in AM 6.5.x that were created in earlier versions.

The idea behind this article is to allow for a complete migration scenario to DS 6.5.x and AM 6.5.x without disruption as far as OAuth 2.0 clients are concerned. This article is split as follows:

OAuth 2.0 token management

A grant-set storage scheme was introduced in AM 6.5, which the release notes describe as "Improved CTS Storage Scheme for OAuth 2.0 tokens"; you can read more about this here: Release Notes › New Features: What's New in AM 6.5. This storage scheme stores the state of multiple authorizations for a given OAuth2 client and resource owner pair in a single OAUTH2_GRANT_SET object. Additionally, it groups Access and Refresh tokens into a single JSON object within the LDAP object (in the coreTokenMultiString03 entry). This approach reduces duplication and the number of calls to the CTS, which in turn improves AM performance. The previous storage scheme by comparison was considered a 1:1 storage scheme. 

You can see what example tokens looks like for the new OAUTH2_GRANT_SET token type and the previous OAUTH token type in How do I know what OAuth2 and session token types are stored in the CTS in AM (All versions)? Additionally, there are fewer indexed attributes with the grant-set storage scheme, which optimizes AM to CTS request response times. The OAUTH2_GRANT_SET has four indexed attributes compared to eight indexed attributes with OAUTH.

Migration Considerations

CTS-based tokens created in pre-AM 6.5 do not have grant-set storage. Unfortunately, the CTS cannot handle both 1:1 and grant-set storage schemes concurrently in a migration scenario. This is a known issue: OPENAM-15627 (Switching CTS Storage Scheme to "Grant-set" fails with refresh-tokens created with "One-To-One" ).

As indicated in the following table, the storage scheme has to be set to 1:1 if legacy Access and/or Refresh tokens are imported from an old AM version into AM 6.5.x:

Tokens Client-Based Grant Token Upgrade Compatibility Mode* CTS Storage Scheme* Legacy Refresh Tokens? Performance
OAUTH tokens ON 1:1 Allowed Sub-optimum
OAUTH2_STATELESS_GRANT tokens ON Grant-Set Not Allowed due to known issue: OPENAM-15627  Improved, but still sub-optimum
OAUTH2_STATELESS_GRANT tokens OFF Grant-Set None Optimum

* The global settings in this table can be found by navigating to Configure > Global Services > OAuth2 Provider > Global Attributes in the console.

Migrating OAuth 2.0 tokens

The high level steps for migrating tokens are as follows:

  1. Deploy the new AM environment. See Release NotesDeployment Planning GuideInstallation Guide and Upgrade Guide for further information.
  2. Create a full backup of DS in both the new and old environments. See Installation Guide › Exporting and Importing Data in DS by using LDIF for further information.
  3. Create a backup of the configuration store and CTS store in both the new and old environments. See Administration Guide › Backing Up and Restoring Data for further information.
  4. Export the data from the old CTS store and import it into the new one.

Once you have completed these steps, you can do some testing such as introspect the OAuth2 Access and Refresh tokens on the new AM version to check the validity of the Refresh tokens. See OAuth 2.0 Guide › /oauth2/introspect for further information.

Export and import CTS data

You can export and import CTS data as follows: 

  1. Export CTS tokens from the old AM version using the export-ldif command. You can export all tokens or a subset of tokens as required. The following examples demonstrate exporting all tokens and different subsets of tokens with the server online; you must include the --offline option if the server is stopped:
    • Export all tokens: $ ./export-ldif --hostname --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --backendID amCts --trustAll
    • Export only Refresh tokens: $ ./export-ldif --hostname --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --includeFilter "(coreTokenString10=refresh_token)" --backendID amCts --trustAll
    • Export Access and Refresh tokens:  $ ./export-ldif --hostname --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --includeFilter "(|(coreTokenString10=refresh_token)(coreTokenString10=access_token))" --backendID amCts --trustAll
  2. Remove any lines from the export file that do not relate to tokens. This may happen if you've exported everything. If you've used the --includeFilter to specify tokens, you can skip this step.
  3. Back up the new CTS using the export-ldif command: $ ./export-ldif --hostname --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/ctsbackup.ldif --backendID amCts --trustAll
  4. Make a backup of the backup you just created: $ cp /path/to/ctsbackup.ldif /path/to/ctsbackup.ldif.backup
  5. Concatenate your backup and the export from step 1 to create a new ldif file: $ cat /path/to/ctsbackup.ldif /path/to/tokens.ldif > /path/to/ctsnew.ldif
  6. Place the resulting file (for example ctsnew.ldif) in the /path/to/ds directory on the new CTS instance.

Do not proceed unless you are certain that the previous steps have completed successfully. If the following import-ldif command fails (for example, due to filename errors or if a previous step was not completed), there is a high chance that you will end up with a corrupted CTS that cannot be recovered.

  1. Import the file into the new AM 6.5.x instance using the import-ldif command, for example: $ ./import-ldif --hostname --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/ctsnew.ldif --backendID amCts --trustAll

See Also

How do I migrate OAuth 2.0 client-based tokens to AM 6.5.x from an earlier version?

Best practice for configuring sessions in AM (All versions) to reduce the impact on the CTS store

Understanding CTS token types in AM

Installation Guide › Implementing the Core Token Service

Installation Guide › General Recommendations for CTS Configuration

AM Upgrade Guide › Supported Upgrade Paths

Related Training


Related Issue Tracker IDs

OPENAM-15627 (Switching CTS Storage Scheme to "Grant-set" fails with refresh-tokens created with "One-To-One" )

Copyright and TrademarksCopyright © 2021 ForgeRock, all rights reserved.