How do I migrate OAuth 2.0 CTS-based tokens to AM 6.5.x from an earlier version?
The purpose of this article is to provide information on migrating OAuth 2.0 CTS-based tokens (Access and Refresh) from an earlier version to DS 6.5.x for use with AM 6.5.x.
2 readers recommend this article
This article describes how to export CTS-based tokens into DS 6.5.x for use in AM 6.5.x that were created in earlier versions.
The idea behind this article is to allow for a complete migration scenario to DS 6.5.x and AM 6.5.x without disruption as far as OAuth 2.0 clients are concerned. This article is split as follows:
- OAuth 2.0 token management describes the underlying principles.
- Migrating OAuth 2.0 tokens shows the step-by-step migration process of the CTS from earlier versions to DS 6.5.x.
OAuth 2.0 token management
A grant-set storage scheme was introduced in AM 6.5 as detailed in the release notes: What's New in AM 6.5 (Improved CTS Storage Scheme for OAuth 2.0 tokens). This storage scheme stores the state of multiple authorizations for a given OAuth2 client and resource owner pair in a single OAUTH2_GRANT_SET object. Additionally, it groups Access and Refresh tokens into a single JSON object within the LDAP object (in the coreTokenMultiString03 entry). This approach reduces duplication and the number of calls to the CTS, which in turn improves AM performance. The previous storage scheme by comparison was considered a 1:1 storage scheme.
You can see what example tokens looks like for the new OAUTH2_GRANT_SET token type and the previous OAUTH token type in How do I know what OAuth 2.0 and session token types are stored in the CTS in AM (All versions)? Additionally, there are fewer indexed attributes with the grant-set storage scheme, which optimizes AM to CTS request response times. The OAUTH2_GRANT_SET has four indexed attributes compared to eight indexed attributes with OAUTH.
CTS-based tokens created in AM 6 do not have grant-set storage. Unfortunately, the CTS cannot handle both 1:1 and grant-set storage schemes concurrently in a migration scenario in pre-AM 6.5.3. This is a known issue: OPENAM-15627 (Switching CTS Storage Scheme to "Grant-set" fails with refresh-tokens created with "One-To-One" ),
As indicated in the following table, the storage scheme has to be set to 1:1 if legacy Access and/or Refresh tokens are imported from an old AM version into AM 6.5.x:
|Tokens||Client-Based Grant Token Upgrade Compatibility Mode*||CTS Storage Scheme*||Legacy Refresh Tokens?||Performance|
|OAUTH2_STATELESS_GRANT tokens||ON||Grant-Set||Allowed in AM 6.5.3 and later (Known issue in pre-AM 6.5.3: OPENAM-15627)||Improved, but still sub-optimum|
* The global settings in this table can be found by navigating to Configure > Global Services > OAuth2 Provider > Global Attributes in the AM admin UI.
Migrating OAuth 2.0 tokens
The high level steps for migrating tokens are as follows:
- Deploy the new AM environment. See Release Notes, Deployment Planning Guide, Installation Guide and Upgrade Guide for further information.
- Create a full backup of DS in both the new and old environments. See Exporting and Importing Data in DS by using LDIF for further information.
- Create a backup of the configuration store and CTS store in both the new and old environments. See Backing Up and Restoring Data for further information.
- Export the data from the old CTS store and import it into the new one.
Once you have completed these steps, you can do some testing such as introspect the OAuth2 Access and Refresh tokens on the new AM version to check the validity of the Refresh tokens. See /oauth2/introspect for further information.
Export and import CTS data
You can export and import CTS data as follows:
- Export CTS tokens from the old AM version using the export-ldif command. You can export all tokens or a subset of tokens as required. The following examples demonstrate exporting all tokens and different subsets of tokens with the server online; you must include the --offline option if the server is stopped:
- Export all tokens: $ ./export-ldif --hostname oldAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --backendID amCts --trustAll
- Export only Refresh tokens: $ ./export-ldif --hostname oldAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --includeFilter "(coreTokenString10=refresh_token)" --backendID amCts --trustAll
- Export Access and Refresh tokens: $ ./export-ldif --hostname oldAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --includeFilter "(|(coreTokenString10=refresh_token)(coreTokenString10=access_token))" --backendID amCts --trustAll
- Remove any lines from the export file that do not relate to tokens. This may happen if you've exported everything. If you've used the --includeFilter to specify tokens, you can skip this step.
- Back up the new CTS using the export-ldif command: $ ./export-ldif --hostname newAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/ctsbackup.ldif --backendID amCts --trustAll
- Make a backup of the backup you just created: $ cp /path/to/ctsbackup.ldif /path/to/ctsbackup.ldif.backup
- Concatenate your backup and the export from step 1 to create a new ldif file: $ cat /path/to/ctsbackup.ldif /path/to/tokens.ldif > /path/to/ctsnew.ldif
- Place the resulting file (for example ctsnew.ldif) in the /path/to/ds directory on the new CTS instance.
Do not proceed unless you are certain that the previous steps have completed successfully. If the following import-ldif command fails (for example, due to filename errors or if a previous step was not completed), there is a high chance that you will end up with a corrupted CTS that cannot be recovered.
- Import the file into the new AM 6.5.x instance using the import-ldif command, for example: $ ./import-ldif --hostname newAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/ctsnew.ldif --backendID amCts --trustAll
How do I migrate OAuth 2.0 client-based tokens to AM 6.5.x from an earlier version?
How do I improve performance with OAuth 2.0 tokens in AM 6.5.x and 7.x?
Best practice for configuring sessions in AM (All versions) to reduce the impact on the CTS store
Understanding CTS token types in AM
Implementing the Core Token Service
General Recommendations for CTS Configuration
Related Issue Tracker IDs
OPENAM-15627 (Switching CTS Storage Scheme to "Grant-set" fails with refresh-tokens created with "One-To-One" )