How To

How do I migrate OAuth 2.0 client-based tokens to AM 6.5.x from an earlier version?

Last updated Jul 11, 2020

The purpose of this article is to provide information on migrating OAuth 2.0 client-based tokens (Access and Refresh) to AM 6.5.x from an earlier version. Client-based tokens were called Stateless in AM 5.x.


1 reader recommends this article

Overview

This article describes OAuth 2.0 client-based token management considerations when migrating from OpenAM 13.x, AM 5.x or AM 6.0.x to AM 6.5.x. There are a number of differences in token structure, storage and processing that make the considerations noteworthy:

  • Token whitelisting versus blacklisting.
  • The grant token upgrade compatibility mode.
  • The CTS storage scheme.

The idea behind this article is to allow for a complete migration scenario to AM 6.5.x without disruption as far as OAuth 2.0 clients are concerned. This article is split as follows:

OAuth 2.0 token management

Token Evolution

The management of OAuth 2.0 tokens in AM first evolved in AM 5.5 with the introduction of the grant token upgrade compatibility mode. The release notes describe this as "Reduced Metadata for Stateless OAuth 2.0 Tokens"; you can read more about this change here: Release Notes › Major Improvements: Access Management 5.5

You can tell which type of token it is by looking at the CoreTokenType value of the token:

  • OAUTH2_STATELESS_GRANT indicates post-grant upgrade tokens.
  • OAUTH_STATELESS indicates pre-grant upgrade tokens.

You can see the reduction in metadata for the OAUTH2_STATELESS_GRANT token type compared to the OAUTH_STATELESS token type if you look for these token types in the following articles: How do I know what LDAP attributes are used by CTS tokens (OAuth2 and session) in AM/OpenAM (All versions)? and How do I know what OAuth2 and session token types are stored in the CTS in AM/OpenAM (All versions)? Additionally, there are far fewer indexed attributes with the post-grant upgrade tokens, which optimizes AM to CTS request response times. The OAUTH2_STATELESS_GRANT has two indexed attributes compared to eight indexed attributes with OAUTH_STATELESS.

A grant-set storage scheme was introduced in AM 6.5, which the release notes describe as "Improved CTS Storage Scheme for OAuth 2.0 tokens"; you can read more about this here: Release Notes › New Features: What's New in AM 6.5. This storage scheme stores the state of multiple authorizations for a given OAuth2 client and resource owner pair in a single OAUTH2_GRANT_SET object. Additionally, it groups Access and Refresh tokens into a single JSON object within the LDAP object (in the coreTokenMultiString03 entry). This approach reduces duplication and the number of calls to the CTS, which in turn improves AM performance. The previous storage scheme by comparison was considered a 1:1 storage scheme. You can see what an example token looks like for the OAUTH2_GRANT_SET token type in How do I know what OAuth2 and session token types are stored in the CTS in AM/OpenAM (All versions)?

Migration Considerations

You can configure client-based OAuth 2.0 token clients to receive Refresh tokens when Access tokens are issued and also to receive Refresh tokens when Refresh tokens are exchanged for Access tokens. In a migration scenario (such as upgrading to AM 6.5.x), this means clients that have previously issued Access tokens and Refresh tokens in the OAUTH_STATELESS format now have to work with tokens in a different format and storage scheme. That is, mixing OAUTH_STATELESS and OAUTH2_STATELESS_GRANT tokens in either a 1:1 or grant set storage scheme. In a scenario where there is a single cut-off point, that is, all the traffic to the old AM/OpenAM version is switched off at the point AM 6.5.x is switched on, all tokens have to be migrated to AM 6.5.x and DS 6.5.x ahead of the move. Therefore use case scenarios have to be tested and a migration path has to be planned.

As can be seen in the following table, all tokens can co-exist in the same CTS instance providing the Client-Based Grant Token Upgrade Compatibility Mode is switched ON, but performance will be affected:

Tokens Client-Based Grant Token Upgrade Compatibility Mode* CTS Storage Scheme* Legacy Refresh Tokens? Performance
Mixed Tokens ON 1:1 Allowed Sub-optimum
Mixed Tokens ON Grant-Set Allowed Improved, but still sub-optimum
OAUTH2_STATELESS_GRANT tokens OFF Grant-Set None Optimum

* The global settings in this table can be found by navigating to Configure > Global Services > OAuth2 Provider > Global Attributes in the console.

Blacklisting

When considering OAUTH_STATELESS tokens and blacklisting, the behavior can be represented in a simplified way as follows (note - on this diagram token IDS refer to the internal CTS referencing to tokens, the token information is client-based and therefore not present in the CTS):

Where, a client receives Refresh and Access tokens, and AM (or the client via APIs) can blacklist any token in order to revoke it.

This should be compared to the following image, which shows more possibilities where all tokens co-exist in AM 6.5.x:

See the key below for token type representations

 

Whitelisting versus Blacklisting

Finally, the documentation of OpenAM 13.x mentions blacklisting, whereas AM 6.5 supports both whitelisting and blacklisting: OAuth 2.0 Guide › About Token Storage Location. Therefore, while a whitelist lookup is enforced, a blacklist lookup is still performed subsequently. Migrating blacklisted tokens from a legacy environment is supported and in the new deployment this will result in denied access and refresh operations as expected. See OAuth 2.0 Guide › Configuring Client-Based OAuth 2.0 Token Blacklisting for further details about the blacklisting feature.

Migrating OAuth 2.0 tokens

The high level steps for migrating tokens are as follows:

  1. Deploy the new AM environment. See Release NotesDeployment Planning GuideInstallation Guide and Upgrade Guide for further information.
  2. Migrate the signature key from the old environment to the new environment.
  3. Create a full backup of DS in both the new and old environments. See Installation Guide › Exporting and Importing Data in DS by using LDIF for further information.
  4. Create a backup of the configuration store and CTS store in both the new and old environments. See Administration Guide › Backing Up and Restoring Data for further information.
  5. Export the data from the old CTS store and import it into the new one.

Once you have completed these steps, you can do some testing such as introspect the OAuth2 Access and Refresh tokens on the new AM version to check the validity of the Refresh tokens. See OAuth 2.0 Guide › /oauth2/introspect for further information.

Migrate the signature keys

Tokens are signed with an algorithm associated with a key. This key must be migrated to the new environment: 

  • HMAC-based signatures - the key is a set of characters, which are randomly set by AM during installation and likely modified before moving into production.
  • RSA-based signatures - the key is in a certificate in the JCEKS keystore.
Note

You can verify the signatures in the keystore using jwt.io. In order to do this, you must extract the key from jks_uri first. Extracting keys and verifying signatures are outside the scope of ForgeRock support; if you want more tailored advice, consider engaging Deployment Support Services.

HMAC-based signatures

You can retrieve the key from the old AM/OpenAM environment and add it to the new AM 6.5.x environment using one of the following options:

  • Create a new secret store in AM 6.5.x and add the key:
    1. In the old AM/OpenAM version, navigate to Configure > Global Services > OAuth2 Provider in the console.
    2. Locate the Token Signing HMAC Shared Secret value (for example: vyt6EvxXYu2t2rbBP8ktFaNm+GWiGRZVQTAoV4/tBNU=) and copy it.
    3. In AM 6.5.x, create a new secret store as follows:
      1. Navigate to Realms > [Realm Name] > Secret Stores and click Add Secret Store.
      2. Enter a name for the new Secret Store in the Secret Store ID field, for example, HMAC.
      3. Select File System Secret Volume from the Store Type field.
      4. Enter the path to the directory where the secret files are located in the Directory field.
      5. Click Create.
      6. Specify a file suffix (for example, .txt) and select BASE64_HMAC_KEY as the file format.
    4. From the command line, create a file with a filename that matches the secret store ID + file suffix (for example, HMAC.txt) in the directory you specified.
    5. Paste the value you copied in step 2 into this file.
  • Add the key to the AM 6.5.x JCEKS keystore
    1. In the old AM/OpenAM version, navigate to Configure > Global Services > OAuth2 Provider in the console.
    2. Locate the Token Signing HMAC Shared Secret value (for example: vyt6EvxXYu2t2rbBP8ktFaNm+GWiGRZVQTAoV4/tBNU=) and copy it.
    3. Convert the key from base64 to Hex (for example, you can use a third-party website such as: https://base64.guru/converter/decode/hex) and copy the resulting value.
    4. Paste the Hex value into a new JCEKS keystore (for example, using Keystore Explorer).
    5. Save the new keystore (for example, newkeystore.jceks).
    6. Import the key from the new keystore you created in step 5 into the AM 6.5 JCEKS keystore using the keytool command, where the default alias for the key is hmacsigningtest. For example:
      $ keytool -importkeystore -srckeystore newkeystore.jceks -srcstoretype JCEKS -destkeystore keystore.jceks -deststoretype JCEKS -srcalias hmacsigningtest -destalias hmacsigningtest -srcstorepass password -srckeypass password -deststorepass $(cat .storepass) -destkeypass $(cat .keypass)

Migrating from AM 5.5.x and AM 6.0.x

AM 5.5.x and AM 6.0.x do not display the HMAC signature secret and therefore it is not readily available for copy/paste. Instead one has to follow the upgrade path and the secret will be moved to a secret store automatically. The secret store will be an encrypted file base HMAC secret dedicated to HMAC signatures. The upgrade route for AM can be found in the AM Upgrade Guide › Configuring Secret Stores After Upgrade.

In practice this is the course of action followed when doing a so called blue/green upgrade. One AM server is isolated and upgraded which then becomes the model to deploy other AM machines. The blue/green upgrade is described here: AM and IG: Zero Downtime Upgrade Strategy Using a Blue/Green Deployment.

RSA-based Signatures

  1. Import the certificate from the old AM keystore into the AM 6.5 keystore. By default, AM stores the certificate under the test alias. You can use the keytool command to do this. For example, with a self-signed certificate, you would copy your old keystore.jceks (here renamed old_openam_keystore.jceks) into the same folder as the new keystore.jceks and use a command such as this:
    $ keytool -importkeystore -srckeystore old_openam_keystore.jceks -srcstoretype JCEKS -srcstorepass $(cat .storepass) -destkeystore keystore.jceks -deststoretype JCEKS -deststorepass $(cat .storepass) -srcalias test -destalias test -srckeypass $(cat .keypass) -destkeypass $(cat .keypass)
    
    In this example, the alias used is test. Therefore, you can assign this same alias to am.services.oauth2.stateless.signing.RSA in the console, by navigating to: Configure > Secret Stores > default-keystore > Mappings.

Export and import CTS data

You can export and import CTS data as follows: 

  1. Export CTS tokens from the old AM/OpenAM version using the export-ldif command. You can export all tokens or a subset of tokens as required. The following examples demonstrate exporting all tokens and different subsets of tokens with the server online; you must include the --offline option if the server is stopped in AM 5 and later / DS 5 and later:
    • Export all tokens:
      $ ./export-ldif --hostname oldAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --backendID amCts --trustAll
      
    • Export only Refresh tokens:
      $ ./export-ldif --hostname oldAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --includeFilter "(coreTokenString10=refresh_token)" --backendID amCts --trustAll
      
    • Export Access and Refresh tokens: 
      $ ./export-ldif --hostname oldAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/tokens.ldif --includeBranch ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org --includeFilter "(|(coreTokenString10=refresh_token)(coreTokenString10=access_token))" --backendID amCts --trustAll
      
  2. Remove any lines from the export file that do not relate to tokens. This may happen if you've exported everything. If you've used the --includeFilter to specify tokens, you can skip this step.
  3. Back up the new CTS using the export-ldif command:
    $ ./export-ldif --hostname newAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/ctsbackup.ldif --backendID amCts --trustAll
  4. Make a backup of the backup you just created:
    $ cp /path/to/ctsbackup.ldif /path/to/ctsbackup.ldif.backup
  5. Concatenate your backup and the export from step 1 to create a new ldif file :
    $ cat /path/to/ctsbackup.ldif /path/to/tokens.ldif > /path/to/ctsnew.ldif
  6. Place the resulting file (for example ctsnew.ldif) in the /path/to/ds directory on the new CTS instance.
Caution

Do not proceed unless you are certain that the previous steps have completed successfully. If the following import-ldif command fails (for example, due to filename errors or if a previous step was not completed), there is a high chance that you will end up with a corrupted CTS that cannot be recovered.

  1. Import the file into the new AM 6.5.x instance using the import-ldif command, for example:
    $ ./import-ldif --hostname newAM.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --ldifFile /path/to/ctsnew.ldif --backendID amCts --trustAll
    

See Also

Knowledge Articles:

How do I migrate OAuth 2.0 CTS-based tokens to AM 6.5.x from an earlier version?

Best practice for configuring sessions in AM (All versions) to reduce the impact on the CTS store

Understanding CTS token types in AM/OpenAM

Shared secret cannot be null error when requesting OAuth2 access tokens in AM 5.x and 6.0.0.x

Documentation

AM Upgrade Guide › Supported Upgrade Paths

Installation Guide › Implementing the Core Token Service

Installation Guide › General Recommendations for CTS Configuration

Setup and Maintenance Guide › Copying Key Aliases

Setup and Maintenance Guide › Secret ID Mapping Defaults

Setup and Maintenance Guide › Managing Key Aliases and Passwords

Related Training

N/A

Related Issue Tracker IDs

OPENAM-16482 (AM 5.5 and 6.0 UI should warn when OAuth 2.0 tokens are signed by an HMAC secret smaller than 256 bits)  



Copyright and TrademarksCopyright © 2020 ForgeRock, all rights reserved.
Loading...