How To
ForgeRock Identity Platform
Does not apply to Identity Cloud

How do I delete all or some of the tokens in the CTS store in AM (All versions)?

Last updated Jul 13, 2021

The purpose of this article is to provide information on how an administrator can clean up tokens in the Core Token Service (CTS) store. This article covers deleting all tokens in the CTS and deleting just a subset (for example, only Refresh tokens). This information should not be used in lieu of a properly configured/tuned CTS.


1 reader recommends this article

Overview

By default, AM manages expired tokens using its reaper service, although you can use DS to manage token expiration instead if you prefer. You should configure the CTS reaper and then tune your CTS store appropriately to ensure tokens are removed in an efficient manner. See the following links for further information:

If you have upgraded to AM 7, you should retune the CTS Reaper as described in Reaper Search Size because AM 7 changes the way the CTS reaper searches for expired tokens.

However, even with the reaper running well and pruning expired tokens as expected, there may be occasions when you need to manually delete all tokens in the CTS or delete just a subset. For example, if you have been load testing, you may want to delete all the test tokens that were created before running further tests. Alternatively, you may need to clean up tokens that have built up as a result of an improperly configured/tuned CTS that is not adequate for your environment or specific deployment needs; if this is the case, you should ensure you tune the CTS properly to prevent a build-up in future.

Note

Tuning the CTS is outside the scope of ForgeRock support; if you want more tailored advice, consider engaging Deployment Support Services.

This article guides you through the following scenarios:

Caution

Please be aware of the following:

  • Deleting tokens will end all sessions associated with them.
  • We strongly recommend testing the procedure in your own development environment first and ensuring you have up-to-date backups in case you need to revert.
  • You must shut down the CTS instance before deleting tokens.

Example values

These example processes use the following values:

  • A BaseDN of dc=example,dc=com
  • A backendID of cts-store for the CTS server database.
  • The DS admin port is 4444
  • The hostname is ds1.example.com

You should adjust these values as needed for your environment and ensure you include the required SSL options if you are using LDAPS.

Deleting all tokens in the CTS using LDIF export and import

This approach removes all tokens. In essence, you create an LDIF file containing all data, but excluding any tokens. You then import this LDIF to override the contents of the database; the import process automatically rebuilds the indexes. These example steps disable the LDAP/LDAPS connection handler that AM is configured to communicate on to stop updates reaching the CTS instance and causing divergence.

DS 7 and later

You can delete all tokens as follows in DS 7 and later:

  1. Navigate to the bin directory of DS.
  2. Disable the LDAP/LDAPS connector so that AM stops sending traffic to this node providing you have an alternative LDAP connector for subsequent LDAP operations. For example, to disable the LDAP connection handler:
    • DS 7.1 and later: $ ./dsconfig set-connection-handler-prop --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --handler-name LDAP --set enabled:false --usePkcs12TrustStore /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --no-prompt
    • DS 7: $ ./dsconfig set-connection-handler-prop --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --handler-name LDAP --set enabled:false --usePkcs12TrustStore /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --no-prompt
  3. Shutdown the CTS server.
  4. Take an LDIF export: $ ./export-ldif --backendID cts-store --ldifFile /path/to/export.ldif --excludeFilter "(objectclass=frCoreToken)" --offline
  5. Check the file excludes all tokens.
  6. Import the LDIF: $ ./import-ldif --backendID cts-store --ldifFile /path/to/export.ldif --skipFile /tmp/skips.txt --rejectFile /tmp/rejects.txt --offline
  7. Start the CTS server.
  8. Re-initialize all other nodes from this instance to bring all other CTS nodes back into sync with this empty instance. For example:
    • DS 7.1 and later: $ ./dsrepl initialize --baseDN dc=example,dc=com --toAllServers --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --no-prompt
    • DS 7: $ ./dsrepl initialize --baseDN dc=example,dc=com --toAllServers --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --no-prompt
  9. Re-enable the LDAP/LDAPS connector to resume AM sending traffic to this node. For example, to enable the LDAP connection handler:
    • DS 7.1 and later: $ ./dsconfig set-connection-handler-prop --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --handler-name LDAP --set enabled:true --usePkcs12TrustStore /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --no-prompt
    • DS 7: $ ./dsconfig set-connection-handler-prop --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --handler-name LDAP --set enabled:true --usePkcs12TrustStore /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --no-prompt

Pre-DS 7

You can delete all tokens as follows in pre-DS 7:

  1. Navigate to the bin directory of DS.
  2. Disable the LDAP/LDAPS connector so that AM stops sending traffic to this node providing you have an alternative LDAP connector for subsequent LDAP operations. For example, to disable the LDAP connection handler: $ ./dsconfig set-connection-handler-prop --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --handler-name LDAP --set enabled:false --trustAll --no-prompt
  3. Shutdown the CTS server.
  4. Take an LDIF export: $ ./export-ldif --backendID cts-store --ldifFile /path/to/export.ldif --excludeFilter "(objectclass=frCoreToken)" --offline
  5. Check the file excludes all tokens.
  6. Import the LDIF: $ ./import-ldif --backendID cts-store --ldifFile /path/to/export.ldif --skipFile /tmp/skips.txt --rejectFile /tmp/rejects.txt --offline
  7. Start the CTS server.
  8. Re-initialize all other nodes from this instance to bring all other CTS nodes back into sync with this empty instance. For example: $ ./dsreplication initialize-all --hostname ds1.example.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --no-prompt
  9. Re-enable the LDAP/LDAPS connector to resume AM sending traffic to this node. For example, to enable the LDAP connection handler: $ ./dsconfig set-connection-handler-prop --hostname ds1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --handler-name LDAP --set enabled:true --trustAll --no-prompt

Defining a subset of tokens

The following example process deletes all Refresh tokens. This is done by filtering/searching on tokens where coreTokenString10 (token type) is set to refresh_token, for example:

"(coreTokenString10=refresh_token)"

You can amend these process to look for other token types or other LDAP attributes as needed to define a subset of tokens. See How do I know what LDAP attributes are used by CTS tokens (OAuth2 and session) in AM (All versions)? for information on the attributes and values available.

You can use these attributes together to refine your subset further. For instance, you could include the coreTokenExpirationDate attribute to filter tokens before or after a certain timestamp. For example, the following would only affect refresh tokens with an expiration date before 01/01/2021:

"(&(coreTokenString10=refresh_token)(coreTokenExpirationDate<=20210101000000.0Z))"

Deleting a subset of tokens using LDIF export and import

This approach removes a subset of tokens. In essence, you create an LDIF file containing all data and tokens, except tokens where coreTokenString10 (token type) is set to refresh_token. You then import this LDIF to override the contents of the database; the import process automatically rebuilds the indexes.

DS 7 and later

You can delete all Refresh tokens as follows in DS 7 and later:

  1. Navigate to the bin directory of DS.
  2. Shutdown the CTS server.
  3. Take an LDIF export: $ ./export-ldif --backendID cts-store --ldifFile /path/to/export.ldif --excludeFilter "(coreTokenString10=refresh_token)" --offline
  4. Check the file excludes all Refresh Tokens (tokens where coreTokenString10=refresh_token).
  5. Import the LDIF: $ ./import-ldif --backendID cts-store --ldifFile /path/to/export.ldif --skipFile /tmp/skips.txt --rejectFile /tmp/rejects.txt --offline
  6. Start the CTS server.
  7. Re-initialize all other nodes from this instance to bring all other CTS nodes back into sync with this empty instance. For example:
    • DS 7.1 and later: $ ./dsrepl initialize --baseDN dc=example,dc=com --toAllServers --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePassword:file /path/to/ds/config/keystore.pin --no-prompt
    • DS 7: $ ./dsrepl initialize --baseDN dc=example,dc=com --toAllServers --hostname ds1.example.com --port 4444 --bindDN uid=admin --bindPassword password --trustStorePath /path/to/ds/config/keystore --trustStorePasswordFile /path/to/ds/config/keystore.pin --no-prompt

Pre-DS 7

You can delete all Refresh tokens as follows in pre-DS 7:

  1. Navigate to the bin directory of DS.
  2. Shutdown the CTS server.
  3. Take an LDIF export: $ ./export-ldif --backendID cts-store --ldifFile /path/to/export.ldif --excludeFilter "(coreTokenString10=refresh_token)" --offline
  4. Check the file excludes all Refresh Tokens (tokens where coreTokenString10=refresh_token).
  5. Import the LDIF: $ ./import-ldif --backendID cts-store --ldifFile /path/to/export.ldif --skipFile /tmp/skips.txt --rejectFile /tmp/rejects.txt --offline
  6. Start the CTS server.
  7. Re-initialize all other nodes from this instance to bring all other CTS nodes back into sync with this empty instance. For example: $ ./dsreplication initialize-all --hostname ds1.example.com --port 4444 --baseDN dc=example,dc=com --adminUID admin --adminPassword password --no-prompt

See Also

Best practice for configuring sessions in AM (All versions) to reduce the impact on the CTS store

Understanding CTS token types in AM

Core Token Service Guide (CTS)

Related Training

N/A

Related Issue Tracker IDs

N/A


Copyright and Trademarks Copyright © 2021 ForgeRock, all rights reserved.