How To

How do I delete all or some of the tokens in the CTS store in AM (All versions)?

Last updated Jun 26, 2019

The purpose of this article is to provide information on how an administrator can clean up tokens in the Core Token Service (CTS) store. This article covers deleting all tokens in the CTS and deleting just a subset (for example, only Refresh tokens). This information should not be used in lieu of a properly configured/tuned CTS.


1 reader recommends this article

Overview

By default, AM manages expired tokens using its reaper service, although you can use DS to manage token expiration instead if you prefer. You should configure the CTS reaper and then tune your CTS store appropriately to ensure tokens are being removed in an efficient manner. See the following links for further information:

However, even with the reaper running well and pruning expired tokens as expected, there may be occasions when you need to manually delete all tokens in the CTS or delete just a subset. For example, if you have been load testing, you may want to delete all the test tokens that were created before running further tests. Alternatively, you may need to clean up tokens that have built up as a result of an improperly configured/tuned CTS that is not adequate for your environment or specific deployment needs; if this is the case, you should ensure you tune the CTS properly to prevent a build up in future.

Note

Tuning the CTS is outside the scope of ForgeRock support; if you want more tailored advice, consider engaging Deployment Support Services.

This article guides you through each scenario, with different processes depending on the size of your dataset:

Caution

Deleting tokens will end all sessions associated with them.

Example values

These example processes use the following values:

  • A BaseDN of "dc=openam,dc=forgerock,dc=org"
  • The parent DN for tokens is “ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org"
  • A backendID of cts-store for the CTS server database.
  • The LDAP port is 50389
  • The DS admin port is 4444
  • The hostname is: host1.example.com

You should adjust these values as needed for your environment and ensure you include  the --useSsl and --trustAll options if you are using LDAPS.

Deleting and re-creating token parent DN (all tokens, less than 500k entries)

The quickest, easiest and most efficient way to delete all tokens from a small dataset is to delete the parent DN (which holds all the tokens) and re-create it. For large datasets, you should use the LDIF export / import approach described in the following section.

Note

These example steps disable the LDAP/LDAPS connection handler that AM is configured to communicate on to stop updates reaching the CTS instance. You must use an alternative LDAP connector for subsequent LDAP operations. If this is not possible, consider temporarily changing the port or blocking communications at the network level instead.

You can delete all tokens as follows:

  1. Navigate to the bin directory of DS.
  2. Disable the LDAP/LDAPS connector so that AM stops sending traffic to this node providing you have an alternative LDAP connector for subsequent LDAP operations. For example, to disable the LDAP connection handler:
    $ ./dsconfig set-connection-handler-prop --hostname host1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --handler-name LDAP --set enabled:false --trustAll --no-prompt 
  3. Create an LDIF of the ou=famrecords DN:
    $ ./ldapsearch --hostname host1.example.com --port 50389 --bindDn "cn=Directory manager" --bindPassword password --baseDn "ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org" --searchScope one "(objectclass=*)" > parent.ldif
  4. Delete the ou=famrecords DN:
    $ ./ldapdelete --hostname host1.example.com --port 50389 --bindDn "cn=Directory manager" --bindPassword password --deleteSubtree "ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org"
  5. Re-create the ou=famrecords DN using the LDIF file you created in step 3:
    $ ./ldapmodify --hostname host1.example.com --port 50389 --bindDn "cn=Directory manager" --bindPassword password -f parent.ldif
  6. Rebuild all indexes:
    $ ./rebuild-index --hostname host1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --baseDN "dc=openam,dc=forgerock,dc=org" --rebuildAll
  7. Allow replication to bring all other CTS nodes back into sync with this empty instance (replication should sync the deletes and then adds across all replicas). If you don't want to generate a lot of replication traffic with these deletes/adds, you can speed up the process by re-initializing all other nodes from this instance. For example:
    $ ./dsreplication initialize-all --hostname host1.example.com --port 4444 --baseDN dc=openam,dc=forgerock,dc=org --adminUID admin --adminPassword password --no-prompt
  8. Re-enable the LDAP/LDAPS connector to resume AM sending traffic to this node. For example, to enable the LDAP connection handler:
    $ ./dsconfig set-connection-handler-prop --hostname host1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --handler-name LDAP --set enabled:true --trustAll --no-prompt 

LDIF exporting and importing all data excluding tokens (all tokens, more than 500k entries)

This approach removes all tokens and is best suited to a large dataset since you must shut down the CTS instance first. In essence, you create an LDIF file containing all data, but excluding any tokens. You then import this LDIF to override the contents of the database; the import process automatically rebuilds the indexes.

Note

These example steps disable the LDAP/LDAPS connection handler that AM is configured to communicate on to stop updates reaching the CTS instance. You must use an alternative LDAP connector for subsequent LDAP operations. If this is not possible, consider temporarily changing the port or blocking communications at the network level instead.

You can delete all tokens as follows:

  1. Navigate to the bin directory of DS.
  2. Disable the LDAP/LDAPS connector so that AM stops sending traffic to this node providing you have an alternative LDAP connector for subsequent LDAP operations. For example, to disable the LDAP connection handler:
    $ ./dsconfig set-connection-handler-prop --hostname host1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --handler-name LDAP --set enabled:false --trustAll --no-prompt 
  3. Shutdown the CTS server.
  4. Take an LDIF export:
    $ ./export-ldif --backendID cts-store --ldifFile /path/to/export.ldif --excludeFilter "(objectclass=frCoreToken)" --offline
  5. Check the file excludes all tokens.
  6. Import the LDIF:
    $ ./import-ldif --backendID cts-store --ldifFile /path/to/export.ldif --skipFile /tmp/skips.txt --rejectFile /tmp/rejects.txt --offline
  7. Start the CTS server.
  8. Re-initialize all other nodes from this instance to bring all other CTS nodes back into sync with this empty instance. For example:
    $ ./dsreplication initialize-all --hostname host1.example.com --port 4444 --baseDN dc=openam,dc=forgerock,dc=org --adminUID admin --adminPassword password --no-prompt
    
  9. Re-enable the LDAP/LDAPS connector to resume AM sending traffic to this node. For example, to enable the LDAP connection handler:
    $ ./dsconfig set-connection-handler-prop --hostname host1.example.com --port 4444 --bindDN "cn=Directory Manager" --bindPassword password --handler-name LDAP --set enabled:true --trustAll --no-prompt 

Defining a subset of tokens

The following example processes delete all Refresh tokens. This is done by filtering/searching on tokens where coreTokenString10 (token type) is set to refresh_token, for example:

"(coreTokenString10=refresh_token)"

You can amend these processes to look for other token types or other LDAP attributes as needed to define a subset of tokens. See How do I know what LDAP attributes are used by CTS tokens in AM (All versions) and OpenAM 13.x? for information on the attributes and values available.

You can use these attributes together to refine your subset further. For instance, you could include the coreTokenExpirationDate attribute as well to filter tokens before or after a certain timestamp. For example, the following would only affect refresh tokens with an expiration date before 01/01/2019:

"(&(coreTokenString10=refresh_token)(coreTokenExpirationDate<=20190101000000.0Z))"

Deleting using ldapsearch and ldapdelete (subset of tokens, less than 500k entries)

The best way to delete a subset of tokens from the CTS for a small dataset is to use ldapsearch to look for all tokens where coreTokenString10 (token type) is set to refresh_token and then issue a delete operation for each one found. For large datasets, you should use the LDIF export / import approach described in the following section.

Note

This process assumes replication will remain in place while executing the following steps. If this is a concern, you can remove this node from the replication topology and re-sync/re-initialize other CTS instances after deletion.

You can delete all Refresh tokens as follows:

  1. Navigate to the bin directory of DS.
  2. Run the following search and delete command:
    $ ./ldapsearch --hostname host1.example.com --port 50389 --bindDN "cn=Directory manager" --bindPassword password --baseDn "ou=famrecords,ou=openam-session,ou=tokens,dc=openam,dc=forgerock,dc=org" "(coreTokenString10=refresh_token)" 1.1 |grep -v '^$' | cut -c5- | ./ldapdelete --hostname host1.example.com --port 50389 --bindDN "cn=Directory manager" --bindPassword password

LDIF exporting and importing (subset of tokens, more than 500k entries)

This approach removes a subset of tokens and is best suited to a large dataset since you must shut down the CTS instance first. In essence, you create an LDIF file containing all data and tokens, except tokens where coreTokenString10 (token type) is set to refresh_token. You then import this LDIF to override the contents of the database; the import process automatically rebuilds the indexes.

You can delete all Refresh tokens as follows:

  1. Navigate to the bin directory of DS.
  2. Shutdown the CTS server.
  3. Take an LDIF export:
    $ ./export-ldif --backendID cts-store --ldifFile /path/to/export.ldif --excludeFilter "(coreTokenString10=refresh_token)" --offline
  4. Check the file excludes all Refresh Tokens (tokens where coreTokenString10=refresh_token).
  5. Import the LDIF:
    $ ./import-ldif --backendID cts-store --ldifFile /path/to/export.ldif --skipFile /tmp/skips.txt --rejectFile /tmp/rejects.txt --offline
  6. Start the CTS server.
  7. Re-initialize all other nodes from this instance to bring all other CTS nodes back into sync with this empty instance. For example:
    $ ./dsreplication initialize-all --hostname host1.example.com --port 4444 --baseDN dc=openam,dc=forgerock,dc=org --adminUID admin --adminPassword password --no-prompt

See Also

Best practices for configuring sessions in AM (All versions) to reduce the impact on the CTS store

Understanding CTS token types in AM/OpenAM

Reference › ldapsearch — perform LDAP search operations

Reference › ldapdelete — perform LDAP delete operations

Reference › ldapmodify — perform LDAP modify, add, delete, mod DN operations

Administration Guide › Rebuilding Indexes

Administration Guide › Importing and Exporting Data

Administration Guide › Initializing Replicas

Related Training

N/A

Related Issue Tracker IDs

N/A



Copyright and TrademarksCopyright © 2019 ForgeRock, all rights reserved.
Loading...