Get audit and debug logs
Overview
Identity Cloud provides audit and debug logs to help you manage your tenant:
-
Use audit logs to investigate user and system behavior.
-
Use debug logs to investigate any issues that can arise in production.
Identity Cloud stores logs for 30 days. Use the /monitoring/logs
endpoint to access the stored data.
You need to get an API key and secret before you can authenticate to the endpoints. |
Sources
Identity Cloud makes browsing the logs easier by storing them in various sources.
The following knowledge base article lists the sources available and describes their purpose: What logging sources are available in Identity Cloud?
View sources
To view a list of the available sources, use the /monitoring/logs/sources
endpoint.
Example request:
$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
'https://<tenant-env-fqdn>/monitoring/logs/sources'
Example response:
{
"result": [
"am-access",
"am-activity",
"am-authentication",
"am-config",
"am-core",
"am-everything",
"idm-access",
"idm-activity",
"idm-authentication",
"idm-config",
"idm-core",
"idm-everything",
"idm-recon",
"idm-sync"
],
"resultCount": 14,
"pagedResultsCookie": null,
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": 1,
"remainingPagedResults": 0
}
Identity Cloud returns the available sources in the result
array.
Access logs
To access the stored logs for a source, use the /monitoring/logs
endpoint, specifying the source as a parameter.
Example request:
$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
'https://<tenant-env-fqdn>/monitoring/logs'
Example response:
{
"result": [{
"payload": "<payload>",
"timestamp": "<dateTime>",
"type": "application/json",
"source": "am-authentication"
}, {
"…": "…"
}],
"resultCount": "1000",
"pagedResultsCookie": "<pagedResultsCookie>",
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}
Identity Cloud returns the available logs in the result
array.
Results are in JSON format or plaintext, depending on the source you request.
Use the Specify UTC times in ISO 8601 format. For example:
|
Update audit configuration
Sometimes a log source is shown in the available sources in Identity Cloud but returns no results when you query the Identity Cloud logging endpoints. In this case, check the underlying IDM audit configuration to ensure that the corresponding audit topic for the source is enabled.
The following example shows how to enable the recon
event handler so that reconciliation events appear in the audit logs:
-
Get the current the audit configuration.
Example request:
$ curl --GET \ --header 'Authorization: Bearer <access-token>' \ --header 'Content-Type: application/json' \ 'https://<tenant-env-fqdn>/openidm/config/audit' | jq
For more information, refer to IDM REST API reference.
-
Update the audit configuration as needed. The following example enables the reconciliation audit event handler.
Example update:
$ curl \ --request PUT \ --header 'Authorization: Bearer <access-token>' \ --header 'Content-Type: application/json' \ --data-raw ' { "_id": "audit", ... "eventHandlers": [ { "class": "org.forgerock.audit.handlers.json.stdout.JsonStdoutAuditEventHandler", "config": { "elasticsearchCompatible": false, "enabled": true, "name": "json", "topics": [ "access", "activity", "sync", "authentication", "config", "recon" ] } }, { "class": "org.forgerock.openidm.audit.impl.RepositoryAuditEventHandler", "config": { "enabled": false, "name": "repo", "topics": [ "access", "activity", "sync", "authentication", "config", "recon" ] } } ], ... }' \ 'https://<tenant-env-fqdn>/openidm/config/audit'
Tail logs
To tail, or get the latest entries in the stored logs for a source, use the /monitoring/logs/tail
endpoint with the source as a parameter.
The first call to the tail endpoint returns results from the last 15 minutes. Subsequent calls return logs from:
-
Time of last line from previous call to tail
-
Log timestamp
-
Now
You can request results in JSON format or plaintext. One source can have both JSON and plaintext logs; for example, when you request am-everything
.
Example request:
$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-everything' \
'https://<tenant-env-fqdn>/monitoring/logs/tail'
Example response:
{
"result": [{
"payload": "<payload>",
"timestamp": "<dateTime>",
"type": "<type>",
"source": "am-core"
}, {
"…": "…"
}],
"resultCount": "100",
"pagedResultsCookie": "<pagedResultsCookie>",
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}
You can specify multiple sources in a single call. Example request:
$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-access,idm-access,idm-sync,idm-activity' \
'https://<tenant-env-fqdn>/monitoring/logs/tail'
To keep tailing, pass the Example request:
|
View logs for a specific request
All log events for an external request into Identity Cloud are assigned the same unique transaction ID.
The x-forgerock-transactionid
response header holds the transaction ID:
$ curl \
--request POST \
--include \
--header 'Content-Type: application/json' \
--header 'X-OpenAM-Username: bjensen' \
--header 'X-OpenAM-Password: Passw0rd!' \
--header 'Accept-API-Version: resource=2.0, protocol=1.0' \
'https://<tenant-env-fqdn>/am/json/realms/root/realms/alpha/authenticate'
…
x-forgerock-transactionid: <transaction-id>
…
To filter the logs for a specific transaction ID, add the transactionId
parameter to your API request; for example:
$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
--data 'transactionId=<transaction-id>' \
'https://<tenant-env-fqdn>/monitoring/logs'
Example response:
{
"result": [{
"payload": "<payload>",
"timestamp": "<dateTime>",
"type": "application/json",
"source": "am-authentication"
}, {
"…": "…"
}],
"resultCount": "8",
"pagedResultsCookie": null,
"totalPagedResultsPolicy": "NONE",
"totalPagedResults": -1,
"remainingPagedResults": -1
}
Add response fields
The following knowledge base article describes how to add additional fields to an audit request: How do I extend auditing in Identity Cloud to include additional fields?
Rate limiting
To reduce unwanted stresses on the system, Identity Cloud limits the number of requests you can make to the /monitoring/logs
endpoint in a certain timeframe:
-
The page-size limit is 1000 logs per request.
-
The request limit is 60 requests per minute.
-
The theoretical upper rate limit is therefore 60,000 logs per minute.
These limits apply per environment, so your development, staging, and production environments each have their own quota.
The following rate limit notification response headers are sent for each request to the /monitoring/logs
endpoint:
- X-Rate-Limit-Limit
-
The maximum number of requests allowed in the current rate limit window.
- X-Rate-Limit-Remaining
-
The number of requests remaining in the current rate limit window.
- X-Rate-Limit-Reset
-
The time in seconds since Jan. 1, 1970, UTC when the rate limit window resets.