PingOne Advanced Identity Cloud

Get audit and debug logs

PingOne Advanced Identity Cloud provides audit and debug logs to help you manage your tenant:

  • Use audit logs to investigate user and system behavior.

  • Use debug logs to investigate any issues that can arise in production.

Advanced Identity Cloud stores logs for 30 days. Use the /monitoring/logs endpoint to access the stored data.

You need to get an API key and secret before you can authenticate to the endpoints.

Sources

Advanced Identity Cloud makes browsing the logs easier by storing them in various sources.

View sources

To view a list of the available sources, use the /monitoring/logs/sources endpoint.

Example request:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
'https://<tenant-env-fqdn>/monitoring/logs/sources'

Example response:

{
  "result": [
    "am-access",
    "am-activity",
    "am-authentication",
    "am-config",
    "am-core",
    "am-everything",
    "idm-access",
    "idm-activity",
    "idm-authentication",
    "idm-config",
    "idm-core",
    "idm-everything",
    "idm-recon",
    "idm-sync"
  ],
  "resultCount": 14,
  "pagedResultsCookie": null,
  "totalPagedResultsPolicy": "NONE",
  "totalPagedResults": 1,
  "remainingPagedResults": 0
}

Advanced Identity Cloud returns the available sources in the result array.

Source descriptions

The following table lists the available sources and describes their purpose:

Source Type Description

am-access

Audit

Captures all incoming Advanced Identity Cloud access calls as audit events. This includes who, what, when, and the output for every access request.

Audit events:

  • AM-ACCESS-ATTEMPT

  • AM-ACCESS-OUTCOME

Show example 
{
  "payload": {
    "_id": "92c9b6a4-f36f-438a-b1d4-c6e6bd909da6-783933",
    "client": {
      "ip": "198.51.101.0"
    },
    "component": "OAuth",
    "eventName": "AM-ACCESS-ATTEMPT",
    "http": {
      "request": {
        "headers": {
          "content-type": [
            "application/x-www-form-urlencoded"
          ],
          "host": [
            "<tenant-env-fqdn>"
          ],
          "user-agent": [
            "Apache-HttpClient/4.5.13 (Java/11.0.11)"
          ],
          "x-forwarded-for": [
            "198.51.101.0, 203.0.116.0, 192.0.3.255"
          ],
          "x-forwarded-proto": [
            "https"
          ]
        },
        "method": "POST",
        "path": "https://<tenant-env-fqdn>/am/oauth2/realms/root/realms/alpha/access_token",
        "secure": true
      }
    },
    "level": "INFO",
    "realm": "/alpha",
    "request": {
      "detail": {
        "client_id": "RCSClient",
        "grant_type": "client_credentials",
        "scope": "fr:idm:*"
      }
    },
    "source": "audit",
    "timestamp": "<dateTime>",
    "topic": "access",
    "transactionId": "1634116808645-2e50ecbf0df5407a6870-226587/0"
  },
  "timestamp": "<dateTime>",
  "type": "application/json"
}

For information on am-access properties, refer to Access log format.

am-activity

Audit

Captures state changes to objects that were created, updated, or deleted by Advanced Identity Cloud end users. This includes session, user profile, and device profile changes.

Audit events:

  • AM-SELFSERVICE-REGISTRATION-COMPLETED

  • AM-SELFSERVICE-PASSWORDCHANGE-COMPLETED

  • AM-SESSION-CREATED

  • AM-SESSION-IDLE_TIME_OUT

  • AM-SESSION-MAX_TIMED_OUT

  • AM-SESSION-LOGGED_OUT

  • AM-SESSION-DESTROYED

  • AM-SESSION-PROPERTY_CHANGED

  • AM-IDENTITY-CHANGE

  • AM-GROUP-CHANGE

Show example 
{
  "timestamp": "<dateTime>",
  "payload": {
    "_id": "3fc956b8-00a1-4e10-b8aa-72295d003bfb-195032",
    "objectId": "3fc956b8-00a1-4e10-b8aa-72295d003bfb-195023",
    "transactionId": "cf2a721c-9cec-4224-bdd1-3a33e1f8ed56/4",
    "level": "INFO",
    "eventName": "AM-SESSION-CREATED",
    "timestamp": "<dateTime>",
    "component": "Session",
    "source": "audit",
    "topic": "activity",
    "trackingIds": [
      "3fc956b8-00a1-4e10-b8aa-72295d003bfb-195023"
    ],
    "realm": "/",
    "userId": "id=amadmin,ou=user,ou=am-config",
    "runAs": "id=amadmin,ou=user,ou=am-config",
    "operation": "CREATE"
  },
  "type": "application/json"
}

For information on am-activity properties, refer to Activity log format.

am-authentication

Audit

Captures when and how a user authenticated and related audit events.

Advanced Identity Cloud records an authentication audit event for each authentication node and the journey outcome. A node can provide extra data in the standard audit event, which is logged when an authentication node completes.

Audit events:

  • AM-LOGOUT

  • AM-LOGIN-COMPLETED

  • AM-LOGIN-MODULE-COMPLETED

  • AM-NODE-LOGIN-COMPLETED

  • AM-TREE-LOGIN-COMPLETED

Advanced Identity Cloud logs an AM-NODE-LOGIN-COMPLETED audit event each time an authentication node completes.

Show example 
{
  "type": "application/json",
  "timestamp": "<dateTime>",
  "payload": {
    "topic": "authentication",
    "eventName": "AM-NODE-LOGIN-COMPLETED",
    "transactionId": "ad56bedd-7dab-45d1-84d9-505b0b64fd6d/6",
    "principal": [
      "amadmin"
    ],
    "timestamp": "<dateTime>",
    "component": "Authentication",
    "source": "audit",
    "realm": "/",
    "entries": [
      {
        "info": {
          "authLevel": "0",
          "displayName": "Page Node",
          "nodeId": "83a9d86e-d6f5-11ea-87d0-0242ac130003",
          "nodeOutcome": "outcome",
          "treeName": "FRLogin",
          "nodeType": "PageNode"
        }
      }
    ],
    "level": "INFO",
    "trackingIds": [
      "3fc956b8-00a1-4e10-b8aa-72295d003bfb-184020"
    ],
    "_id": "3fc956b8-00a1-4e10-b8aa-72295d003bfb-184022"
  }
}

For information on am-authentication properties, refer to Authentication log format.

am-config

Audit

Captures access management configuration changes for Advanced Identity Cloud with a timestamp and by whom.

Configuration changes can only be performed in development environments, so these logs are empty in staging and production environments.

Audit events:

  • AM-CONFIG-CHANGE

Show example 
{
  "payload": {
    "_id": "92c9b6a4-f36f-438a-b1d4-c6e6bd909da6-822860",
    "eventName": "AM-CONFIG-CHANGE",
    "level": "INFO",
    "objectId": "ou=Office365,ou=dashboardApp,ou=default,ou=GlobalConfig,ou=1.0,ou=dashboardService,ou=services,ou=am-config",
    "operation": "CREATE",
    "runAs": "id=bd220328-9762-458b-b05a-982ac3c7fc54,ou=user,ou=am-config",
    "source": "audit",
    "timestamp": "<dateTime>",
    "topic": "config",
    "trackingIds": [
      "92c9b6a4-f36f-438a-b1d4-c6e6bd909da6-821644"
    ],
    "transactionId": "1634122041174-2e50ecbf0df5407a6870-229391/0",
    "userId": "id=bd220328-9762-458b-b05a-982ac3c7fc54,ou=user,ou=am-config"
  },
  "timestamp": "<dateTime>",
  "type": "application/json"
}

For information on am-config properties, refer to Config log format.

am-core

Debug

Captures access management debug logs for Advanced Identity Cloud. Use am-core when debugging anything in access management without capturing audit events. am-core also captures logging in authentication scripts.

Development and sandbox environments provide DEBUG level logs, with logs in several areas tuned to INFO or WARNING.

To reduce log volumes, staging and production environments only provide WARNING level logs and above.

To troubleshoot and view the latest entries in the stored logs, you can tail am-core source. Refer to Tail logs below.

am-everything

Audit, Debug

Captures all access management audit and debug logs for Advanced Identity Cloud.

This includes all the logs captured in am-access, am-activity, am-authentication, am-config, and am-core.

idm-access

Audit

Captures messages for the identity management REST endpoints and the invocation of scheduled tasks. This is the who, what, and output for every identity management access request in Advanced Identity Cloud.

Audit events:

  • access

Show example 
{
  "payload": {
    "_id": "32c02w2f-bafe-4bdf-a8e1-1ce94813c46b-123717",
    "client": {
      "ip": "198.51.101.0",
      "port": 60572
    },
    "eventName": "access",
    "http": {
      "request": {
        "headers": {
          "host": [
            "<tenant-env-fqdn>:443"
          ],
          "user-agent": [
            "Blackbox Exporter/0.25.0"
          ],
          "x-forwarded-for": [
            "34.102.86.57, 34.97.113.137, 120.211.3.20"
          ],
          "x-forwarded-proto": [
            "https"
          ],
          "x-real-ip": [
            "34.102.86.57"
          ]
        },
        "method": "GET",
        "path": "https://<tenant-env-fqdn>/openidm/info/ping",
        "secure": true
      }
    },
    "level": "INFO",
    "request": {
      "operation": "READ",
      "protocol": "CREST"
    },
    "response": {
      "elapsedTime": 10,
      "elapsedTimeUnits": "MILLISECONDS",
      "status": "SUCCESSFUL",
      "statusCode": "200"
    },
    "roles": [
      "internal/role/openidm-reg"
    ],
    "server": {
      "ip": "10.68.2.21",
      "port": 8080
    },
    "source": "audit",
    "timestamp": "dateTime",
    "topic": "access",
    "transactionId": "6b3a1cbb-523d-48ae-bd11-1aca4b65c294/0",
    "userId": "anonymous"
  },
  "source": "idm-access",
  "timestamp": "<dateTime>",
  "type": "application/json"
}

For information on idm-access properties, refer to Access event topic properties.

idm-activity

Audit

Captures operations on internal (managed) and external (system) objects in Advanced Identity Cloud. idm-activity logs the changes to identity content, such as adding or updating users and changing passwords.

Audit events:

  • activity

Show example 
{
  "timestamp": "<dateTime>",
  "type": "application/json",
  "payload": {
    "_id": "eebf2abb-e4f1-428f-8fbb-8c18ed3f9559-218925",
    "transactionId": "1630077288251-f5190abcb8c2d0d42c31-136380/0",
    "message": "",
    "timestamp": "<dateTime>",
    "eventName": "activity",
    "userId": "bd220328-9762-458b-b05a-982ac3c7fc54",
    "revision": "00000000478fd92b",
    "operation": "PATCH",
    "changedFields": [],
    "runAs": "bd220328-9762-458b-b05a-982ac3c7fc54",
    "passwordChanged": true,
    "status": "SUCCESS",
    "objectId": "managed/alpha_user/e70c4476-1305-408a-9246-ac76c64ba039"
  }
}

For information on idm-activity properties, refer to Activity event topic properties.

idm-authentication

Audit

Captures the results when authenticating to an /openidm endpoint to complete certain actions on an object.

If an authentication session already exists in access management, authentication to identity management is not required. In this instance, the authentication logs would appear for am-authentication, with identity management logs in idm-access and idm-activity.

Audit events:

  • authentication

For information on idm-authentication properties, refer to Authentication event topic properties.

idm-config

Audit

Captures identity management configuration changes for Advanced Identity Cloud with a timestamp and by whom.

Configuration changes can only be performed in development environments, so these logs are empty in staging and production environments.

Audit events:

  • CONFIG

Show example 
{
  "payload": {
    "_id": "f6a3a7b2-aaf3-426d-a998-a970f84bdf4b-1519486",
    "changedFields": [
      "/mappings"
    ],
    "eventName": "CONFIG",
    "objectId": "sync",
    "operation": "UPDATE",
    "revision": null,
    "runAs": "bd220328-9762-458b-b05a-982ac3c7fc54",
    "timestamp": "<dateTime>",
    "transactionId": "1634054726312-2e50ecbf0df5407a6870-202437/0",
    "userId": "bd220328-9762-458b-b05a-982ac3c7fc54"
  }
}

For information on idm-config properties, refer to Configuration event topic properties.

idm-core

Debug

Captures identity management debug logs for Advanced Identity Cloud. Use idm-core when debugging anything in identity management without capturing audit events.

Development and sandbox environments provide FINE level logs, with logs in several areas tuned to INFO, WARNING and SEVERE.

To reduce log volumes, staging and production environments only provide INFO and WARNING level logs and above.

To troubleshoot and view the latest entries in the stored logs, you can tail idm-core source. Refer to Tail logs below.

idm-everything

Audit, Debug

Captures identity management audit and debug logs for Advanced Identity Cloud.

This includes all the logs captured in idm-access, idm-activity, idm-authentication, idm-config, idm-recon, idm-sync, and idm-core.

idm-recon

Audit

Captures reconciliation events for Advanced Identity Cloud.

The corresponding audit topic for idm-recon is disabled by default in Advanced Identity Cloud. For reconciliation events to appear in the audit logs, you must enable the recon event handler.

For information on idm-recon event properties, refer to Reconciliation event topic properties.

idm-sync

Audit

Captures any changes to an object resulting in automatic sync (live sync and implicit sync) when a repository is mapped to Advanced Identity Cloud. This includes situations and the actions taken on each object, by account. The idm-activity log contains additional details about each action.

For information on idm-sync event properties, refer to Synchronization event topic properties.

Retrieve log entries

To retrieve the stored log entries for a source, use the /monitoring/logs endpoint, specifying the source as a parameter.

Example request:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
'https://<tenant-env-fqdn>/monitoring/logs'

Example response:

{
  "result": [{
    "payload": "<payload>",
    "timestamp": "<dateTime>",
    "type": "application/json",
    "source": "am-authentication"
  }, {
    "...": "..."
  }],
  "resultCount": "1000",
  "pagedResultsCookie": "<pagedResultsCookie>",
  "totalPagedResultsPolicy": "NONE",
  "totalPagedResults": -1,
  "remainingPagedResults": -1
}

Advanced Identity Cloud returns the available log entries in the result array. Results are in JSON format or plaintext, depending on the source you request.

To reduce the size of the output, log query results are by default restricted to the last 24 hours, unless you add beginTime and/or endTime query parameters. Refer to Get log results for a time period.

Get log results for a time period

Use the beginTime and endTime query parameters to return entries created between two ISO 8601 formatted times.

Example request:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
--data 'beginTime=2023-03-01T12:45:00Z' \
--data 'endTime=2023-03-01T12:50:00Z' \
'https://<tenant-env-fqdn>/monitoring/logs'

The beginTime and endTime query parameters are subject to these rules:

  1. If endTime is not specified, it defaults to the current time.

  2. If startTime is not specified, it defaults to 24 hours before endTime.

  3. If startTime is specified, it must be 24 hours or less before endTime.

Tail logs

To tail, or get the latest entries in the stored logs for a source, use the /monitoring/logs/tail endpoint with the source as a parameter.

The first call to the tail endpoint returns log entries from the last 15 seconds. Subsequent calls return log entries in a range that starts from the last returned log entry in the previous result (inclusive) and ends with the latest log entry but one. If calls to the tail endpoint are not frequent enough to match the rate at which the log entries are produced, the result may not include all available log entries.

The format of the log results depends on the source or sources specified in your request. Some sources return only JSON formatted log entries and some sources return only plaintext log entries. Some sources, such as am-everything, can return log entries in both formats.

Example request:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-everything' \
'https://<tenant-env-fqdn>/monitoring/logs/tail'

Example response:

{
  "result": [{
    "payload": "<payload>",
    "timestamp": "<dateTime>",
    "type": "<type>",
    "source": "am-core"
  }, {
    "...": "..."
  }],
  "resultCount": "100",
  "pagedResultsCookie": "<pagedResultsCookie>",
  "totalPagedResultsPolicy": "NONE",
  "totalPagedResults": -1,
  "remainingPagedResults": -1
}

You can specify multiple sources in a single call. Example request:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-access,idm-access,idm-sync,idm-activity' \
'https://<tenant-env-fqdn>/monitoring/logs/tail'

To keep tailing, pass the <pagedResultsCookie> from the previous response in your request. This retrieves new records since your request.

Example request:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-access,idm-access,idm-sync,idm-activity' \
--data '_pagedResultsCookie=<pagedResultsCookie>' \
'https://<tenant-env-fqdn>/monitoring/logs/tail'

View logs for a specific request

All log events for an external request into Advanced Identity Cloud are assigned the same unique transaction ID. The x-forgerock-transactionid response header holds the transaction ID:

$ curl \
--request POST \
--include \
--header 'Content-Type: application/json' \
--header 'X-OpenAM-Username: bjensen' \
--header 'X-OpenAM-Password: Passw0rd!' \
--header 'Accept-API-Version: resource=2.0, protocol=1.0' \
'https://<tenant-env-fqdn>/am/json/realms/root/realms/alpha/authenticate'
...
x-forgerock-transactionid: <transaction-id>
...

To filter the logs for a specific transaction ID, add the transactionId parameter to your API request; for example:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
--data 'transactionId=<transaction-id>' \
'https://<tenant-env-fqdn>/monitoring/logs'

Example response:

{
  "result": [{
    "payload": "<payload>",
    "timestamp": "<dateTime>",
    "type": "application/json",
    "source": "am-authentication"
  }, {
    "...": "..."
  }],
  "resultCount": "8",
  "pagedResultsCookie": null,
  "totalPagedResultsPolicy": "NONE",
  "totalPagedResults": -1,
  "remainingPagedResults": -1
}

Filter log results

Use the _queryFilter parameter to filter log results on any field or combination of fields in a payload. You can add the parameter to the /monitoring/logs and /monitoring/logs/tail endpoints.

The benefits of the _queryFilter parameter are:

  • Lets you iteratively refine queries to remove extraneous results and find the specific log entries you are interested in. This is useful when searching logs to debug a production issue.

    Use the /monitoring/logs endpoint for iterative searching as the /monitoring/logs/tail endpoint only returns results from the last 15 seconds.

  • Lets you tune queries to reduce Advanced Identity Cloud log volume, making integration with external log tools such as Splunk or Elastic Stack more efficient and potentially reducing storage costs.

The _queryFilter parameter takes a URL-encoded filter expression:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=<source>' \
--data-urlencode '_queryFilter=<filter-expression>' \
'https://<tenant-env-fqdn>/monitoring/logs'

To understand how to construct a filter expression, refer to the filter expression rules for _queryFilter. Here are some basic examples:

Example filter expression Description

/payload co "WARNING"

Search plaintext results for a particular string.

/payload/client/ip co "10.104.1.5"

Search for JSON results containing a particular client IP address

/payload/eventName co "AM-ACCESS-ATTEMPT"

Search for JSON results containing a particular event name

/payload/timestamp eq "2023-05-16T08:21:34.632Z"

Search for JSON results with a particular timestamp

/payload/timestamp sw "2023-05-14T16:34:34"

Search for JSON results with a timestamp that starts with a particular datetime

/payload/client/ip co "10.104.1.5" and /payload/level eq "ERROR"

Search for JSON results containing a particular client IP address and also containing a particular debug level

/payload/entries/info/nodeType pr

Search for JSON results where an authentication node type is present

Filter array items in log results

To filter on array items, do not include an array index in your filter expression.

For example, to search for JSON results where the authentication node type is ScriptedDecisionNode:

  • Wrong: /payload/entries/0/info/nodeType eq "ScriptedDecisionNode"

  • Right: /payload/entries/info/nodeType eq "ScriptedDecisionNode"

where a log entry for an authentication node looks like this:

    {
      "payload": {
        "_id": "7ae37a4b-f22b-4c5e-8621-2130d5bc603c-9310858",
        "component": "Authentication",
        "entries": [
          {
            "info": {
              "authLevel": "0",
              "displayName": "Using Invite?",
              "nodeId": "15edd2f7-22f1-4f32-bf0a-8ca3f98af850",
              "nodeOutcome": "False",
              "nodeType": "ScriptedDecisionNode",
              "treeName": "Login"
            }
          }
        ],
        "eventName": "AM-NODE-LOGIN-COMPLETED",
        ...
    }

Filter log results between two dates

To filter log results between two dates, use the beginTime and endTime query parameters with ISO 8601 datetime values:

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=<source>' \
--data 'beginTime=<begin-datetime>' \
--data 'endTime=<end-datetime>' \
--data-urlencode '_queryFilter=<filter-expression>' \
'https://<tenant-env-fqdn>/monitoring/logs'

For example, to filter log results between two specific dates for a specific user :

$ curl --get \
--header 'x-api-key: <api-key>' \
--header 'x-api-secret: <api-secret>' \
--data 'source=am-authentication' \
--data 'beginTime=2023-05-24T12:40:00.00Z' \
--data 'endTime=2023-05-24T12:45:00.00Z' \
--data-urlencode '_queryFilter=/payload/principal eq "user.name@example.com"' \
'https://<tenant-env-fqdn>/monitoring/logs'

Add response fields

Authentication events

You can use a script to add extra information to log entries for authentication events. Refer to Add information to authentication audit log entries.

Identity object events

You can configure audit log results to include additional identity object events. For example, you may want to log the before and after values of specific activities, such as changes to a user’s last name or email address.

To include additional identity object event fields, add them to the includeIf property in the audit configuration. Make these changes in your development environment and then promote them.

By default, Advanced Identity Cloud audits identity object event fields that are safe to log. When adding audit event fields, be mindful of the type of information that you intend to expose in the logs. For example, you may need to keep personally identifiable information (PII) out of the logs.

Add identity object event fields to audit logging

  1. Get the current audit configuration.

    Example request:

    $ curl --GET \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \
'https://<tenant-env-fqdn>/openidm/config/audit'

    For more information, refer to IDM REST API reference.

    Show response 
    {
  "_id": "audit",
  "auditServiceConfig": {
    "availableAuditEventHandlers": [
      "org.forgerock.audit.handlers.csv.CsvAuditEventHandler",
      "org.forgerock.audit.handlers.elasticsearch.ElasticsearchAuditEventHandler",
      "org.forgerock.audit.handlers.jms.JmsAuditEventHandler",
      "org.forgerock.audit.handlers.json.JsonAuditEventHandler",
      "org.forgerock.audit.handlers.json.stdout.JsonStdoutAuditEventHandler",
      "org.forgerock.openidm.audit.impl.RepositoryAuditEventHandler",
      "org.forgerock.openidm.audit.impl.RouterAuditEventHandler",
      "org.forgerock.audit.handlers.splunk.SplunkAuditEventHandler",
      "org.forgerock.audit.handlers.syslog.SyslogAuditEventHandler"
    ],
    "caseInsensitiveFields": [
      "/access/http/request/headers",
      "/access/http/response/headers"
    ],
    "filterPolicies": {
      "value": {
        "excludeIf": [
          "/access/http/request/cookies/&{com.iplanet.am.cookie.name}",
          "/access/http/request/cookies/session-jwt",
          "/access/http/request/headers/&{com.sun.identity.auth.cookieName}",
          "/access/http/request/headers/&{com.iplanet.am.cookie.name}",
          "/access/http/request/headers/accept-encoding",
          "/access/http/request/headers/accept-language",
          "/access/http/request/headers/Authorization",
          "/access/http/request/headers/cache-control",
          "/access/http/request/headers/connection",
          "/access/http/request/headers/content-length",
          "/access/http/request/headers/content-type",
          "/access/http/request/headers/proxy-authorization",
          "/access/http/request/headers/X-OpenAM-Password",
          "/access/http/request/headers/X-OpenIDM-Password",
          "/access/http/request/queryParameters/access_token",
          "/access/http/request/queryParameters/IDToken1",
          "/access/http/request/queryParameters/id_token_hint",
          "/access/http/request/queryParameters/Login.Token1",
          "/access/http/request/queryParameters/redirect_uri",
          "/access/http/request/queryParameters/requester",
          "/access/http/request/queryParameters/sessionUpgradeSSOTokenId",
          "/access/http/request/queryParameters/tokenId",
          "/access/http/response/headers/Authorization",
          "/access/http/response/headers/Set-Cookie",
          "/access/http/response/headers/X-OpenIDM-Password"
        ],
        "includeIf": []
      }
    },
    "handlerForQueries": "json"
  },
  "eventHandlers": [
    {
      "class": "org.forgerock.audit.handlers.json.JsonAuditEventHandler",
      "config": {
        "buffering": {
          "maxSize": 100000,
          "writeInterval": "100 millis"
        },
        "logDirectory": "&{idm.data.dir}/audit",
        "name": "json",
        "topics": [
          "access",
          "activity",
          "sync",
          "authentication",
          "config"
        ]
      }
    },
    {
      "class": "org.forgerock.openidm.audit.impl.RepositoryAuditEventHandler",
      "config": {
        "enabled": false,
        "name": "repo",
        "topics": [
          "access",
          "activity",
          "sync",
          "authentication",
          "config"
        ]
      }
    }
  ],
  "eventTopics": {
    "activity": {
      "filter": {
        "actions": [
          "create",
          "update",
          "delete",
          "patch",
          "action"
        ]
      },
      "passwordFields": [
        "password"
      ],
      "watchedFields": []
    },
    "config": {
      "filter": {
        "actions": [
          "create",
          "update",
          "delete",
          "patch",
          "action"
        ]
      }
    }
  },
  "exceptionFormatter": {
    "file": "bin/defaults/script/audit/stacktraceFormatter.js",
    "type": "text/javascript"
  }
}

  2. Make a backup of the audit configuration.

  3. Update the includeIf property (under filterPolicies) in the audit configuration to include the fields you want to add.

    The following example updates the audit configuration to include before and after values of a user’s last name and email address:

    $ curl \
--request PUT \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \
--data-raw '
{
  "_id": "audit",
  ...
   "filterPolicies": {
     "value": {
       "excludeIf": [
          "/access/http/request/cookies/&{com.iplanet.am.cookie.name}",
          "/access/http/request/cookies/session-jwt",
        ...
        ],
        "includeIf": [
           "/activity/before/sn", (1)
           "/activity/after/sn", (2)
           "/activity/before/mail", (3)
           "/activity/after/mail" (4)
        ]
      }
   },
  ...
}' \
'https://<tenant-env-fqdn>/openidm/config/audit'

    Fields added:

    1 Logs the user’s last name before change.
    2 Logs the user’s last name after change.
    3 Logs the user’s email address before change.
    4 Logs user’s email address after change.

Once updated, idm-activity and idm-everything audit logs will include the fields you have added.

For example, the following entry in a sample idm-activity log shows before and after values for changes to a user’s last name and email address from "Brown" to "Granger":

{
    "payload": {
        "message": "",
        "runAs": "bd220328-9762-458b-b05a-982ac3c7fc54",
        "transactionId": "1630683558570-abec9e9304c84ad368ba-28676/0",
        "before": {
            "sn": "Brown",
            "mail": "jbrown@example.com"
        },
        "operation": "PATCH",
        "passwordChanged": false,
        "_id": "52f7cea0-285d-4ef6-bda3-83256dda71c5-1300250",
        "revision": "00000000412cae36",
        "eventName": "activity",
        "userId": "bd220328-9762-458b-b05a-982ac3c7fc54",
        "status": "SUCCESS",
        "objectId": "managed/alpha_user/ce7492dc-8759-47b3-b4ee-eda8d4de4ab1",
        "timestamp": "2023-09-03T15:39:42.862Z",
        "changedFields": [],
        "after": {
            "sn": "Granger",
            "mail": "jgranger@example.com"
        }
    "type": "application/json",
    "timestamp": "<date-time>"
}

You can also exclude fields from audit logging by adding them to the excludeIf property in the audit configuration.

For example, to prevent audit logs from showing target object attributes for synchronization and reconciliation events, add the following entries to the excludeIf property in the audit configuration:

"/sync/targetObject/<attribute-name>",
"/recon/targetObject/<attribute-name>"

Rate limiting

Logs endpoint

To reduce unwanted stresses on the system, Advanced Identity Cloud limits the number of requests you can make to the /monitoring/logs endpoint in a certain timeframe:

  • The page-size limit is 1000 logs per request.

    Ping Identity recommends you do not override the page-size limit with a greater value as it could increase request throttling and reduce the overall number of logs you can request per minute.

  • The request limit is 60 requests per minute.

  • The theoretical upper rate limit is therefore 60,000 logs per minute.

These limits apply per environment, so your development, staging, and production environments each have their own quota.

The following rate limit notification response headers are sent for each request to the /monitoring/logs endpoint:

X-RateLimit-Limit

The maximum number of requests allowed in the current rate limit window.

X-RateLimit-Remaining

The number of requests remaining in the current rate limit window.

X-RateLimit-Reset

The time in seconds since Jan. 1, 1970, UTC when the rate limit window resets.

Logs tail endpoint

The /monitoring/logs/tail endpoint has the same limits and response headers as the /monitoring/logs endpoint described above. However, the endpoint also has a limit of 20,000 lines per request, which supersedes the page-size limit of 1000 logs per request.

Because calls to the /monitoring/logs/tail endpoint do not always fetch all logs, use this endpoint for debugging only. Use the /monitoring/logs endpoint when you need to fetch all logs.

Troubleshooting

Update audit configuration

Sometimes a log source is shown in the available sources in Advanced Identity Cloud but returns no results when you query the Advanced Identity Cloud logging endpoints. In this case, check the underlying IDM audit configuration to ensure that the corresponding audit topic for the source is enabled.

The following example shows how to enable the recon event handler so that reconciliation events appear in the audit logs:

  1. Get the current audit configuration.

    Example request:

    $ curl --GET \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \
'https://<tenant-env-fqdn>/openidm/config/audit'

    For more information, refer to IDM REST API reference.

  2. Update the audit configuration as needed. The following example enables the reconciliation audit event handler.

    Example update:

    $ curl \
--request PUT \
--header 'Authorization: Bearer <access-token>' \
--header 'Content-Type: application/json' \
--data-raw '
{
  "_id": "audit",
...
  "eventHandlers": [
    {
      "class": "org.forgerock.audit.handlers.json.stdout.JsonStdoutAuditEventHandler",
      "config": {
        "elasticsearchCompatible": false,
        "enabled": true,
        "name": "json",
        "topics": [
          "access",
          "activity",
          "sync",
          "authentication",
          "config",
          "recon"
        ]
      }
    },
    {
      "class": "org.forgerock.openidm.audit.impl.RepositoryAuditEventHandler",
      "config": {
        "enabled": false,
        "name": "repo",
        "topics": [
          "access",
          "activity",
          "sync",
          "authentication",
          "config",
          "recon"
        ]
      }
    }
  ],
...
}' \
'https://<tenant-env-fqdn>/openidm/config/audit'

Include large log entries in filter log results

Some Advanced Identity Cloud log output is too large to be stored as a single log entry, so is stored across two log entries instead. When this happens, any log output in JSON format is stored as two plaintext log entries rather than a single JSON log entry. Consequently, any filter expression that filters on a specific JSON field will not find any of these plaintext log entries.

To work around this, you can combine a specific field filter with a plaintext filter. For example, if you were searching for log results containing a particular transaction ID using the filter expression:

/payload/transactionId co "<transaction-id>"

you could add a plaintext filter as follows:

/payload/transactionId co "<transaction-id>" or /payload co "<transaction-id>"

to include both JSON and plaintext log entries in the log results.

Copyright © 2010-2024 ForgeRock, all rights reserved.